Note: This is test shard 3 of 3.
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from CatalogManagerConfigurations/MasterStressTest
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/2
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260926 15:02:19.649282 8746 test_util.cc:268] Using random seed: 512900310
2026-09-26T15:02:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:02:19Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskLGH8i2/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskLGH8i2/thirdparty/src/hadoop-3.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/security.properties
2026-09-26 15:02:26: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskLGH8i2/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskLGH8i2/thirdparty/src/hadoop-3.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260926 15:02:33.790966 8746 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu
/tmp/dist-test-taskLGH8i2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.138.190:44281
--webserver_interface=127.8.138.190
--webserver_port=0
--builtin_ntp_servers=127.8.138.148:33835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473
--hive_metastore_uris=thrift://127.0.0.1:40047
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260926 15:02:34.127647 8987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:34.128073 8987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:34.128238 8987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:34.135895 8987 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:02:34.136060 8987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:34.136160 8987 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:02:34.136236 8987 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:02:34.145712 8987 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.138.148:33835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:40047
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.138.190:44281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.138.190
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:34.147189 8987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:34.149137 8987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:34.163256 8993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:34.163259 8995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:34.163336 8987 server_base.cc:1034] running on GCE node
W20260926 15:02:34.163496 8992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:34.164706 8987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:34.166254 8987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:34.167518 8987 hybrid_clock.cc:648] HybridClock initialized: now 1790434954167465 us; error 32 us; skew 500 ppm
I20260926 15:02:34.167961 8987 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:02:34.170950 8987 webserver.cc:460] Webserver started at http://127.8.138.190:37041/ using document root <none> and password file <none>
I20260926 15:02:34.171741 8987 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:34.171963 8987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:34.172312 8987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:34.175123 8987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/data/instance:
uuid: "6315fb5099e1420b95b8d83b3e19375c"
format_stamp: "Formatted at 2026-09-26 15:02:34 on dist-test-slave-7qf4"
I20260926 15:02:34.176101 8987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/wal/instance:
uuid: "6315fb5099e1420b95b8d83b3e19375c"
format_stamp: "Formatted at 2026-09-26 15:02:34 on dist-test-slave-7qf4"
I20260926 15:02:34.181752 8987 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 15:02:34.185952 9001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:34.187754 8987 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 15:02:34.188012 8987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/data,/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/wal
uuid: "6315fb5099e1420b95b8d83b3e19375c"
format_stamp: "Formatted at 2026-09-26 15:02:34 on dist-test-slave-7qf4"
I20260926 15:02:34.188221 8987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:34.204896 8987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:34.205765 8987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:34.206085 8987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:34.227437 8987 rpc_server.cc:307] RPC server started. Bound to: 127.8.138.190:44281
I20260926 15:02:34.227465 9052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.138.190:44281 every 8 connection(s)
I20260926 15:02:34.228971 8987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/data/info.pb
I20260926 15:02:34.233151 9053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:34.240219 8746 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu as pid 8987
I20260926 15:02:34.240525 8746 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/wal/instance
I20260926 15:02:34.240237 9053 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } has no permanent_uuid. Determining permanent_uuid...
I20260926 15:02:34.243616 8746 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu
/tmp/dist-test-taskLGH8i2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.138.189:37105
--webserver_interface=127.8.138.189
--webserver_port=0
--builtin_ntp_servers=127.8.138.148:33835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473
--hive_metastore_uris=thrift://127.0.0.1:40047
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260926 15:02:34.266075 9053 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } has no permanent_uuid. Determining permanent_uuid...
W20260926 15:02:34.267621 9005 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
W20260926 15:02:34.270097 9053 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:02:34.319536 9053 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } attempt: 1
W20260926 15:02:34.321844 9053 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:02:34.390661 9053 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } attempt: 2
W20260926 15:02:34.392962 9053 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:02:34.484287 9053 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } attempt: 3
W20260926 15:02:34.486511 9053 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
W20260926 15:02:34.599805 9056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:34.600150 9056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:34.600292 9056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:34.607599 9056 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:02:34.607717 9056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:34.607789 9056 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:02:34.607892 9056 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:02:34.617542 9056 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.138.148:33835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:40047
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.138.189:37105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.138.189
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:34.618892 9056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:34.620661 9056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 15:02:34.629829 9053 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } attempt: 4
W20260926 15:02:34.632854 9053 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
W20260926 15:02:34.633457 9062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:34.633489 9063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:34.634099 9065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:34.635183 9056 server_base.cc:1034] running on GCE node
I20260926 15:02:34.636061 9056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:34.637611 9056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:34.638859 9056 hybrid_clock.cc:648] HybridClock initialized: now 1790434954638790 us; error 34 us; skew 500 ppm
I20260926 15:02:34.639292 9056 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:02:34.642000 9056 webserver.cc:460] Webserver started at http://127.8.138.189:40173/ using document root <none> and password file <none>
I20260926 15:02:34.642792 9056 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:34.642933 9056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:34.643301 9056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:34.646245 9056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/data/instance:
uuid: "be9958f2085242f7ab82ae933ceb1c42"
format_stamp: "Formatted at 2026-09-26 15:02:34 on dist-test-slave-7qf4"
I20260926 15:02:34.647089 9056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/wal/instance:
uuid: "be9958f2085242f7ab82ae933ceb1c42"
format_stamp: "Formatted at 2026-09-26 15:02:34 on dist-test-slave-7qf4"
I20260926 15:02:34.652765 9056 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260926 15:02:34.656576 9071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:34.658116 9056 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:02:34.658322 9056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/data,/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/wal
uuid: "be9958f2085242f7ab82ae933ceb1c42"
format_stamp: "Formatted at 2026-09-26 15:02:34 on dist-test-slave-7qf4"
I20260926 15:02:34.658648 9056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:34.674193 9056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:34.675128 9056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:34.675480 9056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:34.697598 9056 rpc_server.cc:307] RPC server started. Bound to: 127.8.138.189:37105
I20260926 15:02:34.697641 9122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.138.189:37105 every 8 connection(s)
I20260926 15:02:34.699002 9056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/data/info.pb
I20260926 15:02:34.702999 9123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:34.706540 8746 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu as pid 9056
I20260926 15:02:34.706749 8746 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/wal/instance
I20260926 15:02:34.707944 8746 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu
/tmp/dist-test-taskLGH8i2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.138.188:45473
--webserver_interface=127.8.138.188
--webserver_port=0
--builtin_ntp_servers=127.8.138.148:33835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473
--hive_metastore_uris=thrift://127.0.0.1:40047
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260926 15:02:34.710500 9123 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } has no permanent_uuid. Determining permanent_uuid...
I20260926 15:02:34.730660 9123 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } has no permanent_uuid. Determining permanent_uuid...
I20260926 15:02:34.742879 9123 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } has no permanent_uuid. Determining permanent_uuid...
W20260926 15:02:34.744541 9074 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
W20260926 15:02:34.748023 9123 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.138.188:45473: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
I20260926 15:02:34.797454 9123 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } attempt: 1
W20260926 15:02:34.799639 9123 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.138.188:45473: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
I20260926 15:02:34.867938 9123 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } attempt: 2
W20260926 15:02:34.870201 9123 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.138.188:45473: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
I20260926 15:02:34.932170 9053 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } attempt: 5
I20260926 15:02:34.940085 9053 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } has no permanent_uuid. Determining permanent_uuid...
W20260926 15:02:34.942015 9053 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.138.188:45473: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
I20260926 15:02:34.961443 9123 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } attempt: 3
W20260926 15:02:34.963629 9123 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.138.188:45473: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
I20260926 15:02:34.993254 9053 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } attempt: 1
W20260926 15:02:34.995215 9053 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.138.188:45473: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
I20260926 15:02:35.063473 9053 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } attempt: 2
W20260926 15:02:35.065464 9053 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.138.188:45473: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
W20260926 15:02:35.074149 9124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:35.074447 9124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:35.074522 9124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:35.082203 9124 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:02:35.082324 9124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:35.082365 9124 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:02:35.082412 9124 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:02:35.092097 9124 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.138.148:33835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:40047
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.138.188:45473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.138.188
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:35.093691 9124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:35.095485 9124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 15:02:35.106886 9123 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } attempt: 4
W20260926 15:02:35.109308 9123 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.138.188:45473: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
W20260926 15:02:35.109195 9133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:35.109339 9132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:35.109993 9135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:35.113549 9124 server_base.cc:1034] running on GCE node
I20260926 15:02:35.114514 9124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:35.116107 9124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:35.117372 9124 hybrid_clock.cc:648] HybridClock initialized: now 1790434955117306 us; error 46 us; skew 500 ppm
I20260926 15:02:35.117727 9124 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:02:35.120625 9124 webserver.cc:460] Webserver started at http://127.8.138.188:38457/ using document root <none> and password file <none>
I20260926 15:02:35.121380 9124 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:35.121537 9124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:35.121873 9124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:35.126581 9124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data/instance:
uuid: "320cd09624624312bf1bcc0856c2874f"
format_stamp: "Formatted at 2026-09-26 15:02:35 on dist-test-slave-7qf4"
I20260926 15:02:35.127427 9124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/wal/instance:
uuid: "320cd09624624312bf1bcc0856c2874f"
format_stamp: "Formatted at 2026-09-26 15:02:35 on dist-test-slave-7qf4"
I20260926 15:02:35.132997 9124 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260926 15:02:35.136974 9141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:35.138497 9124 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260926 15:02:35.138651 9124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data,/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/wal
uuid: "320cd09624624312bf1bcc0856c2874f"
format_stamp: "Formatted at 2026-09-26 15:02:35 on dist-test-slave-7qf4"
I20260926 15:02:35.138824 9124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:35.159024 9124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:35.159937 9124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:35.160205 9124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:35.171682 9053 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } attempt: 3
W20260926 15:02:35.173931 9053 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.138.188:45473: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
I20260926 15:02:35.180997 9124 rpc_server.cc:307] RPC server started. Bound to: 127.8.138.188:45473
I20260926 15:02:35.181017 9192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.138.188:45473 every 8 connection(s)
I20260926 15:02:35.182365 9124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data/info.pb
I20260926 15:02:35.186228 9193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:35.187390 8746 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu as pid 9124
I20260926 15:02:35.187587 8746 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/wal/instance
I20260926 15:02:35.194706 9193 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } has no permanent_uuid. Determining permanent_uuid...
I20260926 15:02:35.223829 9193 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } has no permanent_uuid. Determining permanent_uuid...
I20260926 15:02:35.231736 9193 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } has no permanent_uuid. Determining permanent_uuid...
I20260926 15:02:35.248801 9193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: Bootstrap starting.
I20260926 15:02:35.252316 9193 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:35.253348 9193 log.cc:824] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: Log is configured to *not* fsync() on all Append() calls
I20260926 15:02:35.255704 9193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: No bootstrap required, opened a new log
I20260926 15:02:35.263923 9193 raft_consensus.cc:348] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:35.264259 9193 raft_consensus.cc:374] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:35.264394 9193 raft_consensus.cc:729] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 320cd09624624312bf1bcc0856c2874f, State: Initialized, Role: FOLLOWER
I20260926 15:02:35.264984 9193 consensus_queue.cc:260] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:35.265825 9200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:35.266156 9200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:02:35.266806 9193 sys_catalog.cc:564] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: configured and running, proceeding with master startup.
W20260926 15:02:35.279665 9214 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 15:02:35.279870 9214 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 15:02:35.276690 9124 master_runner.cc:186] Error getting master registration for 127.8.138.190:44281: OK, instance_id { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" instance_seqno: 1790434954221295 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 15:02:35.288074 9124 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:02:35.351199 9053 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } attempt: 4
I20260926 15:02:35.363306 9053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c: Bootstrap starting.
I20260926 15:02:35.366571 9053 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:35.367950 9053 log.cc:824] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c: Log is configured to *not* fsync() on all Append() calls
I20260926 15:02:35.370894 9053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c: No bootstrap required, opened a new log
I20260926 15:02:35.378167 9053 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:35.378485 9053 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:35.378621 9053 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6315fb5099e1420b95b8d83b3e19375c, State: Initialized, Role: FOLLOWER
I20260926 15:02:35.379225 9053 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:35.380205 9218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:35.380625 9218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:02:35.381305 9053 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:02:35.408509 9123 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } attempt: 5
I20260926 15:02:35.394407 8987 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: OK, instance_id { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" instance_seqno: 1790434954690549 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 15:02:35.413049 8987 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 15:02:35.414100 9147 tablet.cc:2369] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 15:02:35.416281 9232 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 15:02:35.416498 9232 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 15:02:35.423100 9123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42: Bootstrap starting.
I20260926 15:02:35.427351 9123 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:35.428776 9123 log.cc:824] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42: Log is configured to *not* fsync() on all Append() calls
I20260926 15:02:35.431828 9123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42: No bootstrap required, opened a new log
I20260926 15:02:35.443485 9123 raft_consensus.cc:348] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:35.443974 9123 raft_consensus.cc:374] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:35.444092 9123 raft_consensus.cc:729] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9958f2085242f7ab82ae933ceb1c42, State: Initialized, Role: FOLLOWER
I20260926 15:02:35.445241 9123 consensus_queue.cc:260] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:35.446274 9236 sys_catalog.cc:455] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:35.446746 9236 sys_catalog.cc:458] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:02:35.447561 9123 sys_catalog.cc:564] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [sys.catalog]: configured and running, proceeding with master startup.
W20260926 15:02:35.459146 9007 tablet.cc:2369] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:02:35.492848 9251 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 15:02:35.504781 9056 master_runner.cc:263] No leader master found from master be9958f2085242f7ab82ae933ceb1c42
I20260926 15:02:35.504937 9056 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:02:35.540148 9200 raft_consensus.cc:482] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:02:35.540540 9200 raft_consensus.cc:504] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:35.542268 9200 leader_election.cc:288] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105)
I20260926 15:02:35.542846 9098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "320cd09624624312bf1bcc0856c2874f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9958f2085242f7ab82ae933ceb1c42" is_pre_election: true
I20260926 15:02:35.543274 9028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "320cd09624624312bf1bcc0856c2874f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6315fb5099e1420b95b8d83b3e19375c" is_pre_election: true
I20260926 15:02:35.543488 9098 raft_consensus.cc:2454] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 320cd09624624312bf1bcc0856c2874f in term 0.
I20260926 15:02:35.544014 9028 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 320cd09624624312bf1bcc0856c2874f in term 0.
I20260926 15:02:35.544718 9142 leader_election.cc:302] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f, 6315fb5099e1420b95b8d83b3e19375c; no voters:
I20260926 15:02:35.546430 9200 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:02:35.546581 9200 raft_consensus.cc:482] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:02:35.546695 9200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:35.549418 9200 raft_consensus.cc:504] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:35.550544 9028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "320cd09624624312bf1bcc0856c2874f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6315fb5099e1420b95b8d83b3e19375c"
I20260926 15:02:35.550770 9028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:35.551774 9200 leader_election.cc:288] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 1 election: Requested vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105)
I20260926 15:02:35.552361 9098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "320cd09624624312bf1bcc0856c2874f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9958f2085242f7ab82ae933ceb1c42"
I20260926 15:02:35.552569 9098 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:35.554507 9028 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 320cd09624624312bf1bcc0856c2874f in term 1.
I20260926 15:02:35.554911 9142 leader_election.cc:302] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f, 6315fb5099e1420b95b8d83b3e19375c; no voters:
I20260926 15:02:35.555186 9200 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:02:35.555609 9098 raft_consensus.cc:2454] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 320cd09624624312bf1bcc0856c2874f in term 1.
I20260926 15:02:35.556754 9200 raft_consensus.cc:686] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 1 LEADER]: Becoming Leader. State: Replica: 320cd09624624312bf1bcc0856c2874f, State: Running, Role: LEADER
I20260926 15:02:35.557732 9200 consensus_queue.cc:237] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:35.560672 9252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 320cd09624624312bf1bcc0856c2874f. Latest consensus state: current_term: 1 leader_uuid: "320cd09624624312bf1bcc0856c2874f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:35.561015 9252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: This master's current role is: LEADER
I20260926 15:02:35.563686 9254 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:02:35.565430 9254 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:02:35.573493 9098 raft_consensus.cc:1261] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 1 FOLLOWER]: Refusing update from remote peer 320cd09624624312bf1bcc0856c2874f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:02:35.573611 9028 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 1 FOLLOWER]: Refusing update from remote peer 320cd09624624312bf1bcc0856c2874f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:02:35.574460 9252 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:02:35.579130 9252 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:02:35.601377 9218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 320cd09624624312bf1bcc0856c2874f. Latest consensus state: current_term: 1 leader_uuid: "320cd09624624312bf1bcc0856c2874f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:35.601796 9218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:02:35.608383 9252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "320cd09624624312bf1bcc0856c2874f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:35.608748 9252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: This master's current role is: LEADER
I20260926 15:02:35.610292 9256 mvcc.cc:205] Tried to move back new op lower bound from 7333621578018942976 to 7333621577969774592. Current Snapshot: MvccSnapshot[applied={T|T < 7333621578018942976}]
I20260926 15:02:35.614877 9218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "320cd09624624312bf1bcc0856c2874f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:35.615182 9218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:02:35.617413 9254 catalog_manager.cc:1347] Generated new cluster ID: 497ed160bc39477e99bf3beb637a4391
I20260926 15:02:35.617542 9254 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:02:35.630925 9236 sys_catalog.cc:455] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 320cd09624624312bf1bcc0856c2874f. Latest consensus state: current_term: 1 leader_uuid: "320cd09624624312bf1bcc0856c2874f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:35.631187 9236 sys_catalog.cc:458] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:02:35.632743 9200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "320cd09624624312bf1bcc0856c2874f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:35.633105 9200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: This master's current role is: LEADER
I20260926 15:02:35.642138 9236 sys_catalog.cc:455] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "320cd09624624312bf1bcc0856c2874f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:35.642410 9236 sys_catalog.cc:458] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:02:35.648473 9254 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:02:35.649889 9254 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:02:35.665359 9254 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: Generated new TSK 0
I20260926 15:02:35.666304 9254 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:02:35.667438 9254 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260926 15:02:35.668406 9254 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260926 15:02:35.692848 8746 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu
/tmp/dist-test-taskLGH8i2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.138.129:0
--local_ip_for_outbound_sockets=127.8.138.129
--webserver_interface=127.8.138.129
--webserver_port=0
--tserver_master_addrs=127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473
--builtin_ntp_servers=127.8.138.148:33835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
W20260926 15:02:36.085005 9263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:36.085296 9263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:36.085388 9263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:36.093146 9263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:36.093344 9263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.138.129
I20260926 15:02:36.103506 9263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.138.148:33835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.138.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.138.129
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.138.129
--log_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:36.104912 9263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:36.106941 9263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:36.123461 9268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:36.123478 9271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:36.123664 9269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:36.124629 9263 server_base.cc:1034] running on GCE node
I20260926 15:02:36.125416 9263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:36.126638 9263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:36.127951 9263 hybrid_clock.cc:648] HybridClock initialized: now 1790434956127830 us; error 38 us; skew 500 ppm
I20260926 15:02:36.128374 9263 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:02:36.131644 9263 webserver.cc:460] Webserver started at http://127.8.138.129:41467/ using document root <none> and password file <none>
I20260926 15:02:36.132560 9263 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:36.132670 9263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:36.132925 9263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:36.135725 9263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-0/data/instance:
uuid: "66ca148489d64deabd1dcbf7870c6d57"
format_stamp: "Formatted at 2026-09-26 15:02:36 on dist-test-slave-7qf4"
I20260926 15:02:36.136533 9263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-0/wal/instance:
uuid: "66ca148489d64deabd1dcbf7870c6d57"
format_stamp: "Formatted at 2026-09-26 15:02:36 on dist-test-slave-7qf4"
I20260926 15:02:36.142503 9263 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260926 15:02:36.146798 9277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:36.148661 9263 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260926 15:02:36.148854 9263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-0/wal
uuid: "66ca148489d64deabd1dcbf7870c6d57"
format_stamp: "Formatted at 2026-09-26 15:02:36 on dist-test-slave-7qf4"
I20260926 15:02:36.149072 9263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:36.181176 9263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:36.182508 9263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:36.183313 9263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:36.184515 9263 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:02:36.186159 9263 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:02:36.186228 9263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:36.186365 9263 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:02:36.186432 9263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:36.225858 9263 rpc_server.cc:307] RPC server started. Bound to: 127.8.138.129:40327
I20260926 15:02:36.225980 9389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.138.129:40327 every 8 connection(s)
I20260926 15:02:36.227402 9263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-0/data/info.pb
I20260926 15:02:36.228607 8746 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu as pid 9263
I20260926 15:02:36.228823 8746 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-0/wal/instance
I20260926 15:02:36.235615 8746 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu
/tmp/dist-test-taskLGH8i2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.138.130:0
--local_ip_for_outbound_sockets=127.8.138.130
--webserver_interface=127.8.138.130
--webserver_port=0
--tserver_master_addrs=127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473
--builtin_ntp_servers=127.8.138.148:33835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260926 15:02:36.288298 9390 heartbeater.cc:347] Connected to a master server at 127.8.138.188:45473
I20260926 15:02:36.288594 9390 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:36.289496 9394 heartbeater.cc:347] Connected to a master server at 127.8.138.189:37105
I20260926 15:02:36.289616 9394 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:36.289695 9390 heartbeater.cc:511] Master 127.8.138.188:45473 requested a full tablet report, sending...
I20260926 15:02:36.290126 9394 heartbeater.cc:511] Master 127.8.138.189:37105 requested a full tablet report, sending...
I20260926 15:02:36.292891 9158 ts_manager.cc:194] Registered new tserver with Master: 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327)
I20260926 15:02:36.294677 9158 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.138.129:42635
I20260926 15:02:36.296020 9088 ts_manager.cc:194] Registered new tserver with Master: 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327)
I20260926 15:02:36.303655 9391 heartbeater.cc:347] Connected to a master server at 127.8.138.190:44281
I20260926 15:02:36.303848 9391 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:36.304343 9391 heartbeater.cc:511] Master 127.8.138.190:44281 requested a full tablet report, sending...
I20260926 15:02:36.306245 9017 ts_manager.cc:194] Registered new tserver with Master: 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327)
I20260926 15:02:36.418545 9232 catalog_manager.cc:1259] Loaded cluster ID: 497ed160bc39477e99bf3beb637a4391
I20260926 15:02:36.418658 9232 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c: loading cluster ID for follower catalog manager: success
I20260926 15:02:36.421206 9232 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c: acquiring CA information for follower catalog manager: success
I20260926 15:02:36.423054 9232 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 15:02:36.498852 9251 catalog_manager.cc:1259] Loaded cluster ID: 497ed160bc39477e99bf3beb637a4391
I20260926 15:02:36.499192 9251 catalog_manager.cc:1552] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42: loading cluster ID for follower catalog manager: success
I20260926 15:02:36.504415 9251 catalog_manager.cc:1574] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42: acquiring CA information for follower catalog manager: success
I20260926 15:02:36.508061 9251 catalog_manager.cc:1602] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 15:02:36.582136 9402 raft_consensus.cc:482] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:02:36.582444 9402 raft_consensus.cc:504] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:36.585428 9027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "be9958f2085242f7ab82ae933ceb1c42" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6315fb5099e1420b95b8d83b3e19375c" is_pre_election: true
I20260926 15:02:36.585553 9168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "be9958f2085242f7ab82ae933ceb1c42" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "320cd09624624312bf1bcc0856c2874f" is_pre_election: true
I20260926 15:02:36.586535 9072 leader_election.cc:302] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9958f2085242f7ab82ae933ceb1c42; no voters: 320cd09624624312bf1bcc0856c2874f, 6315fb5099e1420b95b8d83b3e19375c
I20260926 15:02:36.586695 9402 leader_election.cc:288] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
I20260926 15:02:36.587236 9402 raft_consensus.cc:2726] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 15:02:36.628361 9403 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:02:36.628635 9403 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:36.630194 9403 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
I20260926 15:02:36.631376 9168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6315fb5099e1420b95b8d83b3e19375c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "320cd09624624312bf1bcc0856c2874f" is_pre_election: true
I20260926 15:02:36.633121 9098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6315fb5099e1420b95b8d83b3e19375c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9958f2085242f7ab82ae933ceb1c42" is_pre_election: true
I20260926 15:02:36.633374 9098 raft_consensus.cc:2454] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6315fb5099e1420b95b8d83b3e19375c in term 1.
I20260926 15:02:36.633975 9005 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c, be9958f2085242f7ab82ae933ceb1c42; no voters: 320cd09624624312bf1bcc0856c2874f
I20260926 15:02:36.634717 9403 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 15:02:36.634895 9403 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:02:36.635043 9403 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 1 FOLLOWER]: Advancing to term 2
I20260926 15:02:36.738762 9403 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
W20260926 15:02:36.739023 9395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:36.739410 9395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:36.739565 9395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260926 15:02:36.740061 9403 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 2 election: Requested vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
I20260926 15:02:36.740849 9168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6315fb5099e1420b95b8d83b3e19375c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "320cd09624624312bf1bcc0856c2874f"
I20260926 15:02:36.742054 9027 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 2 FOLLOWER]: Rejecting Update request from peer 320cd09624624312bf1bcc0856c2874f for earlier term 1. Current term is 2. Ops: []
I20260926 15:02:36.742669 9098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6315fb5099e1420b95b8d83b3e19375c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9958f2085242f7ab82ae933ceb1c42"
I20260926 15:02:36.743445 9005 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:02:36.743872 9403 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260926 15:02:36.745987 9407 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260926 15:02:36.754208 9408 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 1 LEADER]: Stepping down as leader of term 1
I20260926 15:02:36.754344 9408 raft_consensus.cc:729] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 320cd09624624312bf1bcc0856c2874f, State: Running, Role: LEADER
I20260926 15:02:36.754760 9408 consensus_queue.cc:260] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:36.755384 9408 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 1 FOLLOWER]: Advancing to term 2
W20260926 15:02:36.755950 9395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:36.756176 9395 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.138.130
I20260926 15:02:36.767192 9395 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.138.148:33835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.138.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.138.130
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.138.130
--log_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:36.768719 9395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:36.770605 9395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:36.787110 9411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:36.787599 9395 server_base.cc:1034] running on GCE node
W20260926 15:02:36.787921 9413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:36.788100 9410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:36.788822 9395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:36.789877 9395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:36.791126 9395 hybrid_clock.cc:648] HybridClock initialized: now 1790434956791048 us; error 57 us; skew 500 ppm
I20260926 15:02:36.791509 9395 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:02:36.794762 9395 webserver.cc:460] Webserver started at http://127.8.138.130:35533/ using document root <none> and password file <none>
I20260926 15:02:36.795504 9395 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:36.795651 9395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:36.796028 9395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:36.798702 9395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-1/data/instance:
uuid: "ae330d635b1848aaaecee2c7ce96520a"
format_stamp: "Formatted at 2026-09-26 15:02:36 on dist-test-slave-7qf4"
I20260926 15:02:36.799396 9390 heartbeater.cc:503] Master 127.8.138.188:45473 was elected leader, sending a full tablet report...
I20260926 15:02:36.799575 9395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-1/wal/instance:
uuid: "ae330d635b1848aaaecee2c7ce96520a"
format_stamp: "Formatted at 2026-09-26 15:02:36 on dist-test-slave-7qf4"
I20260926 15:02:36.806141 9395 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260926 15:02:36.810526 9419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:36.812139 9395 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260926 15:02:36.812322 9395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-1/wal
uuid: "ae330d635b1848aaaecee2c7ce96520a"
format_stamp: "Formatted at 2026-09-26 15:02:36 on dist-test-slave-7qf4"
I20260926 15:02:36.812515 9395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:36.835209 9395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:36.836131 9395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:36.836437 9395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:36.837455 9395 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:02:36.839082 9395 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:02:36.839185 9395 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:36.839321 9395 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:02:36.839389 9395 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:36.892611 9395 rpc_server.cc:307] RPC server started. Bound to: 127.8.138.130:44309
I20260926 15:02:36.892633 9531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.138.130:44309 every 8 connection(s)
I20260926 15:02:36.893919 9395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-1/data/info.pb
I20260926 15:02:36.895336 8746 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu as pid 9395
I20260926 15:02:36.895550 8746 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-1/wal/instance
I20260926 15:02:36.927135 9533 heartbeater.cc:347] Connected to a master server at 127.8.138.190:44281
I20260926 15:02:36.927495 9533 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:36.928470 9533 heartbeater.cc:511] Master 127.8.138.190:44281 requested a full tablet report, sending...
I20260926 15:02:36.930166 9017 ts_manager.cc:194] Registered new tserver with Master: ae330d635b1848aaaecee2c7ce96520a (127.8.138.130:44309)
I20260926 15:02:36.935530 9532 heartbeater.cc:347] Connected to a master server at 127.8.138.188:45473
I20260926 15:02:36.935740 9532 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:36.936158 9532 heartbeater.cc:511] Master 127.8.138.188:45473 requested a full tablet report, sending...
I20260926 15:02:36.937304 9158 ts_manager.cc:194] Registered new tserver with Master: ae330d635b1848aaaecee2c7ce96520a (127.8.138.130:44309)
I20260926 15:02:36.941139 9534 heartbeater.cc:347] Connected to a master server at 127.8.138.189:37105
I20260926 15:02:36.941335 9534 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:36.941884 9534 heartbeater.cc:511] Master 127.8.138.189:37105 requested a full tablet report, sending...
I20260926 15:02:36.943171 9087 ts_manager.cc:194] Registered new tserver with Master: ae330d635b1848aaaecee2c7ce96520a (127.8.138.130:44309)
I20260926 15:02:37.149725 9542 raft_consensus.cc:482] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:02:37.149925 9542 raft_consensus.cc:504] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:37.150529 9542 leader_election.cc:288] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
I20260926 15:02:37.151192 9027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "be9958f2085242f7ab82ae933ceb1c42" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6315fb5099e1420b95b8d83b3e19375c" is_pre_election: true
I20260926 15:02:37.151255 9168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "be9958f2085242f7ab82ae933ceb1c42" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "320cd09624624312bf1bcc0856c2874f" is_pre_election: true
I20260926 15:02:37.151510 9027 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be9958f2085242f7ab82ae933ceb1c42 in current term 2: Already voted for candidate 6315fb5099e1420b95b8d83b3e19375c in this term.
I20260926 15:02:37.151537 9168 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9958f2085242f7ab82ae933ceb1c42 in term 2.
I20260926 15:02:37.152007 9074 leader_election.cc:302] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42; no voters: 6315fb5099e1420b95b8d83b3e19375c
I20260926 15:02:37.152283 9542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 15:02:37.152398 9542 raft_consensus.cc:482] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:02:37.152491 9542 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 1 FOLLOWER]: Advancing to term 2
I20260926 15:02:37.155326 9542 raft_consensus.cc:504] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:37.156150 9542 leader_election.cc:288] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 2 election: Requested vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
I20260926 15:02:37.156405 9168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "be9958f2085242f7ab82ae933ceb1c42" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "320cd09624624312bf1bcc0856c2874f"
I20260926 15:02:37.156426 9027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "be9958f2085242f7ab82ae933ceb1c42" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6315fb5099e1420b95b8d83b3e19375c"
I20260926 15:02:37.156711 9027 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate be9958f2085242f7ab82ae933ceb1c42 in current term 2: Already voted for candidate 6315fb5099e1420b95b8d83b3e19375c in this term.
I20260926 15:02:37.250316 9168 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9958f2085242f7ab82ae933ceb1c42 in term 2.
I20260926 15:02:37.250968 9074 leader_election.cc:302] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42; no voters: 6315fb5099e1420b95b8d83b3e19375c
I20260926 15:02:37.251334 9542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 15:02:37.251893 9542 raft_consensus.cc:686] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 2 LEADER]: Becoming Leader. State: Replica: be9958f2085242f7ab82ae933ceb1c42, State: Running, Role: LEADER
I20260926 15:02:37.252424 9542 consensus_queue.cc:237] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:37.254289 9543 sys_catalog.cc:455] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be9958f2085242f7ab82ae933ceb1c42. Latest consensus state: current_term: 2 leader_uuid: "be9958f2085242f7ab82ae933ceb1c42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:37.254647 9543 sys_catalog.cc:458] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [sys.catalog]: This master's current role is: LEADER
I20260926 15:02:37.255297 9545 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:02:37.257288 9545 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:02:37.258709 9545 catalog_manager.cc:1259] Loaded cluster ID: 497ed160bc39477e99bf3beb637a4391
I20260926 15:02:37.258775 9545 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:02:37.259900 9545 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:02:37.260691 9545 catalog_manager.cc:5879] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42: Loaded TSK: 0
I20260926 15:02:37.261561 9545 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:02:37.262425 9545 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260926 15:02:37.263128 9545 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260926 15:02:37.275573 8746 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu
/tmp/dist-test-taskLGH8i2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.138.131:0
--local_ip_for_outbound_sockets=127.8.138.131
--webserver_interface=127.8.138.131
--webserver_port=0
--tserver_master_addrs=127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473
--builtin_ntp_servers=127.8.138.148:33835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260926 15:02:37.283203 9214 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 15:02:37.447822 9087 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.138.130:33109
I20260926 15:02:37.477777 9547 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:02:37.478015 9547 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:37.478742 9547 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
I20260926 15:02:37.479235 9098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6315fb5099e1420b95b8d83b3e19375c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9958f2085242f7ab82ae933ceb1c42" is_pre_election: true
I20260926 15:02:37.479300 9168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6315fb5099e1420b95b8d83b3e19375c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "320cd09624624312bf1bcc0856c2874f" is_pre_election: true
I20260926 15:02:37.480370 9168 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6315fb5099e1420b95b8d83b3e19375c in term 2.
I20260926 15:02:37.480763 9004 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f, 6315fb5099e1420b95b8d83b3e19375c; no voters: be9958f2085242f7ab82ae933ceb1c42
I20260926 15:02:37.481077 9547 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260926 15:02:37.481323 9547 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:02:37.481420 9547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 2 FOLLOWER]: Advancing to term 3
I20260926 15:02:37.509064 9547 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:37.509984 9547 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 3 election: Requested vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
I20260926 15:02:37.510402 9098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6315fb5099e1420b95b8d83b3e19375c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "be9958f2085242f7ab82ae933ceb1c42"
I20260926 15:02:37.510811 9168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6315fb5099e1420b95b8d83b3e19375c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "320cd09624624312bf1bcc0856c2874f"
I20260926 15:02:37.511073 9168 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 2 FOLLOWER]: Advancing to term 3
I20260926 15:02:37.513209 9168 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6315fb5099e1420b95b8d83b3e19375c in term 3.
I20260926 15:02:37.513813 9004 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f, 6315fb5099e1420b95b8d83b3e19375c; no voters: be9958f2085242f7ab82ae933ceb1c42
I20260926 15:02:37.514264 9547 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 3 FOLLOWER]: Leader election won for term 3
I20260926 15:02:37.514755 9547 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 3 LEADER]: Becoming Leader. State: Replica: 6315fb5099e1420b95b8d83b3e19375c, State: Running, Role: LEADER
I20260926 15:02:37.515300 9547 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:37.517254 9548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6315fb5099e1420b95b8d83b3e19375c. Latest consensus state: current_term: 3 leader_uuid: "6315fb5099e1420b95b8d83b3e19375c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:37.517585 9548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: This master's current role is: LEADER
I20260926 15:02:37.518683 9550 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:02:37.520443 9550 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:02:37.521193 9550 catalog_manager.cc:1259] Loaded cluster ID: 497ed160bc39477e99bf3beb637a4391
I20260926 15:02:37.521294 9550 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:02:37.522766 9550 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:02:37.524396 9550 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c: Loaded TSK: 0
I20260926 15:02:37.525688 9550 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:02:37.526489 9550 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260926 15:02:37.527684 9550 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260926 15:02:37.649716 9027 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 3 LEADER]: Rejecting Update request from peer be9958f2085242f7ab82ae933ceb1c42 for earlier term 2. Current term is 3. Ops: []
I20260926 15:02:37.650504 9543 consensus_queue.cc:1046] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260926 15:02:37.650811 9543 consensus_queue.cc:1225] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [LEADER]: Peer 6315fb5099e1420b95b8d83b3e19375c log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260926 15:02:37.651495 9542 raft_consensus.cc:3032] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 2 LEADER]: Stepping down as leader of term 2
I20260926 15:02:37.651600 9542 raft_consensus.cc:729] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: be9958f2085242f7ab82ae933ceb1c42, State: Running, Role: LEADER
W20260926 15:02:37.651604 9546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:37.652067 9546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
I20260926 15:02:37.651984 9542 consensus_queue.cc:260] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
W20260926 15:02:37.652266 9546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260926 15:02:37.652544 9542 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 2 FOLLOWER]: Advancing to term 3
W20260926 15:02:37.660164 9546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:37.660399 9546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.138.131
I20260926 15:02:37.671063 9546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.138.148:33835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.138.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.138.131
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.138.131
--log_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:37.672585 9546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:37.674511 9546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:37.689576 9556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:37.689615 9558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:37.690156 9555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:37.690845 9546 server_base.cc:1034] running on GCE node
I20260926 15:02:37.691772 9546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:37.693104 9546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:37.694423 9546 hybrid_clock.cc:648] HybridClock initialized: now 1790434957694355 us; error 72 us; skew 500 ppm
I20260926 15:02:37.694784 9546 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:02:37.698076 9546 webserver.cc:460] Webserver started at http://127.8.138.131:42421/ using document root <none> and password file <none>
I20260926 15:02:37.699079 9546 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:37.699237 9546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:37.699625 9546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:37.703647 9546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-2/data/instance:
uuid: "f052d5cbee65406593d8e44f74225bc2"
format_stamp: "Formatted at 2026-09-26 15:02:37 on dist-test-slave-7qf4"
I20260926 15:02:37.704751 9546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-2/wal/instance:
uuid: "f052d5cbee65406593d8e44f74225bc2"
format_stamp: "Formatted at 2026-09-26 15:02:37 on dist-test-slave-7qf4"
I20260926 15:02:37.712244 9546 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260926 15:02:37.717094 9564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:37.719269 9546 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 15:02:37.719446 9546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-2/wal
uuid: "f052d5cbee65406593d8e44f74225bc2"
format_stamp: "Formatted at 2026-09-26 15:02:37 on dist-test-slave-7qf4"
I20260926 15:02:37.719645 9546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:37.736884 9546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:37.737767 9546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:37.738065 9546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:37.739063 9546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:02:37.740965 9546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:02:37.741050 9546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:37.741142 9546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:02:37.741189 9546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:37.784256 9546 rpc_server.cc:307] RPC server started. Bound to: 127.8.138.131:35445
I20260926 15:02:37.784827 9676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.138.131:35445 every 8 connection(s)
I20260926 15:02:37.785964 9546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-2/data/info.pb
I20260926 15:02:37.796228 8746 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu as pid 9546
I20260926 15:02:37.796455 8746 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-2/wal/instance
I20260926 15:02:37.804463 9394 heartbeater.cc:503] Master 127.8.138.189:37105 was elected leader, sending a full tablet report...
I20260926 15:02:37.815089 9677 heartbeater.cc:347] Connected to a master server at 127.8.138.188:45473
I20260926 15:02:37.815487 9677 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:37.815575 9678 heartbeater.cc:347] Connected to a master server at 127.8.138.190:44281
I20260926 15:02:37.815729 9678 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:37.816534 9679 heartbeater.cc:347] Connected to a master server at 127.8.138.189:37105
I20260926 15:02:37.816685 9679 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:37.817363 9679 heartbeater.cc:511] Master 127.8.138.189:37105 requested a full tablet report, sending...
I20260926 15:02:37.817435 9678 heartbeater.cc:511] Master 127.8.138.190:44281 requested a full tablet report, sending...
I20260926 15:02:37.817458 9677 heartbeater.cc:511] Master 127.8.138.188:45473 requested a full tablet report, sending...
I20260926 15:02:37.819197 9158 ts_manager.cc:194] Registered new tserver with Master: f052d5cbee65406593d8e44f74225bc2 (127.8.138.131:35445)
I20260926 15:02:37.819370 9087 ts_manager.cc:194] Registered new tserver with Master: f052d5cbee65406593d8e44f74225bc2 (127.8.138.131:35445)
I20260926 15:02:37.819905 9017 ts_manager.cc:194] Registered new tserver with Master: f052d5cbee65406593d8e44f74225bc2 (127.8.138.131:35445)
I20260926 15:02:37.821676 9017 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.138.131:55843
I20260926 15:02:37.826660 8746 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 15:02:37.863345 9017 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57370:
name: "default.Table_f356bc0dfc6a47f98430002dccbb873f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:37.865084 9018 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57370:
name: "default.Table_d2243274908043e58210a197141af813"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:37.866650 9016 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57370:
name: "default.Table_7b56039466d14dcab4dc63192de3b08e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:37.868150 9014 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57370:
name: "default.Table_96be20374f7e43e7a865549e39bd64f6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:37.869659 9015 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57370:
name: "default.Table_854219551f9048c6925c711ce33d5256"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:37.870683 9013 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57370:
name: "default.Table_22fd4957e3d14964803428457f8490ac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:37.870716 9012 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57370:
name: "default.Table_e3b713aac3c94e449bb5882310d55a40"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:37.871405 9010 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57370:
name: "default.Table_43176f08d9e24974abdd53171465a15c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:37.872085 9011 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57370:
name: "default.Table_41a3ce3c89684cbb9054c4251a0cc51c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:37.872169 9009 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57370:
name: "default.Table_95f48725b6ce47fab5f96e18ccb69dbc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 15:02:37.879199 9017 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_f356bc0dfc6a47f98430002dccbb873f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 15:02:37.886399 9018 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_d2243274908043e58210a197141af813 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 15:02:37.887812 9009 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_95f48725b6ce47fab5f96e18ccb69dbc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 15:02:37.888816 9010 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_43176f08d9e24974abdd53171465a15c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 15:02:37.889626 9012 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_e3b713aac3c94e449bb5882310d55a40 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 15:02:37.890264 9013 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_22fd4957e3d14964803428457f8490ac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 15:02:37.890961 9015 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_854219551f9048c6925c711ce33d5256 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 15:02:37.891871 9014 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_96be20374f7e43e7a865549e39bd64f6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 15:02:37.892767 9016 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_7b56039466d14dcab4dc63192de3b08e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 15:02:37.893613 9011 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_41a3ce3c89684cbb9054c4251a0cc51c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 15:02:37.895475 8746 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu pid 8987
I20260926 15:02:37.897004 9048 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:02:37.952091 9534 heartbeater.cc:503] Master 127.8.138.189:37105 was elected leader, sending a full tablet report...
I20260926 15:02:38.052378 9542 raft_consensus.cc:482] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:02:38.052651 9542 raft_consensus.cc:504] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:38.053453 9542 leader_election.cc:288] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
I20260926 15:02:38.054075 9168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "be9958f2085242f7ab82ae933ceb1c42" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "320cd09624624312bf1bcc0856c2874f" is_pre_election: true
I20260926 15:02:38.054339 9168 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9958f2085242f7ab82ae933ceb1c42 in term 3.
I20260926 15:02:38.054804 9074 leader_election.cc:302] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42; no voters:
I20260926 15:02:38.055095 9542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260926 15:02:38.055192 9542 raft_consensus.cc:482] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:02:38.055308 9542 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 3 FOLLOWER]: Advancing to term 4
I20260926 15:02:38.076841 9027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "be9958f2085242f7ab82ae933ceb1c42" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6315fb5099e1420b95b8d83b3e19375c" is_pre_election: true
I20260926 15:02:38.078948 9168 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 3 FOLLOWER]: Refusing update from remote peer 6315fb5099e1420b95b8d83b3e19375c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260926 15:02:38.079761 8746 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu with pid 8987
I20260926 15:02:38.079772 9548 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [LEADER]: Connected to new peer: Peer: permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260926 15:02:38.100600 9533 heartbeater.cc:644] Failed to heartbeat to 127.8.138.190:44281 (0 consecutive failures): Network error: Failed to send heartbeat to master: recv got EOF from 127.8.138.190:44281 (error 108)
W20260926 15:02:38.101119 9687 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.8.138.190:44281)
I20260926 15:02:38.128420 9542 raft_consensus.cc:504] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:38.128969 9098 raft_consensus.cc:1226] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 4 FOLLOWER]: Rejecting Update request from peer 6315fb5099e1420b95b8d83b3e19375c for earlier term 3. Current term is 4. Ops: []
I20260926 15:02:38.129326 9542 leader_election.cc:288] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 4 election: Requested vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:02:38.129921 9075 connection.cc:441] server connection from 127.0.0.1:59396 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59396 (request call id 6) could send its response
I20260926 15:02:38.130251 9168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "be9958f2085242f7ab82ae933ceb1c42" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "320cd09624624312bf1bcc0856c2874f"
I20260926 15:02:38.130559 9168 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 3 FOLLOWER]: Advancing to term 4
W20260926 15:02:38.139359 9072 leader_election.cc:334] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281): Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
I20260926 15:02:38.232947 9168 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9958f2085242f7ab82ae933ceb1c42 in term 4.
I20260926 15:02:38.233637 9074 leader_election.cc:302] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42; no voters: 6315fb5099e1420b95b8d83b3e19375c
I20260926 15:02:38.234164 9542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 4 FOLLOWER]: Leader election won for term 4
I20260926 15:02:38.234659 9542 raft_consensus.cc:686] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 4 LEADER]: Becoming Leader. State: Replica: be9958f2085242f7ab82ae933ceb1c42, State: Running, Role: LEADER
I20260926 15:02:38.234943 9542 consensus_queue.cc:237] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:38.236639 9729 sys_catalog.cc:455] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be9958f2085242f7ab82ae933ceb1c42. Latest consensus state: current_term: 4 leader_uuid: "be9958f2085242f7ab82ae933ceb1c42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:38.236943 9729 sys_catalog.cc:458] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [sys.catalog]: This master's current role is: LEADER
I20260926 15:02:38.239125 9730 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:02:38.241312 9730 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:02:38.242357 9730 catalog_manager.cc:1259] Loaded cluster ID: 497ed160bc39477e99bf3beb637a4391
I20260926 15:02:38.242502 9730 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:02:38.244087 9730 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:02:38.245090 9730 catalog_manager.cc:5879] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42: Loaded TSK: 0
I20260926 15:02:38.245669 9730 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:02:38.246688 9730 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260926 15:02:38.247617 9730 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260926 15:02:38.252014 9087 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59430:
name: "default.Table_95f48725b6ce47fab5f96e18ccb69dbc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:38.252583 9088 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59430:
name: "default.Table_43176f08d9e24974abdd53171465a15c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:38.252969 9086 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59430:
name: "default.Table_41a3ce3c89684cbb9054c4251a0cc51c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:38.253750 9084 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59430:
name: "default.Table_d2243274908043e58210a197141af813"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:38.253364 9085 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59430:
name: "default.Table_e3b713aac3c94e449bb5882310d55a40"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:38.253898 9079 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59430:
name: "default.Table_22fd4957e3d14964803428457f8490ac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:38.255138 9082 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59430:
name: "default.Table_7b56039466d14dcab4dc63192de3b08e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:38.254352 9080 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59430:
name: "default.Table_96be20374f7e43e7a865549e39bd64f6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:38.255533 9083 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59430:
name: "default.Table_f356bc0dfc6a47f98430002dccbb873f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:38.254758 9081 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59430:
name: "default.Table_854219551f9048c6925c711ce33d5256"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 15:02:38.268419 9086 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_41a3ce3c89684cbb9054c4251a0cc51c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 15:02:38.268658 9084 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_d2243274908043e58210a197141af813 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 15:02:38.268484 9088 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_43176f08d9e24974abdd53171465a15c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 15:02:38.271348 9087 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_95f48725b6ce47fab5f96e18ccb69dbc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 15:02:38.272725 9085 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_e3b713aac3c94e449bb5882310d55a40 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 15:02:38.272995 9082 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_7b56039466d14dcab4dc63192de3b08e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 15:02:38.273660 9083 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_f356bc0dfc6a47f98430002dccbb873f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 15:02:38.273998 9080 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_96be20374f7e43e7a865549e39bd64f6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 15:02:38.272734 9079 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_22fd4957e3d14964803428457f8490ac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 15:02:38.274962 9081 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_854219551f9048c6925c711ce33d5256 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 15:02:38.313745 9391 heartbeater.cc:503] Master 127.8.138.190:44281 was elected leader, sending a full tablet report...
W20260926 15:02:38.315498 9391 heartbeater.cc:644] Failed to heartbeat to 127.8.138.190:44281 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
I20260926 15:02:38.325317 9678 heartbeater.cc:503] Master 127.8.138.190:44281 was elected leader, sending a full tablet report...
W20260926 15:02:38.326718 9678 heartbeater.cc:644] Failed to heartbeat to 127.8.138.190:44281 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
I20260926 15:02:38.620539 9168 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 4 FOLLOWER]: Refusing update from remote peer be9958f2085242f7ab82ae933ceb1c42: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260926 15:02:38.621290 9729 consensus_queue.cc:1035] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260926 15:02:38.623975 9734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: SysCatalogTable state changed. Reason: New leader be9958f2085242f7ab82ae933ceb1c42. Latest consensus state: current_term: 4 leader_uuid: "be9958f2085242f7ab82ae933ceb1c42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:38.624256 9734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:02:38.626277 9542 sys_catalog.cc:455] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "be9958f2085242f7ab82ae933ceb1c42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:38.626574 9542 sys_catalog.cc:458] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [sys.catalog]: This master's current role is: LEADER
W20260926 15:02:38.629361 9072 consensus_peers.cc:487] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 -> Peer 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281): Couldn't send request to peer 6315fb5099e1420b95b8d83b3e19375c. Status: Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 15:02:38.636981 9734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "be9958f2085242f7ab82ae933ceb1c42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:38.637300 9734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:02:38.637036 9739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "be9958f2085242f7ab82ae933ceb1c42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:38.637588 9739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:02:38.808252 8746 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu
/tmp/dist-test-taskLGH8i2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.138.190:44281
--webserver_interface=127.8.138.190
--webserver_port=37041
--builtin_ntp_servers=127.8.138.148:33835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473
--hive_metastore_uris=thrift://127.0.0.1:40047
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260926 15:02:38.851308 9086 catalog_manager.cc:2148] created HMS catalog entry for table [id=7cc0c844d49c4ff08cef9e2c410991f8]
I20260926 15:02:38.863514 9086 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:59430:
table { table_name: "default.Table_41a3ce3c89684cbb9054c4251a0cc51c" } new_table_name: "default.Table_476468e6a23a414ebac439484ad9a68f" modify_external_catalogs: true
I20260926 15:02:38.908190 9467 tablet_service.cc:1467] Processing CreateTablet for tablet bb17748dce784ed3856e7bf01d7da3ba (DEFAULT_TABLE table=default.table_41a3ce3c89684cbb9054c4251a0cc51c [id=7cc0c844d49c4ff08cef9e2c410991f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:38.910053 9467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb17748dce784ed3856e7bf01d7da3ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:38.911458 9325 tablet_service.cc:1467] Processing CreateTablet for tablet bb17748dce784ed3856e7bf01d7da3ba (DEFAULT_TABLE table=default.table_41a3ce3c89684cbb9054c4251a0cc51c [id=7cc0c844d49c4ff08cef9e2c410991f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:38.912882 9325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb17748dce784ed3856e7bf01d7da3ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:38.932760 9750 tablet_bootstrap.cc:492] T bb17748dce784ed3856e7bf01d7da3ba P ae330d635b1848aaaecee2c7ce96520a: Bootstrap starting.
I20260926 15:02:38.935639 9612 tablet_service.cc:1467] Processing CreateTablet for tablet bb17748dce784ed3856e7bf01d7da3ba (DEFAULT_TABLE table=default.table_41a3ce3c89684cbb9054c4251a0cc51c [id=7cc0c844d49c4ff08cef9e2c410991f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:38.936271 9751 tablet_bootstrap.cc:492] T bb17748dce784ed3856e7bf01d7da3ba P 66ca148489d64deabd1dcbf7870c6d57: Bootstrap starting.
I20260926 15:02:38.937242 9612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb17748dce784ed3856e7bf01d7da3ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:38.940943 9751 tablet_bootstrap.cc:654] T bb17748dce784ed3856e7bf01d7da3ba P 66ca148489d64deabd1dcbf7870c6d57: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:38.951611 9750 tablet_bootstrap.cc:654] T bb17748dce784ed3856e7bf01d7da3ba P ae330d635b1848aaaecee2c7ce96520a: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:38.953135 9750 log.cc:824] T bb17748dce784ed3856e7bf01d7da3ba P ae330d635b1848aaaecee2c7ce96520a: Log is configured to *not* fsync() on all Append() calls
I20260926 15:02:38.954766 9751 log.cc:824] T bb17748dce784ed3856e7bf01d7da3ba P 66ca148489d64deabd1dcbf7870c6d57: Log is configured to *not* fsync() on all Append() calls
I20260926 15:02:38.969698 9750 tablet_bootstrap.cc:492] T bb17748dce784ed3856e7bf01d7da3ba P ae330d635b1848aaaecee2c7ce96520a: No bootstrap required, opened a new log
I20260926 15:02:38.970093 9750 ts_tablet_manager.cc:1397] T bb17748dce784ed3856e7bf01d7da3ba P ae330d635b1848aaaecee2c7ce96520a: Time spent bootstrapping tablet: real 0.038s user 0.006s sys 0.006s
I20260926 15:02:38.966593 9751 tablet_bootstrap.cc:492] T bb17748dce784ed3856e7bf01d7da3ba P 66ca148489d64deabd1dcbf7870c6d57: No bootstrap required, opened a new log
I20260926 15:02:38.972314 9751 ts_tablet_manager.cc:1397] T bb17748dce784ed3856e7bf01d7da3ba P 66ca148489d64deabd1dcbf7870c6d57: Time spent bootstrapping tablet: real 0.036s user 0.013s sys 0.000s
I20260926 15:02:38.984637 9754 tablet_bootstrap.cc:492] T bb17748dce784ed3856e7bf01d7da3ba P f052d5cbee65406593d8e44f74225bc2: Bootstrap starting.
I20260926 15:02:38.988597 9754 tablet_bootstrap.cc:654] T bb17748dce784ed3856e7bf01d7da3ba P f052d5cbee65406593d8e44f74225bc2: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:38.989933 9754 log.cc:824] T bb17748dce784ed3856e7bf01d7da3ba P f052d5cbee65406593d8e44f74225bc2: Log is configured to *not* fsync() on all Append() calls
I20260926 15:02:38.992502 9750 raft_consensus.cc:348] T bb17748dce784ed3856e7bf01d7da3ba P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } } peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } }
I20260926 15:02:38.996318 9750 raft_consensus.cc:374] T bb17748dce784ed3856e7bf01d7da3ba P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:38.996515 9750 raft_consensus.cc:729] T bb17748dce784ed3856e7bf01d7da3ba P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae330d635b1848aaaecee2c7ce96520a, State: Initialized, Role: FOLLOWER
I20260926 15:02:38.997614 9751 raft_consensus.cc:348] T bb17748dce784ed3856e7bf01d7da3ba P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } } peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } }
I20260926 15:02:38.998179 9751 raft_consensus.cc:374] T bb17748dce784ed3856e7bf01d7da3ba P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:38.997287 9750 consensus_queue.cc:260] T bb17748dce784ed3856e7bf01d7da3ba P ae330d635b1848aaaecee2c7ce96520a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } } peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } }
I20260926 15:02:38.998536 9751 raft_consensus.cc:729] T bb17748dce784ed3856e7bf01d7da3ba P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ca148489d64deabd1dcbf7870c6d57, State: Initialized, Role: FOLLOWER
I20260926 15:02:38.999503 9754 tablet_bootstrap.cc:492] T bb17748dce784ed3856e7bf01d7da3ba P f052d5cbee65406593d8e44f74225bc2: No bootstrap required, opened a new log
I20260926 15:02:38.999476 9751 consensus_queue.cc:260] T bb17748dce784ed3856e7bf01d7da3ba P 66ca148489d64deabd1dcbf7870c6d57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } } peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } }
I20260926 15:02:38.999811 9754 ts_tablet_manager.cc:1397] T bb17748dce784ed3856e7bf01d7da3ba P f052d5cbee65406593d8e44f74225bc2: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.009s
I20260926 15:02:39.012693 9534 heartbeater.cc:503] Master 127.8.138.189:37105 was elected leader, sending a full tablet report...
I20260926 15:02:39.013651 9750 ts_tablet_manager.cc:1428] T bb17748dce784ed3856e7bf01d7da3ba P ae330d635b1848aaaecee2c7ce96520a: Time spent starting tablet: real 0.043s user 0.011s sys 0.004s
I20260926 15:02:39.019449 9754 raft_consensus.cc:348] T bb17748dce784ed3856e7bf01d7da3ba P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } } peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } }
I20260926 15:02:39.019989 9754 raft_consensus.cc:374] T bb17748dce784ed3856e7bf01d7da3ba P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:39.020094 9754 raft_consensus.cc:729] T bb17748dce784ed3856e7bf01d7da3ba P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f052d5cbee65406593d8e44f74225bc2, State: Initialized, Role: FOLLOWER
I20260926 15:02:39.021849 9394 heartbeater.cc:503] Master 127.8.138.189:37105 was elected leader, sending a full tablet report...
I20260926 15:02:39.022182 9754 consensus_queue.cc:260] T bb17748dce784ed3856e7bf01d7da3ba P f052d5cbee65406593d8e44f74225bc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } } peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } }
I20260926 15:02:39.024140 9751 ts_tablet_manager.cc:1428] T bb17748dce784ed3856e7bf01d7da3ba P 66ca148489d64deabd1dcbf7870c6d57: Time spent starting tablet: real 0.052s user 0.013s sys 0.021s
I20260926 15:02:39.028535 9754 ts_tablet_manager.cc:1428] T bb17748dce784ed3856e7bf01d7da3ba P f052d5cbee65406593d8e44f74225bc2: Time spent starting tablet: real 0.029s user 0.017s sys 0.000s
I20260926 15:02:39.028851 9679 heartbeater.cc:503] Master 127.8.138.189:37105 was elected leader, sending a full tablet report...
W20260926 15:02:39.041914 9398 tablet.cc:2369] T bb17748dce784ed3856e7bf01d7da3ba P 66ca148489d64deabd1dcbf7870c6d57: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 15:02:39.048044 9680 tablet.cc:2369] T bb17748dce784ed3856e7bf01d7da3ba P f052d5cbee65406593d8e44f74225bc2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:02:39.156754 9084 catalog_manager.cc:2148] created HMS catalog entry for table [id=5357ddcf21634f99bb4f9abc5b6c4d80]
I20260926 15:02:39.172576 9325 tablet_service.cc:1467] Processing CreateTablet for tablet 570182804db146ccacf0150199c5d2be (DEFAULT_TABLE table=default.table_d2243274908043e58210a197141af813 [id=5357ddcf21634f99bb4f9abc5b6c4d80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:39.173166 9325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 570182804db146ccacf0150199c5d2be. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 15:02:39.175279 9541 tablet.cc:2369] T bb17748dce784ed3856e7bf01d7da3ba P ae330d635b1848aaaecee2c7ce96520a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:02:39.176556 9467 tablet_service.cc:1467] Processing CreateTablet for tablet 570182804db146ccacf0150199c5d2be (DEFAULT_TABLE table=default.table_d2243274908043e58210a197141af813 [id=5357ddcf21634f99bb4f9abc5b6c4d80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:39.177060 9467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 570182804db146ccacf0150199c5d2be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:39.180676 9084 master_service.cc:946] ReplaceTablet: received request to replace tablet bb17748dce784ed3856e7bf01d7da3ba from {username='slave'} at 127.0.0.1:59430
I20260926 15:02:39.186194 9612 tablet_service.cc:1467] Processing CreateTablet for tablet 570182804db146ccacf0150199c5d2be (DEFAULT_TABLE table=default.table_d2243274908043e58210a197141af813 [id=5357ddcf21634f99bb4f9abc5b6c4d80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:39.186738 9612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 570182804db146ccacf0150199c5d2be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:39.200317 9084 catalog_manager.cc:5782] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42: Sending DeleteTablet for 3 replicas of tablet bb17748dce784ed3856e7bf01d7da3ba
I20260926 15:02:39.201577 9611 tablet_service.cc:1514] Processing DeleteTablet for tablet bb17748dce784ed3856e7bf01d7da3ba with delete_type TABLET_DATA_DELETED (replaced by tablet 2f0bbbe1ae394215b43195ad9662c9bb) from {username='slave'} at 127.0.0.1:55968
I20260926 15:02:39.202076 9751 tablet_bootstrap.cc:492] T 570182804db146ccacf0150199c5d2be P 66ca148489d64deabd1dcbf7870c6d57: Bootstrap starting.
I20260926 15:02:39.204514 9751 tablet_bootstrap.cc:654] T 570182804db146ccacf0150199c5d2be P 66ca148489d64deabd1dcbf7870c6d57: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:39.208267 9325 tablet_service.cc:1514] Processing DeleteTablet for tablet bb17748dce784ed3856e7bf01d7da3ba with delete_type TABLET_DATA_DELETED (replaced by tablet 2f0bbbe1ae394215b43195ad9662c9bb) from {username='slave'} at 127.0.0.1:48112
I20260926 15:02:39.209241 9084 catalog_manager.cc:6565] ReplaceTablet: tablet bb17748dce784ed3856e7bf01d7da3ba deleted and replaced by tablet 2f0bbbe1ae394215b43195ad9662c9bb
I20260926 15:02:39.210026 9466 tablet_service.cc:1514] Processing DeleteTablet for tablet bb17748dce784ed3856e7bf01d7da3ba with delete_type TABLET_DATA_DELETED (replaced by tablet 2f0bbbe1ae394215b43195ad9662c9bb) from {username='slave'} at 127.0.0.1:58558
I20260926 15:02:39.211722 9084 master_service.cc:946] ReplaceTablet: received request to replace tablet bb17748dce784ed3856e7bf01d7da3ba from {username='slave'} at 127.0.0.1:59430
I20260926 15:02:39.216825 9084 master_service.cc:946] ReplaceTablet: received request to replace tablet bb17748dce784ed3856e7bf01d7da3ba from {username='slave'} at 127.0.0.1:59430
I20260926 15:02:39.217958 9750 tablet_bootstrap.cc:492] T 570182804db146ccacf0150199c5d2be P ae330d635b1848aaaecee2c7ce96520a: Bootstrap starting.
I20260926 15:02:39.218953 9084 master_service.cc:946] ReplaceTablet: received request to replace tablet bb17748dce784ed3856e7bf01d7da3ba from {username='slave'} at 127.0.0.1:59430
I20260926 15:02:39.219482 9084 master_service.cc:946] ReplaceTablet: received request to replace tablet bb17748dce784ed3856e7bf01d7da3ba from {username='slave'} at 127.0.0.1:59430
I20260926 15:02:39.219888 9750 tablet_bootstrap.cc:654] T 570182804db146ccacf0150199c5d2be P ae330d635b1848aaaecee2c7ce96520a: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:39.220192 9084 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59430:
name: "default.Table_a320fb1edf9a4372928664c4263b213e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:39.221503 9610 tablet_service.cc:1467] Processing CreateTablet for tablet 2f0bbbe1ae394215b43195ad9662c9bb (DEFAULT_TABLE table=default.table_41a3ce3c89684cbb9054c4251a0cc51c [id=7cc0c844d49c4ff08cef9e2c410991f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:39.221935 9465 tablet_service.cc:1467] Processing CreateTablet for tablet 2f0bbbe1ae394215b43195ad9662c9bb (DEFAULT_TABLE table=default.table_41a3ce3c89684cbb9054c4251a0cc51c [id=7cc0c844d49c4ff08cef9e2c410991f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:39.222105 9610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f0bbbe1ae394215b43195ad9662c9bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:39.222204 9324 tablet_service.cc:1467] Processing CreateTablet for tablet 2f0bbbe1ae394215b43195ad9662c9bb (DEFAULT_TABLE table=default.table_41a3ce3c89684cbb9054c4251a0cc51c [id=7cc0c844d49c4ff08cef9e2c410991f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:39.222455 9465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f0bbbe1ae394215b43195ad9662c9bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:39.222693 9324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f0bbbe1ae394215b43195ad9662c9bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:39.234786 9750 tablet_bootstrap.cc:492] T 570182804db146ccacf0150199c5d2be P ae330d635b1848aaaecee2c7ce96520a: No bootstrap required, opened a new log
I20260926 15:02:39.235082 9750 ts_tablet_manager.cc:1397] T 570182804db146ccacf0150199c5d2be P ae330d635b1848aaaecee2c7ce96520a: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.008s
I20260926 15:02:39.236454 9750 raft_consensus.cc:348] T 570182804db146ccacf0150199c5d2be P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } }
I20260926 15:02:39.236963 9750 raft_consensus.cc:374] T 570182804db146ccacf0150199c5d2be P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:39.237105 9750 raft_consensus.cc:729] T 570182804db146ccacf0150199c5d2be P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae330d635b1848aaaecee2c7ce96520a, State: Initialized, Role: FOLLOWER
I20260926 15:02:39.237448 9750 consensus_queue.cc:260] T 570182804db146ccacf0150199c5d2be P ae330d635b1848aaaecee2c7ce96520a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } }
I20260926 15:02:39.238258 9750 ts_tablet_manager.cc:1428] T 570182804db146ccacf0150199c5d2be P ae330d635b1848aaaecee2c7ce96520a: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260926 15:02:39.239940 9750 tablet_bootstrap.cc:492] T 2f0bbbe1ae394215b43195ad9662c9bb P ae330d635b1848aaaecee2c7ce96520a: Bootstrap starting.
I20260926 15:02:39.245311 9762 tablet_replica.cc:330] T bb17748dce784ed3856e7bf01d7da3ba P ae330d635b1848aaaecee2c7ce96520a: stopping tablet replica
I20260926 15:02:39.245963 9762 raft_consensus.cc:2229] T bb17748dce784ed3856e7bf01d7da3ba P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 15:02:39.247097 9754 tablet_bootstrap.cc:492] T 2f0bbbe1ae394215b43195ad9662c9bb P f052d5cbee65406593d8e44f74225bc2: Bootstrap starting.
I20260926 15:02:39.247356 9750 tablet_bootstrap.cc:654] T 2f0bbbe1ae394215b43195ad9662c9bb P ae330d635b1848aaaecee2c7ce96520a: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:39.248602 9761 tablet_replica.cc:330] T bb17748dce784ed3856e7bf01d7da3ba P 66ca148489d64deabd1dcbf7870c6d57: stopping tablet replica
I20260926 15:02:39.248773 9762 raft_consensus.cc:2258] T bb17748dce784ed3856e7bf01d7da3ba P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:39.249213 9761 raft_consensus.cc:2229] T bb17748dce784ed3856e7bf01d7da3ba P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 15:02:39.253512 9754 tablet_bootstrap.cc:654] T 2f0bbbe1ae394215b43195ad9662c9bb P f052d5cbee65406593d8e44f74225bc2: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:39.254292 9761 raft_consensus.cc:2258] T bb17748dce784ed3856e7bf01d7da3ba P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:39.259614 9088 catalog_manager.cc:2148] created HMS catalog entry for table [id=08b331ce5f9c44389fae209bac801f42]
I20260926 15:02:39.260039 9762 ts_tablet_manager.cc:1905] T bb17748dce784ed3856e7bf01d7da3ba P ae330d635b1848aaaecee2c7ce96520a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:39.262527 9761 ts_tablet_manager.cc:1905] T bb17748dce784ed3856e7bf01d7da3ba P 66ca148489d64deabd1dcbf7870c6d57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:39.264701 9759 tablet_replica.cc:330] T bb17748dce784ed3856e7bf01d7da3ba P f052d5cbee65406593d8e44f74225bc2: stopping tablet replica
I20260926 15:02:39.265235 9759 raft_consensus.cc:2229] T bb17748dce784ed3856e7bf01d7da3ba P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 15:02:39.265715 9759 raft_consensus.cc:2258] T bb17748dce784ed3856e7bf01d7da3ba P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:39.268769 9750 tablet_bootstrap.cc:492] T 2f0bbbe1ae394215b43195ad9662c9bb P ae330d635b1848aaaecee2c7ce96520a: No bootstrap required, opened a new log
I20260926 15:02:39.268985 9750 ts_tablet_manager.cc:1397] T 2f0bbbe1ae394215b43195ad9662c9bb P ae330d635b1848aaaecee2c7ce96520a: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.002s
I20260926 15:02:39.272542 9088 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:59430 (request call id 6) took 1021 ms. Trace:
I20260926 15:02:39.272678 9088 rpcz_store.cc:276] 0926 15:02:38.251359 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 15:02:38.252366 (+ 1007us) service_pool.cc:227] Handling call
0926 15:02:38.268971 (+ 16605us) catalog_manager.cc:2075] Acquired catalog manager lock
0926 15:02:38.270985 (+ 2014us) catalog_manager.cc:2121] Created new table and tablet info
0926 15:02:39.259576 (+988591us) catalog_manager.cc:2147] Created new table in HMS catalog
0926 15:02:39.272230 (+ 12654us) catalog_manager.cc:2181] Wrote table and tablets to system table
0926 15:02:39.272371 (+ 141us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0926 15:02:39.272476 (+ 105us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 15:02:39.261549 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 15:02:39.261727 (+ 178us) write_op.cc:432] Acquiring schema lock in shared mode
0926 15:02:39.261733 (+ 6us) write_op.cc:435] Acquired schema lock
0926 15:02:39.261736 (+ 3us) tablet.cc:660] Decoding operations
0926 15:02:39.261913 (+ 177us) write_op.cc:620] Acquiring the partition lock for write op
0926 15:02:39.261959 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0926 15:02:39.261966 (+ 7us) tablet.cc:683] Acquiring locks for 2 operations
0926 15:02:39.262095 (+ 129us) tablet.cc:699] Row locks acquired
0926 15:02:39.262126 (+ 31us) write_op.cc:260] PREPARE: finished
0926 15:02:39.262170 (+ 44us) write_op.cc:270] Start()
0926 15:02:39.262239 (+ 69us) write_op.cc:276] Timestamp: P: 1790434959262108 usec, L: 0
0926 15:02:39.262244 (+ 5us) op_driver.cc:331] REPLICATION: starting
0926 15:02:39.263641 (+ 1397us) log.cc:842] Serialized 432 byte log entry
0926 15:02:39.271259 (+ 7618us) op_driver.cc:444] REPLICATION: finished
0926 15:02:39.271384 (+ 125us) write_op.cc:301] APPLY: starting
0926 15:02:39.271405 (+ 21us) tablet.cc:1365] starting BulkCheckPresence
0926 15:02:39.271453 (+ 48us) tablet.cc:1368] finished BulkCheckPresence
0926 15:02:39.271481 (+ 28us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 15:02:39.271591 (+ 110us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 15:02:39.271628 (+ 37us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0926 15:02:39.271635 (+ 7us) write_op.cc:312] APPLY: finished
0926 15:02:39.271719 (+ 84us) log.cc:842] Serialized 28 byte log entry
0926 15:02:39.271788 (+ 69us) write_op.cc:489] Releasing partition, row and schema locks
0926 15:02:39.271811 (+ 23us) write_op.cc:454] Released schema lock
0926 15:02:39.271838 (+ 27us) write_op.cc:341] FINISH: Updating metrics
0926 15:02:39.271915 (+ 77us) write_op.cc:489] Releasing partition, row and schema locks
0926 15:02:39.271920 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":885625,"Hive Metastore.run_cpu_time_us":314,"Hive Metastore.run_wall_time_us":102744,"mutex_wait_us":414,"spinlock_wait_cycles":302208,"child_traces":[["op",{"apply.queue_time_us":46,"apply.run_cpu_time_us":598,"apply.run_wall_time_us":598,"num_ops":2,"prepare.queue_time_us":89,"prepare.run_cpu_time_us":1082,"prepare.run_wall_time_us":5666,"raft.queue_time_us":477,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":280,"replication_time_us":8989}]]}
I20260926 15:02:39.274480 9750 raft_consensus.cc:348] T 2f0bbbe1ae394215b43195ad9662c9bb P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:39.274772 9750 raft_consensus.cc:374] T 2f0bbbe1ae394215b43195ad9662c9bb P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:39.274870 9750 raft_consensus.cc:729] T 2f0bbbe1ae394215b43195ad9662c9bb P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae330d635b1848aaaecee2c7ce96520a, State: Initialized, Role: FOLLOWER
I20260926 15:02:39.275120 9750 consensus_queue.cc:260] T 2f0bbbe1ae394215b43195ad9662c9bb P ae330d635b1848aaaecee2c7ce96520a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:39.281718 9751 tablet_bootstrap.cc:492] T 570182804db146ccacf0150199c5d2be P 66ca148489d64deabd1dcbf7870c6d57: No bootstrap required, opened a new log
I20260926 15:02:39.283418 9751 ts_tablet_manager.cc:1397] T 570182804db146ccacf0150199c5d2be P 66ca148489d64deabd1dcbf7870c6d57: Time spent bootstrapping tablet: real 0.081s user 0.011s sys 0.017s
I20260926 15:02:39.286578 9466 tablet_service.cc:1467] Processing CreateTablet for tablet c5d07d3a7a164d098e61f8f7f2ac4a67 (DEFAULT_TABLE table=default.table_43176f08d9e24974abdd53171465a15c [id=08b331ce5f9c44389fae209bac801f42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:39.287266 9751 raft_consensus.cc:348] T 570182804db146ccacf0150199c5d2be P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } }
I20260926 15:02:39.288355 9751 raft_consensus.cc:374] T 570182804db146ccacf0150199c5d2be P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:39.288539 9751 raft_consensus.cc:729] T 570182804db146ccacf0150199c5d2be P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ca148489d64deabd1dcbf7870c6d57, State: Initialized, Role: FOLLOWER
I20260926 15:02:39.289024 9754 tablet_bootstrap.cc:492] T 2f0bbbe1ae394215b43195ad9662c9bb P f052d5cbee65406593d8e44f74225bc2: No bootstrap required, opened a new log
I20260926 15:02:39.288887 9751 consensus_queue.cc:260] T 570182804db146ccacf0150199c5d2be P 66ca148489d64deabd1dcbf7870c6d57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } }
I20260926 15:02:39.289278 9754 ts_tablet_manager.cc:1397] T 2f0bbbe1ae394215b43195ad9662c9bb P f052d5cbee65406593d8e44f74225bc2: Time spent bootstrapping tablet: real 0.042s user 0.006s sys 0.002s
I20260926 15:02:39.289497 9751 ts_tablet_manager.cc:1428] T 570182804db146ccacf0150199c5d2be P 66ca148489d64deabd1dcbf7870c6d57: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260926 15:02:39.289870 9751 tablet_bootstrap.cc:492] T 2f0bbbe1ae394215b43195ad9662c9bb P 66ca148489d64deabd1dcbf7870c6d57: Bootstrap starting.
I20260926 15:02:39.290752 9088 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:59430:
table { table_name: "default.Table_d2243274908043e58210a197141af813" } new_table_name: "default.Table_42b67a58948144f5a3b3abe972e975ba" modify_external_catalogs: true
I20260926 15:02:39.291538 9324 tablet_service.cc:1467] Processing CreateTablet for tablet c5d07d3a7a164d098e61f8f7f2ac4a67 (DEFAULT_TABLE table=default.table_43176f08d9e24974abdd53171465a15c [id=08b331ce5f9c44389fae209bac801f42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:39.292088 9611 tablet_service.cc:1467] Processing CreateTablet for tablet c5d07d3a7a164d098e61f8f7f2ac4a67 (DEFAULT_TABLE table=default.table_43176f08d9e24974abdd53171465a15c [id=08b331ce5f9c44389fae209bac801f42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:39.292619 9611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5d07d3a7a164d098e61f8f7f2ac4a67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:39.292865 9754 raft_consensus.cc:348] T 2f0bbbe1ae394215b43195ad9662c9bb P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:39.293053 9759 ts_tablet_manager.cc:1905] T bb17748dce784ed3856e7bf01d7da3ba P f052d5cbee65406593d8e44f74225bc2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:39.293102 9754 raft_consensus.cc:374] T 2f0bbbe1ae394215b43195ad9662c9bb P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:39.293244 9754 raft_consensus.cc:729] T 2f0bbbe1ae394215b43195ad9662c9bb P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f052d5cbee65406593d8e44f74225bc2, State: Initialized, Role: FOLLOWER
I20260926 15:02:39.293586 9754 consensus_queue.cc:260] T 2f0bbbe1ae394215b43195ad9662c9bb P f052d5cbee65406593d8e44f74225bc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:39.294171 9754 ts_tablet_manager.cc:1428] T 2f0bbbe1ae394215b43195ad9662c9bb P f052d5cbee65406593d8e44f74225bc2: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260926 15:02:39.294482 9754 tablet_bootstrap.cc:492] T 570182804db146ccacf0150199c5d2be P f052d5cbee65406593d8e44f74225bc2: Bootstrap starting.
I20260926 15:02:39.291890 9751 tablet_bootstrap.cc:654] T 2f0bbbe1ae394215b43195ad9662c9bb P 66ca148489d64deabd1dcbf7870c6d57: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:39.296239 9754 tablet_bootstrap.cc:654] T 570182804db146ccacf0150199c5d2be P f052d5cbee65406593d8e44f74225bc2: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:39.296272 9750 ts_tablet_manager.cc:1428] T 2f0bbbe1ae394215b43195ad9662c9bb P ae330d635b1848aaaecee2c7ce96520a: Time spent starting tablet: real 0.027s user 0.001s sys 0.001s
I20260926 15:02:39.304880 9754 tablet_bootstrap.cc:492] T 570182804db146ccacf0150199c5d2be P f052d5cbee65406593d8e44f74225bc2: No bootstrap required, opened a new log
I20260926 15:02:39.308827 9754 ts_tablet_manager.cc:1397] T 570182804db146ccacf0150199c5d2be P f052d5cbee65406593d8e44f74225bc2: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.000s
I20260926 15:02:39.309703 9754 raft_consensus.cc:348] T 570182804db146ccacf0150199c5d2be P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } }
I20260926 15:02:39.310765 9754 raft_consensus.cc:374] T 570182804db146ccacf0150199c5d2be P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:39.310918 9754 raft_consensus.cc:729] T 570182804db146ccacf0150199c5d2be P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f052d5cbee65406593d8e44f74225bc2, State: Initialized, Role: FOLLOWER
I20260926 15:02:39.311707 9754 consensus_queue.cc:260] T 570182804db146ccacf0150199c5d2be P f052d5cbee65406593d8e44f74225bc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } }
I20260926 15:02:39.315739 9754 ts_tablet_manager.cc:1428] T 570182804db146ccacf0150199c5d2be P f052d5cbee65406593d8e44f74225bc2: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260926 15:02:39.319617 9754 tablet_bootstrap.cc:492] T c5d07d3a7a164d098e61f8f7f2ac4a67 P f052d5cbee65406593d8e44f74225bc2: Bootstrap starting.
I20260926 15:02:39.321065 9751 tablet_bootstrap.cc:492] T 2f0bbbe1ae394215b43195ad9662c9bb P 66ca148489d64deabd1dcbf7870c6d57: No bootstrap required, opened a new log
I20260926 15:02:39.321376 9751 ts_tablet_manager.cc:1397] T 2f0bbbe1ae394215b43195ad9662c9bb P 66ca148489d64deabd1dcbf7870c6d57: Time spent bootstrapping tablet: real 0.032s user 0.013s sys 0.007s
I20260926 15:02:39.321959 9754 tablet_bootstrap.cc:654] T c5d07d3a7a164d098e61f8f7f2ac4a67 P f052d5cbee65406593d8e44f74225bc2: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:39.322397 9751 raft_consensus.cc:348] T 2f0bbbe1ae394215b43195ad9662c9bb P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:39.322702 9751 raft_consensus.cc:374] T 2f0bbbe1ae394215b43195ad9662c9bb P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:39.322852 9751 raft_consensus.cc:729] T 2f0bbbe1ae394215b43195ad9662c9bb P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ca148489d64deabd1dcbf7870c6d57, State: Initialized, Role: FOLLOWER
I20260926 15:02:39.323292 9751 consensus_queue.cc:260] T 2f0bbbe1ae394215b43195ad9662c9bb P 66ca148489d64deabd1dcbf7870c6d57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:39.324559 9751 ts_tablet_manager.cc:1428] T 2f0bbbe1ae394215b43195ad9662c9bb P 66ca148489d64deabd1dcbf7870c6d57: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260926 15:02:39.334643 9754 tablet_bootstrap.cc:492] T c5d07d3a7a164d098e61f8f7f2ac4a67 P f052d5cbee65406593d8e44f74225bc2: No bootstrap required, opened a new log
I20260926 15:02:39.334924 9754 ts_tablet_manager.cc:1397] T c5d07d3a7a164d098e61f8f7f2ac4a67 P f052d5cbee65406593d8e44f74225bc2: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.004s
I20260926 15:02:39.335819 9754 raft_consensus.cc:348] T c5d07d3a7a164d098e61f8f7f2ac4a67 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:39.336189 9754 raft_consensus.cc:374] T c5d07d3a7a164d098e61f8f7f2ac4a67 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:39.336340 9754 raft_consensus.cc:729] T c5d07d3a7a164d098e61f8f7f2ac4a67 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f052d5cbee65406593d8e44f74225bc2, State: Initialized, Role: FOLLOWER
I20260926 15:02:39.336664 9754 consensus_queue.cc:260] T c5d07d3a7a164d098e61f8f7f2ac4a67 P f052d5cbee65406593d8e44f74225bc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:39.339821 9754 ts_tablet_manager.cc:1428] T c5d07d3a7a164d098e61f8f7f2ac4a67 P f052d5cbee65406593d8e44f74225bc2: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260926 15:02:39.364305 9466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5d07d3a7a164d098e61f8f7f2ac4a67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:39.367324 9759 ts_tablet_manager.cc:1918] T bb17748dce784ed3856e7bf01d7da3ba P f052d5cbee65406593d8e44f74225bc2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 15:02:39.367471 9759 log.cc:1196] T bb17748dce784ed3856e7bf01d7da3ba P f052d5cbee65406593d8e44f74225bc2: Deleting WAL directory at /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-2/wal/wals/bb17748dce784ed3856e7bf01d7da3ba
I20260926 15:02:39.367808 9759 ts_tablet_manager.cc:1939] T bb17748dce784ed3856e7bf01d7da3ba P f052d5cbee65406593d8e44f74225bc2: Deleting consensus metadata
I20260926 15:02:39.368081 9324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5d07d3a7a164d098e61f8f7f2ac4a67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:39.377902 9751 tablet_bootstrap.cc:492] T c5d07d3a7a164d098e61f8f7f2ac4a67 P 66ca148489d64deabd1dcbf7870c6d57: Bootstrap starting.
I20260926 15:02:39.379393 9751 tablet_bootstrap.cc:654] T c5d07d3a7a164d098e61f8f7f2ac4a67 P 66ca148489d64deabd1dcbf7870c6d57: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:39.381943 9751 tablet_bootstrap.cc:492] T c5d07d3a7a164d098e61f8f7f2ac4a67 P 66ca148489d64deabd1dcbf7870c6d57: No bootstrap required, opened a new log
I20260926 15:02:39.382139 9751 ts_tablet_manager.cc:1397] T c5d07d3a7a164d098e61f8f7f2ac4a67 P 66ca148489d64deabd1dcbf7870c6d57: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260926 15:02:39.382822 9751 raft_consensus.cc:348] T c5d07d3a7a164d098e61f8f7f2ac4a67 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:39.383076 9751 raft_consensus.cc:374] T c5d07d3a7a164d098e61f8f7f2ac4a67 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:39.383276 9751 raft_consensus.cc:729] T c5d07d3a7a164d098e61f8f7f2ac4a67 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ca148489d64deabd1dcbf7870c6d57, State: Initialized, Role: FOLLOWER
I20260926 15:02:39.383541 9751 consensus_queue.cc:260] T c5d07d3a7a164d098e61f8f7f2ac4a67 P 66ca148489d64deabd1dcbf7870c6d57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:39.384323 9751 ts_tablet_manager.cc:1428] T c5d07d3a7a164d098e61f8f7f2ac4a67 P 66ca148489d64deabd1dcbf7870c6d57: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260926 15:02:39.426787 9740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:39.427233 9740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:39.427392 9740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260926 15:02:39.428956 9072 catalog_manager.cc:4841] TS f052d5cbee65406593d8e44f74225bc2 (127.8.138.131:35445): tablet bb17748dce784ed3856e7bf01d7da3ba (table default.table_41a3ce3c89684cbb9054c4251a0cc51c [id=7cc0c844d49c4ff08cef9e2c410991f8]) successfully deleted
W20260926 15:02:39.443792 9740 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:02:39.444027 9740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:39.444114 9740 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:02:39.444196 9740 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:02:39.458639 9762 ts_tablet_manager.cc:1918] T bb17748dce784ed3856e7bf01d7da3ba P ae330d635b1848aaaecee2c7ce96520a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 15:02:39.458877 9762 log.cc:1196] T bb17748dce784ed3856e7bf01d7da3ba P ae330d635b1848aaaecee2c7ce96520a: Deleting WAL directory at /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-1/wal/wals/bb17748dce784ed3856e7bf01d7da3ba
I20260926 15:02:39.459333 9762 ts_tablet_manager.cc:1939] T bb17748dce784ed3856e7bf01d7da3ba P ae330d635b1848aaaecee2c7ce96520a: Deleting consensus metadata
I20260926 15:02:39.461511 9761 ts_tablet_manager.cc:1918] T bb17748dce784ed3856e7bf01d7da3ba P 66ca148489d64deabd1dcbf7870c6d57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 15:02:39.461685 9761 log.cc:1196] T bb17748dce784ed3856e7bf01d7da3ba P 66ca148489d64deabd1dcbf7870c6d57: Deleting WAL directory at /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-0/wal/wals/bb17748dce784ed3856e7bf01d7da3ba
I20260926 15:02:39.462116 9761 ts_tablet_manager.cc:1939] T bb17748dce784ed3856e7bf01d7da3ba P 66ca148489d64deabd1dcbf7870c6d57: Deleting consensus metadata
I20260926 15:02:39.462605 9074 catalog_manager.cc:4841] TS ae330d635b1848aaaecee2c7ce96520a (127.8.138.130:44309): tablet bb17748dce784ed3856e7bf01d7da3ba (table default.table_41a3ce3c89684cbb9054c4251a0cc51c [id=7cc0c844d49c4ff08cef9e2c410991f8]) successfully deleted
I20260926 15:02:39.464605 9740 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.138.148:33835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:40047
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.138.190:44281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.138.190
--webserver_port=37041
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:39.465626 9750 tablet_bootstrap.cc:492] T c5d07d3a7a164d098e61f8f7f2ac4a67 P ae330d635b1848aaaecee2c7ce96520a: Bootstrap starting.
I20260926 15:02:39.466904 9740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:39.467367 9750 tablet_bootstrap.cc:654] T c5d07d3a7a164d098e61f8f7f2ac4a67 P ae330d635b1848aaaecee2c7ce96520a: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:39.468565 9087 catalog_manager.cc:2148] created HMS catalog entry for table [id=630deebc15b04878ae13844ff2808f52]
I20260926 15:02:39.469630 9740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 15:02:39.475430 9087 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:59430 (request call id 5) took 1224 ms. Trace:
I20260926 15:02:39.475596 9087 rpcz_store.cc:276] 0926 15:02:38.251151 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 15:02:38.251241 (+ 90us) service_pool.cc:227] Handling call
0926 15:02:38.271485 (+ 20244us) catalog_manager.cc:2075] Acquired catalog manager lock
0926 15:02:38.271982 (+ 497us) catalog_manager.cc:2121] Created new table and tablet info
0926 15:02:39.468514 (+1196532us) catalog_manager.cc:2147] Created new table in HMS catalog
0926 15:02:39.475134 (+ 6620us) catalog_manager.cc:2181] Wrote table and tablets to system table
0926 15:02:39.475279 (+ 145us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0926 15:02:39.475387 (+ 108us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 15:02:39.469290 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 15:02:39.469398 (+ 108us) write_op.cc:432] Acquiring schema lock in shared mode
0926 15:02:39.469404 (+ 6us) write_op.cc:435] Acquired schema lock
0926 15:02:39.469406 (+ 2us) tablet.cc:660] Decoding operations
0926 15:02:39.469540 (+ 134us) write_op.cc:620] Acquiring the partition lock for write op
0926 15:02:39.469561 (+ 21us) write_op.cc:641] Partition lock acquired for write op
0926 15:02:39.469599 (+ 38us) tablet.cc:683] Acquiring locks for 2 operations
0926 15:02:39.469737 (+ 138us) tablet.cc:699] Row locks acquired
0926 15:02:39.469742 (+ 5us) write_op.cc:260] PREPARE: finished
0926 15:02:39.469789 (+ 47us) write_op.cc:270] Start()
0926 15:02:39.469833 (+ 44us) write_op.cc:276] Timestamp: P: 1790434959469706 usec, L: 0
0926 15:02:39.469839 (+ 6us) op_driver.cc:331] REPLICATION: starting
0926 15:02:39.470086 (+ 247us) log.cc:842] Serialized 432 byte log entry
0926 15:02:39.474107 (+ 4021us) op_driver.cc:444] REPLICATION: finished
0926 15:02:39.474251 (+ 144us) write_op.cc:301] APPLY: starting
0926 15:02:39.474268 (+ 17us) tablet.cc:1365] starting BulkCheckPresence
0926 15:02:39.474316 (+ 48us) tablet.cc:1368] finished BulkCheckPresence
0926 15:02:39.474351 (+ 35us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 15:02:39.474457 (+ 106us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 15:02:39.474494 (+ 37us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0926 15:02:39.474500 (+ 6us) write_op.cc:312] APPLY: finished
0926 15:02:39.474587 (+ 87us) log.cc:842] Serialized 28 byte log entry
0926 15:02:39.474657 (+ 70us) write_op.cc:489] Releasing partition, row and schema locks
0926 15:02:39.474680 (+ 23us) write_op.cc:454] Released schema lock
0926 15:02:39.474687 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0926 15:02:39.474775 (+ 88us) write_op.cc:489] Releasing partition, row and schema locks
0926 15:02:39.474779 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":987645,"Hive Metastore.run_cpu_time_us":377,"Hive Metastore.run_wall_time_us":208476,"child_traces":[["op",{"apply.queue_time_us":96,"apply.run_cpu_time_us":583,"apply.run_wall_time_us":582,"num_ops":2,"prepare.queue_time_us":59,"prepare.run_cpu_time_us":932,"prepare.run_wall_time_us":932,"raft.queue_time_us":1473,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":299,"replication_time_us":4230}]]}
I20260926 15:02:39.484184 9750 tablet_bootstrap.cc:492] T c5d07d3a7a164d098e61f8f7f2ac4a67 P ae330d635b1848aaaecee2c7ce96520a: No bootstrap required, opened a new log
I20260926 15:02:39.484975 9750 ts_tablet_manager.cc:1397] T c5d07d3a7a164d098e61f8f7f2ac4a67 P ae330d635b1848aaaecee2c7ce96520a: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.000s
I20260926 15:02:39.486109 9750 raft_consensus.cc:348] T c5d07d3a7a164d098e61f8f7f2ac4a67 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:39.486471 9750 raft_consensus.cc:374] T c5d07d3a7a164d098e61f8f7f2ac4a67 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:39.487016 9750 raft_consensus.cc:729] T c5d07d3a7a164d098e61f8f7f2ac4a67 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae330d635b1848aaaecee2c7ce96520a, State: Initialized, Role: FOLLOWER
I20260926 15:02:39.487370 9750 consensus_queue.cc:260] T c5d07d3a7a164d098e61f8f7f2ac4a67 P ae330d635b1848aaaecee2c7ce96520a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:39.488031 9750 ts_tablet_manager.cc:1428] T c5d07d3a7a164d098e61f8f7f2ac4a67 P ae330d635b1848aaaecee2c7ce96520a: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260926 15:02:39.497294 9611 tablet_service.cc:1467] Processing CreateTablet for tablet 3fae8a76002849de9bb26610a0c33406 (DEFAULT_TABLE table=default.table_95f48725b6ce47fab5f96e18ccb69dbc [id=630deebc15b04878ae13844ff2808f52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:39.497870 9611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fae8a76002849de9bb26610a0c33406. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:39.498482 9087 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:59430:
table { table_name: "default.Table_43176f08d9e24974abdd53171465a15c" } new_table_name: "default.Table_1b2fcfff41b54f21b229a3e2064d7c26" modify_external_catalogs: true
W20260926 15:02:39.500375 9775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:39.501880 9466 tablet_service.cc:1467] Processing CreateTablet for tablet 3fae8a76002849de9bb26610a0c33406 (DEFAULT_TABLE table=default.table_95f48725b6ce47fab5f96e18ccb69dbc [id=630deebc15b04878ae13844ff2808f52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:39.502730 9466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fae8a76002849de9bb26610a0c33406. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:39.503150 9324 tablet_service.cc:1467] Processing CreateTablet for tablet 3fae8a76002849de9bb26610a0c33406 (DEFAULT_TABLE table=default.table_95f48725b6ce47fab5f96e18ccb69dbc [id=630deebc15b04878ae13844ff2808f52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:39.503587 9324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fae8a76002849de9bb26610a0c33406. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:39.509518 9750 tablet_bootstrap.cc:492] T 3fae8a76002849de9bb26610a0c33406 P ae330d635b1848aaaecee2c7ce96520a: Bootstrap starting.
I20260926 15:02:39.511329 9750 tablet_bootstrap.cc:654] T 3fae8a76002849de9bb26610a0c33406 P ae330d635b1848aaaecee2c7ce96520a: Neither blocks nor log segments found. Creating new log.
W20260926 15:02:39.514683 9776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:39.522698 9751 tablet_bootstrap.cc:492] T 3fae8a76002849de9bb26610a0c33406 P 66ca148489d64deabd1dcbf7870c6d57: Bootstrap starting.
I20260926 15:02:39.524552 9751 tablet_bootstrap.cc:654] T 3fae8a76002849de9bb26610a0c33406 P 66ca148489d64deabd1dcbf7870c6d57: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:39.525951 9754 tablet_bootstrap.cc:492] T 3fae8a76002849de9bb26610a0c33406 P f052d5cbee65406593d8e44f74225bc2: Bootstrap starting.
I20260926 15:02:39.528343 9754 tablet_bootstrap.cc:654] T 3fae8a76002849de9bb26610a0c33406 P f052d5cbee65406593d8e44f74225bc2: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:39.531332 9073 catalog_manager.cc:4841] TS 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327): tablet bb17748dce784ed3856e7bf01d7da3ba (table default.table_41a3ce3c89684cbb9054c4251a0cc51c [id=7cc0c844d49c4ff08cef9e2c410991f8]) successfully deleted
I20260926 15:02:39.535713 9751 tablet_bootstrap.cc:492] T 3fae8a76002849de9bb26610a0c33406 P 66ca148489d64deabd1dcbf7870c6d57: No bootstrap required, opened a new log
I20260926 15:02:39.536192 9751 ts_tablet_manager.cc:1397] T 3fae8a76002849de9bb26610a0c33406 P 66ca148489d64deabd1dcbf7870c6d57: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.000s
I20260926 15:02:39.537323 9751 raft_consensus.cc:348] T 3fae8a76002849de9bb26610a0c33406 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:39.537752 9751 raft_consensus.cc:374] T 3fae8a76002849de9bb26610a0c33406 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:39.537894 9751 raft_consensus.cc:729] T 3fae8a76002849de9bb26610a0c33406 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ca148489d64deabd1dcbf7870c6d57, State: Initialized, Role: FOLLOWER
I20260926 15:02:39.538484 9751 consensus_queue.cc:260] T 3fae8a76002849de9bb26610a0c33406 P 66ca148489d64deabd1dcbf7870c6d57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:39.540195 9751 ts_tablet_manager.cc:1428] T 3fae8a76002849de9bb26610a0c33406 P 66ca148489d64deabd1dcbf7870c6d57: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260926 15:02:39.548877 9754 tablet_bootstrap.cc:492] T 3fae8a76002849de9bb26610a0c33406 P f052d5cbee65406593d8e44f74225bc2: No bootstrap required, opened a new log
I20260926 15:02:39.549182 9754 ts_tablet_manager.cc:1397] T 3fae8a76002849de9bb26610a0c33406 P f052d5cbee65406593d8e44f74225bc2: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.009s
I20260926 15:02:39.550312 9754 raft_consensus.cc:348] T 3fae8a76002849de9bb26610a0c33406 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:39.550661 9754 raft_consensus.cc:374] T 3fae8a76002849de9bb26610a0c33406 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:39.550763 9754 raft_consensus.cc:729] T 3fae8a76002849de9bb26610a0c33406 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f052d5cbee65406593d8e44f74225bc2, State: Initialized, Role: FOLLOWER
I20260926 15:02:39.551129 9754 consensus_queue.cc:260] T 3fae8a76002849de9bb26610a0c33406 P f052d5cbee65406593d8e44f74225bc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:39.552006 9754 ts_tablet_manager.cc:1428] T 3fae8a76002849de9bb26610a0c33406 P f052d5cbee65406593d8e44f74225bc2: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260926 15:02:39.552335 9750 tablet_bootstrap.cc:492] T 3fae8a76002849de9bb26610a0c33406 P ae330d635b1848aaaecee2c7ce96520a: No bootstrap required, opened a new log
I20260926 15:02:39.552518 9750 ts_tablet_manager.cc:1397] T 3fae8a76002849de9bb26610a0c33406 P ae330d635b1848aaaecee2c7ce96520a: Time spent bootstrapping tablet: real 0.043s user 0.001s sys 0.019s
I20260926 15:02:39.553462 9750 raft_consensus.cc:348] T 3fae8a76002849de9bb26610a0c33406 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:39.553766 9750 raft_consensus.cc:374] T 3fae8a76002849de9bb26610a0c33406 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:39.553936 9750 raft_consensus.cc:729] T 3fae8a76002849de9bb26610a0c33406 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae330d635b1848aaaecee2c7ce96520a, State: Initialized, Role: FOLLOWER
I20260926 15:02:39.554311 9750 consensus_queue.cc:260] T 3fae8a76002849de9bb26610a0c33406 P ae330d635b1848aaaecee2c7ce96520a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
W20260926 15:02:39.556452 9781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:39.556854 9750 ts_tablet_manager.cc:1428] T 3fae8a76002849de9bb26610a0c33406 P ae330d635b1848aaaecee2c7ce96520a: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260926 15:02:39.559657 9740 server_base.cc:1034] running on GCE node
I20260926 15:02:39.560823 9740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:39.562711 9740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:39.563983 9740 hybrid_clock.cc:648] HybridClock initialized: now 1790434959563912 us; error 30 us; skew 500 ppm
I20260926 15:02:39.564495 9740 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:02:39.576841 9740 webserver.cc:460] Webserver started at http://127.8.138.190:37041/ using document root <none> and password file <none>
I20260926 15:02:39.577674 9740 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:39.577804 9740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:39.584041 9740 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 15:02:39.587504 9787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:39.589160 9740 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 15:02:39.589380 9740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/data,/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/wal
uuid: "6315fb5099e1420b95b8d83b3e19375c"
format_stamp: "Formatted at 2026-09-26 15:02:34 on dist-test-slave-7qf4"
I20260926 15:02:39.590958 9740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:39.612473 9740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:39.613402 9740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:39.613693 9740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:39.635293 9740 rpc_server.cc:307] RPC server started. Bound to: 127.8.138.190:44281
I20260926 15:02:39.635315 9838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.138.190:44281 every 8 connection(s)
I20260926 15:02:39.637549 9740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/data/info.pb
I20260926 15:02:39.640287 8746 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu as pid 9740
I20260926 15:02:39.647750 9839 sys_catalog.cc:263] Verifying existing consensus state
I20260926 15:02:39.654915 9839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c: Bootstrap starting.
I20260926 15:02:39.666826 9839 log.cc:824] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c: Log is configured to *not* fsync() on all Append() calls
I20260926 15:02:39.680842 9839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260926 15:02:39.681725 9839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c: Bootstrap complete.
I20260926 15:02:39.697016 9839 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 3 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:39.699103 9839 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6315fb5099e1420b95b8d83b3e19375c, State: Initialized, Role: FOLLOWER
I20260926 15:02:39.700062 9839 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:39.702279 9844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:39.702950 9844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:02:39.703831 9839 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:02:39.704658 9814 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 3 FOLLOWER]: Advancing to term 4
I20260926 15:02:39.769456 9814 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 4 FOLLOWER]: Refusing update from remote peer be9958f2085242f7ab82ae933ceb1c42: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 2 index: 5. (term mismatch)
I20260926 15:02:39.769620 9814 pending_rounds.cc:76] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c: Aborting all ops after (but not including) 4
I20260926 15:02:39.769697 9814 pending_rounds.cc:97] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260926 15:02:39.769840 9814 raft_consensus.cc:2866] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260926 15:02:39.771720 9729 consensus_queue.cc:1037] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [LEADER]: Got LMP mismatch error from peer: Peer: permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260926 15:02:39.791774 9844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: SysCatalogTable state changed. Reason: New leader be9958f2085242f7ab82ae933ceb1c42. Latest consensus state: current_term: 4 leader_uuid: "be9958f2085242f7ab82ae933ceb1c42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:39.792167 9844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:02:39.793872 9857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "be9958f2085242f7ab82ae933ceb1c42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:39.794512 9857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:02:39.793921 9858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "be9958f2085242f7ab82ae933ceb1c42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:39.794780 9858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:02:39.800312 9855 mvcc.cc:205] Tried to move back new op lower bound from 7333621594034659328 to 7333621584910340096. Current Snapshot: MvccSnapshot[applied={T|T < 7333621594034659328}]
I20260926 15:02:39.819032 9758 raft_consensus.cc:482] T 570182804db146ccacf0150199c5d2be P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:02:39.819365 9758 raft_consensus.cc:504] T 570182804db146ccacf0150199c5d2be P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } }
I20260926 15:02:39.821556 9758 leader_election.cc:288] T 570182804db146ccacf0150199c5d2be P f052d5cbee65406593d8e44f74225bc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327), ae330d635b1848aaaecee2c7ce96520a (127.8.138.130:44309)
I20260926 15:02:39.825706 9865 catalog_manager.cc:1259] Loaded cluster ID: 497ed160bc39477e99bf3beb637a4391
I20260926 15:02:39.825817 9865 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c: loading cluster ID for follower catalog manager: success
I20260926 15:02:39.829938 9865 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c: acquiring CA information for follower catalog manager: success
I20260926 15:02:39.832623 9865 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 15:02:39.835157 9345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "570182804db146ccacf0150199c5d2be" candidate_uuid: "f052d5cbee65406593d8e44f74225bc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ca148489d64deabd1dcbf7870c6d57" is_pre_election: true
I20260926 15:02:39.835696 9345 raft_consensus.cc:2454] T 570182804db146ccacf0150199c5d2be P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f052d5cbee65406593d8e44f74225bc2 in term 0.
I20260926 15:02:39.836568 9566 leader_election.cc:302] T 570182804db146ccacf0150199c5d2be P f052d5cbee65406593d8e44f74225bc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ca148489d64deabd1dcbf7870c6d57, f052d5cbee65406593d8e44f74225bc2; no voters:
I20260926 15:02:39.837086 9758 raft_consensus.cc:2781] T 570182804db146ccacf0150199c5d2be P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:02:39.837271 9758 raft_consensus.cc:482] T 570182804db146ccacf0150199c5d2be P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:02:39.837384 9758 raft_consensus.cc:3037] T 570182804db146ccacf0150199c5d2be P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:39.837812 9487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "570182804db146ccacf0150199c5d2be" candidate_uuid: "f052d5cbee65406593d8e44f74225bc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae330d635b1848aaaecee2c7ce96520a" is_pre_election: true
I20260926 15:02:39.838335 9487 raft_consensus.cc:2454] T 570182804db146ccacf0150199c5d2be P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f052d5cbee65406593d8e44f74225bc2 in term 0.
I20260926 15:02:39.880584 9758 raft_consensus.cc:504] T 570182804db146ccacf0150199c5d2be P f052d5cbee65406593d8e44f74225bc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } }
I20260926 15:02:39.881515 9758 leader_election.cc:288] T 570182804db146ccacf0150199c5d2be P f052d5cbee65406593d8e44f74225bc2 [CANDIDATE]: Term 1 election: Requested vote from peers 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327), ae330d635b1848aaaecee2c7ce96520a (127.8.138.130:44309)
I20260926 15:02:39.882511 9487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "570182804db146ccacf0150199c5d2be" candidate_uuid: "f052d5cbee65406593d8e44f74225bc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae330d635b1848aaaecee2c7ce96520a"
I20260926 15:02:39.882777 9487 raft_consensus.cc:3037] T 570182804db146ccacf0150199c5d2be P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:39.886242 9758 raft_consensus.cc:482] T 3fae8a76002849de9bb26610a0c33406 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:02:39.886466 9345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "570182804db146ccacf0150199c5d2be" candidate_uuid: "f052d5cbee65406593d8e44f74225bc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ca148489d64deabd1dcbf7870c6d57"
I20260926 15:02:39.886426 9758 raft_consensus.cc:504] T 3fae8a76002849de9bb26610a0c33406 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:39.886729 9345 raft_consensus.cc:3037] T 570182804db146ccacf0150199c5d2be P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:39.887151 9758 leader_election.cc:288] T 3fae8a76002849de9bb26610a0c33406 P f052d5cbee65406593d8e44f74225bc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327), ae330d635b1848aaaecee2c7ce96520a (127.8.138.130:44309)
I20260926 15:02:39.887766 9344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fae8a76002849de9bb26610a0c33406" candidate_uuid: "f052d5cbee65406593d8e44f74225bc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ca148489d64deabd1dcbf7870c6d57" is_pre_election: true
I20260926 15:02:39.888141 9344 raft_consensus.cc:2454] T 3fae8a76002849de9bb26610a0c33406 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f052d5cbee65406593d8e44f74225bc2 in term 0.
I20260926 15:02:39.888424 9486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fae8a76002849de9bb26610a0c33406" candidate_uuid: "f052d5cbee65406593d8e44f74225bc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae330d635b1848aaaecee2c7ce96520a" is_pre_election: true
I20260926 15:02:39.888675 9566 leader_election.cc:302] T 3fae8a76002849de9bb26610a0c33406 P f052d5cbee65406593d8e44f74225bc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ca148489d64deabd1dcbf7870c6d57, f052d5cbee65406593d8e44f74225bc2; no voters:
I20260926 15:02:39.888712 9486 raft_consensus.cc:2454] T 3fae8a76002849de9bb26610a0c33406 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f052d5cbee65406593d8e44f74225bc2 in term 0.
I20260926 15:02:39.888947 9758 raft_consensus.cc:2781] T 3fae8a76002849de9bb26610a0c33406 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:02:39.889060 9758 raft_consensus.cc:482] T 3fae8a76002849de9bb26610a0c33406 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:02:39.889153 9758 raft_consensus.cc:3037] T 3fae8a76002849de9bb26610a0c33406 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:39.987771 9345 raft_consensus.cc:2454] T 570182804db146ccacf0150199c5d2be P 66ca148489d64deabd1dcbf7870c6d57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f052d5cbee65406593d8e44f74225bc2 in term 1.
I20260926 15:02:39.987778 9487 raft_consensus.cc:2454] T 570182804db146ccacf0150199c5d2be P ae330d635b1848aaaecee2c7ce96520a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f052d5cbee65406593d8e44f74225bc2 in term 1.
I20260926 15:02:39.988543 9566 leader_election.cc:302] T 570182804db146ccacf0150199c5d2be P f052d5cbee65406593d8e44f74225bc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ca148489d64deabd1dcbf7870c6d57, f052d5cbee65406593d8e44f74225bc2; no voters:
I20260926 15:02:39.993655 9873 raft_consensus.cc:2781] T 570182804db146ccacf0150199c5d2be P f052d5cbee65406593d8e44f74225bc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:02:39.996080 9758 raft_consensus.cc:504] T 3fae8a76002849de9bb26610a0c33406 P f052d5cbee65406593d8e44f74225bc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:39.998230 9487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fae8a76002849de9bb26610a0c33406" candidate_uuid: "f052d5cbee65406593d8e44f74225bc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae330d635b1848aaaecee2c7ce96520a"
I20260926 15:02:39.998673 9487 raft_consensus.cc:3037] T 3fae8a76002849de9bb26610a0c33406 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:40.000819 9345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fae8a76002849de9bb26610a0c33406" candidate_uuid: "f052d5cbee65406593d8e44f74225bc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ca148489d64deabd1dcbf7870c6d57"
I20260926 15:02:40.001256 9345 raft_consensus.cc:3037] T 3fae8a76002849de9bb26610a0c33406 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:40.004738 9873 raft_consensus.cc:686] T 570182804db146ccacf0150199c5d2be P f052d5cbee65406593d8e44f74225bc2 [term 1 LEADER]: Becoming Leader. State: Replica: f052d5cbee65406593d8e44f74225bc2, State: Running, Role: LEADER
I20260926 15:02:40.005432 9873 consensus_queue.cc:237] T 570182804db146ccacf0150199c5d2be P f052d5cbee65406593d8e44f74225bc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } }
I20260926 15:02:40.014635 9678 heartbeater.cc:347] Connected to a master server at 127.8.138.190:44281
I20260926 15:02:40.014876 9678 heartbeater.cc:503] Master 127.8.138.190:44281 was elected leader, sending a full tablet report...
I20260926 15:02:40.015887 9758 leader_election.cc:288] T 3fae8a76002849de9bb26610a0c33406 P f052d5cbee65406593d8e44f74225bc2 [CANDIDATE]: Term 1 election: Requested vote from peers 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327), ae330d635b1848aaaecee2c7ce96520a (127.8.138.130:44309)
I20260926 15:02:40.019023 9804 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" instance_seqno: 1790434957773527) as {username='slave'} at 127.8.138.131:58023; Asking this server to re-register.
I20260926 15:02:40.020691 9678 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:40.022053 9876 raft_consensus.cc:482] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader be9958f2085242f7ab82ae933ceb1c42)
I20260926 15:02:40.022293 9876 raft_consensus.cc:504] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:40.023015 9876 leader_election.cc:288] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105)
I20260926 15:02:40.023887 9098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "320cd09624624312bf1bcc0856c2874f" candidate_term: 5 candidate_status { last_received { term: 4 index: 16 } } ignore_live_leader: false dest_uuid: "be9958f2085242f7ab82ae933ceb1c42" is_pre_election: true
I20260926 15:02:40.025424 9804 ts_manager.cc:194] Registered new tserver with Master: f052d5cbee65406593d8e44f74225bc2 (127.8.138.131:35445)
I20260926 15:02:40.035583 9814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "320cd09624624312bf1bcc0856c2874f" candidate_term: 5 candidate_status { last_received { term: 4 index: 16 } } ignore_live_leader: false dest_uuid: "6315fb5099e1420b95b8d83b3e19375c" is_pre_election: true
I20260926 15:02:40.036785 9142 leader_election.cc:302] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f; no voters: 6315fb5099e1420b95b8d83b3e19375c, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:02:40.037142 9876 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260926 15:02:40.049132 9391 heartbeater.cc:347] Connected to a master server at 127.8.138.190:44281
I20260926 15:02:40.049377 9391 heartbeater.cc:503] Master 127.8.138.190:44281 was elected leader, sending a full tablet report...
I20260926 15:02:40.066398 9533 heartbeater.cc:347] Connected to a master server at 127.8.138.190:44281
I20260926 15:02:40.095086 9880 raft_consensus.cc:482] T c5d07d3a7a164d098e61f8f7f2ac4a67 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:02:40.095382 9880 raft_consensus.cc:504] T c5d07d3a7a164d098e61f8f7f2ac4a67 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.097134 9880 leader_election.cc:288] T c5d07d3a7a164d098e61f8f7f2ac4a67 P 66ca148489d64deabd1dcbf7870c6d57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae330d635b1848aaaecee2c7ce96520a (127.8.138.130:44309), f052d5cbee65406593d8e44f74225bc2 (127.8.138.131:35445)
I20260926 15:02:40.097970 9487 raft_consensus.cc:2454] T 3fae8a76002849de9bb26610a0c33406 P ae330d635b1848aaaecee2c7ce96520a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f052d5cbee65406593d8e44f74225bc2 in term 1.
I20260926 15:02:40.098649 9567 leader_election.cc:302] T 3fae8a76002849de9bb26610a0c33406 P f052d5cbee65406593d8e44f74225bc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae330d635b1848aaaecee2c7ce96520a, f052d5cbee65406593d8e44f74225bc2; no voters:
I20260926 15:02:40.098958 9758 raft_consensus.cc:2781] T 3fae8a76002849de9bb26610a0c33406 P f052d5cbee65406593d8e44f74225bc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:02:40.099077 9758 raft_consensus.cc:686] T 3fae8a76002849de9bb26610a0c33406 P f052d5cbee65406593d8e44f74225bc2 [term 1 LEADER]: Becoming Leader. State: Replica: f052d5cbee65406593d8e44f74225bc2, State: Running, Role: LEADER
I20260926 15:02:40.099334 9758 consensus_queue.cc:237] T 3fae8a76002849de9bb26610a0c33406 P f052d5cbee65406593d8e44f74225bc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.104517 9345 raft_consensus.cc:2454] T 3fae8a76002849de9bb26610a0c33406 P 66ca148489d64deabd1dcbf7870c6d57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f052d5cbee65406593d8e44f74225bc2 in term 1.
I20260926 15:02:40.107143 9803 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" instance_seqno: 1790434956879292) as {username='slave'} at 127.8.138.130:35509; Asking this server to re-register.
I20260926 15:02:40.109938 9803 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" instance_seqno: 1790434956216737) as {username='slave'} at 127.8.138.129:39951; Asking this server to re-register.
I20260926 15:02:40.110836 9391 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:40.112488 9803 ts_manager.cc:194] Registered new tserver with Master: 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327)
I20260926 15:02:40.113365 9533 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:40.114888 9803 ts_manager.cc:194] Registered new tserver with Master: ae330d635b1848aaaecee2c7ce96520a (127.8.138.130:44309)
I20260926 15:02:40.117305 9632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5d07d3a7a164d098e61f8f7f2ac4a67" candidate_uuid: "66ca148489d64deabd1dcbf7870c6d57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f052d5cbee65406593d8e44f74225bc2" is_pre_election: true
I20260926 15:02:40.117764 9632 raft_consensus.cc:2454] T c5d07d3a7a164d098e61f8f7f2ac4a67 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66ca148489d64deabd1dcbf7870c6d57 in term 0.
I20260926 15:02:40.118571 9278 leader_election.cc:302] T c5d07d3a7a164d098e61f8f7f2ac4a67 P 66ca148489d64deabd1dcbf7870c6d57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ca148489d64deabd1dcbf7870c6d57, f052d5cbee65406593d8e44f74225bc2; no voters:
I20260926 15:02:40.119091 9880 raft_consensus.cc:2781] T c5d07d3a7a164d098e61f8f7f2ac4a67 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:02:40.119289 9880 raft_consensus.cc:482] T c5d07d3a7a164d098e61f8f7f2ac4a67 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:02:40.119412 9880 raft_consensus.cc:3037] T c5d07d3a7a164d098e61f8f7f2ac4a67 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:40.122092 9487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5d07d3a7a164d098e61f8f7f2ac4a67" candidate_uuid: "66ca148489d64deabd1dcbf7870c6d57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae330d635b1848aaaecee2c7ce96520a" is_pre_election: true
I20260926 15:02:40.122314 9487 raft_consensus.cc:2454] T c5d07d3a7a164d098e61f8f7f2ac4a67 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66ca148489d64deabd1dcbf7870c6d57 in term 0.
I20260926 15:02:40.163120 9729 sys_catalog.cc:455] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "be9958f2085242f7ab82ae933ceb1c42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:40.163436 9729 sys_catalog.cc:458] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [sys.catalog]: This master's current role is: LEADER
I20260926 15:02:40.231407 9880 raft_consensus.cc:504] T c5d07d3a7a164d098e61f8f7f2ac4a67 P 66ca148489d64deabd1dcbf7870c6d57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.232236 9880 leader_election.cc:288] T c5d07d3a7a164d098e61f8f7f2ac4a67 P 66ca148489d64deabd1dcbf7870c6d57 [CANDIDATE]: Term 1 election: Requested vote from peers ae330d635b1848aaaecee2c7ce96520a (127.8.138.130:44309), f052d5cbee65406593d8e44f74225bc2 (127.8.138.131:35445)
I20260926 15:02:40.234179 9487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5d07d3a7a164d098e61f8f7f2ac4a67" candidate_uuid: "66ca148489d64deabd1dcbf7870c6d57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae330d635b1848aaaecee2c7ce96520a"
I20260926 15:02:40.234401 9487 raft_consensus.cc:3037] T c5d07d3a7a164d098e61f8f7f2ac4a67 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:40.235208 9632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5d07d3a7a164d098e61f8f7f2ac4a67" candidate_uuid: "66ca148489d64deabd1dcbf7870c6d57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f052d5cbee65406593d8e44f74225bc2"
I20260926 15:02:40.235431 9632 raft_consensus.cc:3037] T c5d07d3a7a164d098e61f8f7f2ac4a67 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:40.237206 9487 raft_consensus.cc:2454] T c5d07d3a7a164d098e61f8f7f2ac4a67 P ae330d635b1848aaaecee2c7ce96520a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ca148489d64deabd1dcbf7870c6d57 in term 1.
I20260926 15:02:40.237875 9280 leader_election.cc:302] T c5d07d3a7a164d098e61f8f7f2ac4a67 P 66ca148489d64deabd1dcbf7870c6d57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ca148489d64deabd1dcbf7870c6d57, ae330d635b1848aaaecee2c7ce96520a; no voters:
I20260926 15:02:40.238281 9880 raft_consensus.cc:2781] T c5d07d3a7a164d098e61f8f7f2ac4a67 P 66ca148489d64deabd1dcbf7870c6d57 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:02:40.238409 9632 raft_consensus.cc:2454] T c5d07d3a7a164d098e61f8f7f2ac4a67 P f052d5cbee65406593d8e44f74225bc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ca148489d64deabd1dcbf7870c6d57 in term 1.
I20260926 15:02:40.244338 9880 raft_consensus.cc:686] T c5d07d3a7a164d098e61f8f7f2ac4a67 P 66ca148489d64deabd1dcbf7870c6d57 [term 1 LEADER]: Becoming Leader. State: Replica: 66ca148489d64deabd1dcbf7870c6d57, State: Running, Role: LEADER
I20260926 15:02:40.244939 9880 consensus_queue.cc:237] T c5d07d3a7a164d098e61f8f7f2ac4a67 P 66ca148489d64deabd1dcbf7870c6d57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.247181 9085 catalog_manager.cc:2148] created HMS catalog entry for table [id=e9e378126d2649dab722c82f99d654b7]
I20260926 15:02:40.261238 9887 raft_consensus.cc:482] T 2f0bbbe1ae394215b43195ad9662c9bb P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:02:40.261655 9887 raft_consensus.cc:504] T 2f0bbbe1ae394215b43195ad9662c9bb P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.275010 9632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f0bbbe1ae394215b43195ad9662c9bb" candidate_uuid: "ae330d635b1848aaaecee2c7ce96520a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f052d5cbee65406593d8e44f74225bc2" is_pre_election: true
I20260926 15:02:40.275303 9632 raft_consensus.cc:2454] T 2f0bbbe1ae394215b43195ad9662c9bb P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae330d635b1848aaaecee2c7ce96520a in term 0.
I20260926 15:02:40.275796 9420 leader_election.cc:302] T 2f0bbbe1ae394215b43195ad9662c9bb P ae330d635b1848aaaecee2c7ce96520a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae330d635b1848aaaecee2c7ce96520a, f052d5cbee65406593d8e44f74225bc2; no voters:
I20260926 15:02:40.282440 9887 leader_election.cc:288] T 2f0bbbe1ae394215b43195ad9662c9bb P ae330d635b1848aaaecee2c7ce96520a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327), f052d5cbee65406593d8e44f74225bc2 (127.8.138.131:35445)
I20260926 15:02:40.282706 9887 raft_consensus.cc:2781] T 2f0bbbe1ae394215b43195ad9662c9bb P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:02:40.282838 9887 raft_consensus.cc:482] T 2f0bbbe1ae394215b43195ad9662c9bb P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:02:40.282946 9887 raft_consensus.cc:3037] T 2f0bbbe1ae394215b43195ad9662c9bb P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:40.285754 9887 raft_consensus.cc:504] T 2f0bbbe1ae394215b43195ad9662c9bb P ae330d635b1848aaaecee2c7ce96520a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.286369 9887 leader_election.cc:288] T 2f0bbbe1ae394215b43195ad9662c9bb P ae330d635b1848aaaecee2c7ce96520a [CANDIDATE]: Term 1 election: Requested vote from peers 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327), f052d5cbee65406593d8e44f74225bc2 (127.8.138.131:35445)
I20260926 15:02:40.288760 9345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f0bbbe1ae394215b43195ad9662c9bb" candidate_uuid: "ae330d635b1848aaaecee2c7ce96520a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ca148489d64deabd1dcbf7870c6d57" is_pre_election: true
I20260926 15:02:40.289002 9345 raft_consensus.cc:2454] T 2f0bbbe1ae394215b43195ad9662c9bb P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae330d635b1848aaaecee2c7ce96520a in term 0.
I20260926 15:02:40.289526 9085 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:59430 (request call id 8) took 2037 ms. Trace:
I20260926 15:02:40.289665 9085 rpcz_store.cc:276] 0926 15:02:38.251602 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 15:02:38.253064 (+ 1462us) service_pool.cc:227] Handling call
0926 15:02:38.272857 (+ 19793us) catalog_manager.cc:2075] Acquired catalog manager lock
0926 15:02:38.273377 (+ 520us) catalog_manager.cc:2121] Created new table and tablet info
0926 15:02:40.247130 (+1973753us) catalog_manager.cc:2147] Created new table in HMS catalog
0926 15:02:40.289248 (+ 42118us) catalog_manager.cc:2181] Wrote table and tablets to system table
0926 15:02:40.289368 (+ 120us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0926 15:02:40.289479 (+ 111us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 15:02:40.257394 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 15:02:40.257511 (+ 117us) write_op.cc:432] Acquiring schema lock in shared mode
0926 15:02:40.257516 (+ 5us) write_op.cc:435] Acquired schema lock
0926 15:02:40.257518 (+ 2us) tablet.cc:660] Decoding operations
0926 15:02:40.257636 (+ 118us) write_op.cc:620] Acquiring the partition lock for write op
0926 15:02:40.257653 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0926 15:02:40.257660 (+ 7us) tablet.cc:683] Acquiring locks for 2 operations
0926 15:02:40.257804 (+ 144us) tablet.cc:699] Row locks acquired
0926 15:02:40.257807 (+ 3us) write_op.cc:260] PREPARE: finished
0926 15:02:40.257854 (+ 47us) write_op.cc:270] Start()
0926 15:02:40.257887 (+ 33us) write_op.cc:276] Timestamp: P: 1790434960257780 usec, L: 0
0926 15:02:40.257889 (+ 2us) op_driver.cc:331] REPLICATION: starting
0926 15:02:40.258169 (+ 280us) log.cc:842] Serialized 432 byte log entry
0926 15:02:40.261897 (+ 3728us) op_driver.cc:444] REPLICATION: finished
0926 15:02:40.287187 (+ 25290us) write_op.cc:301] APPLY: starting
0926 15:02:40.287216 (+ 29us) tablet.cc:1365] starting BulkCheckPresence
0926 15:02:40.287266 (+ 50us) tablet.cc:1368] finished BulkCheckPresence
0926 15:02:40.287293 (+ 27us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 15:02:40.287392 (+ 99us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 15:02:40.287422 (+ 30us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0926 15:02:40.287426 (+ 4us) write_op.cc:312] APPLY: finished
0926 15:02:40.287505 (+ 79us) log.cc:842] Serialized 28 byte log entry
0926 15:02:40.289104 (+ 1599us) write_op.cc:489] Releasing partition, row and schema locks
0926 15:02:40.289135 (+ 31us) write_op.cc:454] Released schema lock
0926 15:02:40.289144 (+ 9us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"Hive Metastore.queue_time_us":1194791,"Hive Metastore.run_cpu_time_us":324,"Hive Metastore.run_wall_time_us":778581,"spinlock_wait_cycles":7552,"child_traces":[["op",{"apply.queue_time_us":25214,"num_ops":2,"prepare.queue_time_us":9440,"prepare.run_cpu_time_us":866,"prepare.run_wall_time_us":1669,"raft.queue_time_us":69,"raft.run_cpu_time_us":580,"raft.run_wall_time_us":579,"replication_time_us":3986,"thread_start_us":35095,"threads_started":2}]]}
W20260926 15:02:40.291446 9085 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.8.138.130:33109 (request call id 10) took 789 ms (client timeout 1000 ms). Trace:
W20260926 15:02:40.291584 9085 rpcz_store.cc:269] 0926 15:02:39.502113 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 15:02:40.290396 (+788283us) service_pool.cc:227] Handling call
0926 15:02:40.291403 (+ 1007us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":1}
I20260926 15:02:40.293812 9345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f0bbbe1ae394215b43195ad9662c9bb" candidate_uuid: "ae330d635b1848aaaecee2c7ce96520a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ca148489d64deabd1dcbf7870c6d57"
I20260926 15:02:40.294019 9345 raft_consensus.cc:3037] T 2f0bbbe1ae394215b43195ad9662c9bb P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:40.296836 9345 raft_consensus.cc:2454] T 2f0bbbe1ae394215b43195ad9662c9bb P 66ca148489d64deabd1dcbf7870c6d57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae330d635b1848aaaecee2c7ce96520a in term 1.
I20260926 15:02:40.301980 9632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f0bbbe1ae394215b43195ad9662c9bb" candidate_uuid: "ae330d635b1848aaaecee2c7ce96520a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f052d5cbee65406593d8e44f74225bc2"
I20260926 15:02:40.302263 9632 raft_consensus.cc:3037] T 2f0bbbe1ae394215b43195ad9662c9bb P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:40.305063 9632 raft_consensus.cc:2454] T 2f0bbbe1ae394215b43195ad9662c9bb P f052d5cbee65406593d8e44f74225bc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae330d635b1848aaaecee2c7ce96520a in term 1.
I20260926 15:02:40.306102 9420 leader_election.cc:302] T 2f0bbbe1ae394215b43195ad9662c9bb P ae330d635b1848aaaecee2c7ce96520a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae330d635b1848aaaecee2c7ce96520a, f052d5cbee65406593d8e44f74225bc2; no voters:
I20260926 15:02:40.306644 9895 raft_consensus.cc:2781] T 2f0bbbe1ae394215b43195ad9662c9bb P ae330d635b1848aaaecee2c7ce96520a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:02:40.306933 9895 raft_consensus.cc:686] T 2f0bbbe1ae394215b43195ad9662c9bb P ae330d635b1848aaaecee2c7ce96520a [term 1 LEADER]: Becoming Leader. State: Replica: ae330d635b1848aaaecee2c7ce96520a, State: Running, Role: LEADER
W20260926 15:02:40.308560 9085 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.8.138.129:35895 (request call id 10) took 762 ms (client timeout 1000 ms). Trace:
I20260926 15:02:40.308569 9611 tablet_service.cc:1467] Processing CreateTablet for tablet 9134d14fae4d46b6bd8ef1c05dc5b1c2 (DEFAULT_TABLE table=default.table_e3b713aac3c94e449bb5882310d55a40 [id=e9e378126d2649dab722c82f99d654b7]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 15:02:40.308704 9085 rpcz_store.cc:269] 0926 15:02:39.545910 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 15:02:40.298075 (+752165us) service_pool.cc:227] Handling call
0926 15:02:40.308513 (+ 10438us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":1}
I20260926 15:02:40.309008 9611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9134d14fae4d46b6bd8ef1c05dc5b1c2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 15:02:40.309794 9085 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.8.138.131:39841 (request call id 7) took 754 ms (client timeout 1000 ms). Trace:
W20260926 15:02:40.309913 9085 rpcz_store.cc:269] 0926 15:02:39.554937 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 15:02:40.308911 (+753974us) service_pool.cc:227] Handling call
0926 15:02:40.309758 (+ 847us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":1}
I20260926 15:02:40.311748 9085 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59430:
name: "default.Table_985d0971fb8a4d8eae0b2cb73ca99c6f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:40.312739 9895 consensus_queue.cc:237] T 2f0bbbe1ae394215b43195ad9662c9bb P ae330d635b1848aaaecee2c7ce96520a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.312906 9324 tablet_service.cc:1467] Processing CreateTablet for tablet 9134d14fae4d46b6bd8ef1c05dc5b1c2 (DEFAULT_TABLE table=default.table_e3b713aac3c94e449bb5882310d55a40 [id=e9e378126d2649dab722c82f99d654b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:40.313354 9324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9134d14fae4d46b6bd8ef1c05dc5b1c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:40.315907 9466 tablet_service.cc:1467] Processing CreateTablet for tablet 9134d14fae4d46b6bd8ef1c05dc5b1c2 (DEFAULT_TABLE table=default.table_e3b713aac3c94e449bb5882310d55a40 [id=e9e378126d2649dab722c82f99d654b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:40.316366 9466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9134d14fae4d46b6bd8ef1c05dc5b1c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:40.324206 9082 catalog_manager.cc:2148] created HMS catalog entry for table [id=b84ce6d15e4246dbbb299468138add55]
I20260926 15:02:40.341843 9082 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:59430 (request call id 11) took 2089 ms. Trace:
I20260926 15:02:40.342051 9082 rpcz_store.cc:276] 0926 15:02:38.251980 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 15:02:38.254865 (+ 2885us) service_pool.cc:227] Handling call
0926 15:02:38.273100 (+ 18235us) catalog_manager.cc:2075] Acquired catalog manager lock
0926 15:02:38.273609 (+ 509us) catalog_manager.cc:2121] Created new table and tablet info
0926 15:02:40.324115 (+2050506us) catalog_manager.cc:2147] Created new table in HMS catalog
0926 15:02:40.341454 (+ 17339us) catalog_manager.cc:2181] Wrote table and tablets to system table
0926 15:02:40.341650 (+ 196us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0926 15:02:40.341789 (+ 139us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 15:02:40.325997 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 15:02:40.326136 (+ 139us) write_op.cc:432] Acquiring schema lock in shared mode
0926 15:02:40.326142 (+ 6us) write_op.cc:435] Acquired schema lock
0926 15:02:40.326144 (+ 2us) tablet.cc:660] Decoding operations
0926 15:02:40.326243 (+ 99us) write_op.cc:620] Acquiring the partition lock for write op
0926 15:02:40.326258 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0926 15:02:40.326262 (+ 4us) tablet.cc:683] Acquiring locks for 2 operations
0926 15:02:40.326377 (+ 115us) tablet.cc:699] Row locks acquired
0926 15:02:40.326382 (+ 5us) write_op.cc:260] PREPARE: finished
0926 15:02:40.326436 (+ 54us) write_op.cc:270] Start()
0926 15:02:40.326487 (+ 51us) write_op.cc:276] Timestamp: P: 1790434960326394 usec, L: 0
0926 15:02:40.326495 (+ 8us) op_driver.cc:331] REPLICATION: starting
0926 15:02:40.326778 (+ 283us) log.cc:842] Serialized 432 byte log entry
0926 15:02:40.340085 (+ 13307us) op_driver.cc:444] REPLICATION: finished
0926 15:02:40.340223 (+ 138us) write_op.cc:301] APPLY: starting
0926 15:02:40.340244 (+ 21us) tablet.cc:1365] starting BulkCheckPresence
0926 15:02:40.340311 (+ 67us) tablet.cc:1368] finished BulkCheckPresence
0926 15:02:40.340351 (+ 40us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 15:02:40.340658 (+ 307us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 15:02:40.340715 (+ 57us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0926 15:02:40.340726 (+ 11us) write_op.cc:312] APPLY: finished
0926 15:02:40.340855 (+ 129us) log.cc:842] Serialized 28 byte log entry
0926 15:02:40.340945 (+ 90us) write_op.cc:489] Releasing partition, row and schema locks
0926 15:02:40.340976 (+ 31us) write_op.cc:454] Released schema lock
0926 15:02:40.340986 (+ 10us) write_op.cc:341] FINISH: Updating metrics
0926 15:02:40.341059 (+ 73us) write_op.cc:489] Releasing partition, row and schema locks
0926 15:02:40.341065 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":1973175,"Hive Metastore.run_cpu_time_us":348,"Hive Metastore.run_wall_time_us":76877,"child_traces":[["op",{"apply.queue_time_us":47,"apply.run_cpu_time_us":917,"apply.run_wall_time_us":916,"num_ops":2,"prepare.queue_time_us":889,"prepare.run_cpu_time_us":931,"prepare.run_wall_time_us":931,"raft.queue_time_us":6558,"raft.run_cpu_time_us":679,"raft.run_wall_time_us":679,"replication_time_us":13553}]]}
I20260926 15:02:40.345953 9897 tablet_bootstrap.cc:492] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P f052d5cbee65406593d8e44f74225bc2: Bootstrap starting.
I20260926 15:02:40.345387 9082 catalog_manager.cc:5495] T c5d07d3a7a164d098e61f8f7f2ac4a67 P 66ca148489d64deabd1dcbf7870c6d57 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129). New cstate: current_term: 1 leader_uuid: "66ca148489d64deabd1dcbf7870c6d57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } health_report { overall_health: UNKNOWN } } }
I20260926 15:02:40.367769 9082 catalog_manager.cc:5495] T 3fae8a76002849de9bb26610a0c33406 P f052d5cbee65406593d8e44f74225bc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to f052d5cbee65406593d8e44f74225bc2 (127.8.138.131). New cstate: current_term: 1 leader_uuid: "f052d5cbee65406593d8e44f74225bc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } health_report { overall_health: HEALTHY } } }
I20260926 15:02:40.368462 9082 catalog_manager.cc:5495] T 570182804db146ccacf0150199c5d2be P f052d5cbee65406593d8e44f74225bc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to f052d5cbee65406593d8e44f74225bc2 (127.8.138.131). New cstate: current_term: 1 leader_uuid: "f052d5cbee65406593d8e44f74225bc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } health_report { overall_health: UNKNOWN } } }
I20260926 15:02:40.370126 9465 tablet_service.cc:1467] Processing CreateTablet for tablet 317305a7fdb54f0bb2bb569272550b7b (DEFAULT_TABLE table=default.table_7b56039466d14dcab4dc63192de3b08e [id=b84ce6d15e4246dbbb299468138add55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:40.370584 9465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 317305a7fdb54f0bb2bb569272550b7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:40.373937 9611 tablet_service.cc:1467] Processing CreateTablet for tablet 317305a7fdb54f0bb2bb569272550b7b (DEFAULT_TABLE table=default.table_7b56039466d14dcab4dc63192de3b08e [id=b84ce6d15e4246dbbb299468138add55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:40.374477 9611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 317305a7fdb54f0bb2bb569272550b7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:40.377522 9325 tablet_service.cc:1467] Processing CreateTablet for tablet 317305a7fdb54f0bb2bb569272550b7b (DEFAULT_TABLE table=default.table_7b56039466d14dcab4dc63192de3b08e [id=b84ce6d15e4246dbbb299468138add55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:40.377981 9325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 317305a7fdb54f0bb2bb569272550b7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:40.388947 9082 catalog_manager.cc:5495] T 2f0bbbe1ae394215b43195ad9662c9bb P ae330d635b1848aaaecee2c7ce96520a reported cstate change: term changed from 0 to 1, leader changed from <none> to ae330d635b1848aaaecee2c7ce96520a (127.8.138.130). New cstate: current_term: 1 leader_uuid: "ae330d635b1848aaaecee2c7ce96520a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } health_report { overall_health: UNKNOWN } } }
I20260926 15:02:40.400398 9082 master_service.cc:946] ReplaceTablet: received request to replace tablet 570182804db146ccacf0150199c5d2be from {username='slave'} at 127.0.0.1:59430
I20260926 15:02:40.408629 9082 catalog_manager.cc:5782] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42: Sending DeleteTablet for 3 replicas of tablet 570182804db146ccacf0150199c5d2be
I20260926 15:02:40.409422 9325 tablet_service.cc:1514] Processing DeleteTablet for tablet 570182804db146ccacf0150199c5d2be with delete_type TABLET_DATA_DELETED (replaced by tablet 41e641e28d1f48b2a7e0c8fb8964f678) from {username='slave'} at 127.0.0.1:48112
I20260926 15:02:40.415118 9610 tablet_service.cc:1514] Processing DeleteTablet for tablet 570182804db146ccacf0150199c5d2be with delete_type TABLET_DATA_DELETED (replaced by tablet 41e641e28d1f48b2a7e0c8fb8964f678) from {username='slave'} at 127.0.0.1:55968
I20260926 15:02:40.424677 9465 tablet_service.cc:1514] Processing DeleteTablet for tablet 570182804db146ccacf0150199c5d2be with delete_type TABLET_DATA_DELETED (replaced by tablet 41e641e28d1f48b2a7e0c8fb8964f678) from {username='slave'} at 127.0.0.1:58558
I20260926 15:02:40.438581 9323 tablet_service.cc:1467] Processing CreateTablet for tablet 41e641e28d1f48b2a7e0c8fb8964f678 (DEFAULT_TABLE table=default.table_d2243274908043e58210a197141af813 [id=5357ddcf21634f99bb4f9abc5b6c4d80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:40.438997 9467 tablet_service.cc:1467] Processing CreateTablet for tablet 41e641e28d1f48b2a7e0c8fb8964f678 (DEFAULT_TABLE table=default.table_d2243274908043e58210a197141af813 [id=5357ddcf21634f99bb4f9abc5b6c4d80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:40.438951 9612 tablet_service.cc:1467] Processing CreateTablet for tablet 41e641e28d1f48b2a7e0c8fb8964f678 (DEFAULT_TABLE table=default.table_d2243274908043e58210a197141af813 [id=5357ddcf21634f99bb4f9abc5b6c4d80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:40.439203 9323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41e641e28d1f48b2a7e0c8fb8964f678. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:40.439462 9467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41e641e28d1f48b2a7e0c8fb8964f678. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:40.439532 9612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41e641e28d1f48b2a7e0c8fb8964f678. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:40.440475 9082 catalog_manager.cc:6565] ReplaceTablet: tablet 570182804db146ccacf0150199c5d2be deleted and replaced by tablet 41e641e28d1f48b2a7e0c8fb8964f678
I20260926 15:02:40.440652 9082 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:59430 (request call id 25) took 1026 ms. Trace:
I20260926 15:02:40.440752 9082 rpcz_store.cc:276] 0926 15:02:39.413903 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 15:02:40.400302 (+986399us) service_pool.cc:227] Handling call
0926 15:02:40.408304 (+ 8002us) write_op.cc:489] Releasing partition, row and schema locks
0926 15:02:40.408310 (+ 6us) write_op.cc:454] Released schema lock
0926 15:02:40.440599 (+ 32289us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 15:02:40.401837 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 15:02:40.401965 (+ 128us) write_op.cc:432] Acquiring schema lock in shared mode
0926 15:02:40.401973 (+ 8us) write_op.cc:435] Acquired schema lock
0926 15:02:40.401977 (+ 4us) tablet.cc:660] Decoding operations
0926 15:02:40.402115 (+ 138us) write_op.cc:620] Acquiring the partition lock for write op
0926 15:02:40.402132 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0926 15:02:40.402140 (+ 8us) tablet.cc:683] Acquiring locks for 2 operations
0926 15:02:40.402248 (+ 108us) tablet.cc:699] Row locks acquired
0926 15:02:40.402252 (+ 4us) write_op.cc:260] PREPARE: finished
0926 15:02:40.402291 (+ 39us) write_op.cc:270] Start()
0926 15:02:40.402329 (+ 38us) write_op.cc:276] Timestamp: P: 1790434960402246 usec, L: 0
0926 15:02:40.402335 (+ 6us) op_driver.cc:331] REPLICATION: starting
0926 15:02:40.402603 (+ 268us) log.cc:842] Serialized 661 byte log entry
0926 15:02:40.406898 (+ 4295us) op_driver.cc:444] REPLICATION: finished
0926 15:02:40.407015 (+ 117us) write_op.cc:301] APPLY: starting
0926 15:02:40.407037 (+ 22us) tablet.cc:1365] starting BulkCheckPresence
0926 15:02:40.407088 (+ 51us) tablet.cc:1368] finished BulkCheckPresence
0926 15:02:40.407131 (+ 43us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 15:02:40.407267 (+ 136us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 15:02:40.407338 (+ 71us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 15:02:40.407348 (+ 10us) write_op.cc:312] APPLY: finished
0926 15:02:40.407440 (+ 92us) log.cc:842] Serialized 28 byte log entry
0926 15:02:40.407507 (+ 67us) write_op.cc:489] Releasing partition, row and schema locks
0926 15:02:40.407532 (+ 25us) write_op.cc:454] Released schema lock
0926 15:02:40.407541 (+ 9us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":62,"apply.run_cpu_time_us":605,"apply.run_wall_time_us":604,"num_ops":2,"prepare.queue_time_us":44,"prepare.run_cpu_time_us":900,"prepare.run_wall_time_us":900,"raft.queue_time_us":484,"raft.run_cpu_time_us":481,"raft.run_wall_time_us":480,"replication_time_us":4533}]]}
I20260926 15:02:40.444308 9897 tablet_bootstrap.cc:654] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P f052d5cbee65406593d8e44f74225bc2: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:40.445946 9082 master_service.cc:946] ReplaceTablet: received request to replace tablet 570182804db146ccacf0150199c5d2be from {username='slave'} at 127.0.0.1:59430
I20260926 15:02:40.446353 9082 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:59430 (request call id 26) took 1031 ms. Trace:
I20260926 15:02:40.446465 9082 rpcz_store.cc:276] 0926 15:02:39.414973 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 15:02:40.445860 (+1030887us) service_pool.cc:227] Handling call
0926 15:02:40.446323 (+ 463us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260926 15:02:40.447965 9899 tablet_bootstrap.cc:492] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P ae330d635b1848aaaecee2c7ce96520a: Bootstrap starting.
I20260926 15:02:40.448439 9082 master_service.cc:946] ReplaceTablet: received request to replace tablet 2f0bbbe1ae394215b43195ad9662c9bb from {username='slave'} at 127.0.0.1:59430
I20260926 15:02:40.450140 9898 tablet_bootstrap.cc:492] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P 66ca148489d64deabd1dcbf7870c6d57: Bootstrap starting.
I20260926 15:02:40.450341 9899 tablet_bootstrap.cc:654] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P ae330d635b1848aaaecee2c7ce96520a: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:40.452059 9898 tablet_bootstrap.cc:654] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P 66ca148489d64deabd1dcbf7870c6d57: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:40.452513 9902 tablet_replica.cc:330] T 570182804db146ccacf0150199c5d2be P ae330d635b1848aaaecee2c7ce96520a: stopping tablet replica
I20260926 15:02:40.452883 9902 raft_consensus.cc:2229] T 570182804db146ccacf0150199c5d2be P ae330d635b1848aaaecee2c7ce96520a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:02:40.453657 9902 raft_consensus.cc:2258] T 570182804db146ccacf0150199c5d2be P ae330d635b1848aaaecee2c7ce96520a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:40.454954 9899 tablet_bootstrap.cc:492] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P ae330d635b1848aaaecee2c7ce96520a: No bootstrap required, opened a new log
I20260926 15:02:40.455648 9899 ts_tablet_manager.cc:1397] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P ae330d635b1848aaaecee2c7ce96520a: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.004s
I20260926 15:02:40.455799 9900 tablet_replica.cc:330] T 570182804db146ccacf0150199c5d2be P 66ca148489d64deabd1dcbf7870c6d57: stopping tablet replica
I20260926 15:02:40.455442 9902 ts_tablet_manager.cc:1905] T 570182804db146ccacf0150199c5d2be P ae330d635b1848aaaecee2c7ce96520a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:40.456270 9901 tablet_replica.cc:330] T 570182804db146ccacf0150199c5d2be P f052d5cbee65406593d8e44f74225bc2: stopping tablet replica
I20260926 15:02:40.457430 9901 raft_consensus.cc:2229] T 570182804db146ccacf0150199c5d2be P f052d5cbee65406593d8e44f74225bc2 [term 1 LEADER]: Raft consensus shutting down.
I20260926 15:02:40.457742 9900 raft_consensus.cc:2229] T 570182804db146ccacf0150199c5d2be P 66ca148489d64deabd1dcbf7870c6d57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:02:40.458328 9901 pending_rounds.cc:62] T 570182804db146ccacf0150199c5d2be P f052d5cbee65406593d8e44f74225bc2: Trying to abort 1 pending ops.
I20260926 15:02:40.458592 9900 raft_consensus.cc:2258] T 570182804db146ccacf0150199c5d2be P 66ca148489d64deabd1dcbf7870c6d57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:40.458616 9899 raft_consensus.cc:348] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.458962 9899 raft_consensus.cc:374] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:40.459082 9899 raft_consensus.cc:729] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae330d635b1848aaaecee2c7ce96520a, State: Initialized, Role: FOLLOWER
I20260926 15:02:40.457223 9897 tablet_bootstrap.cc:492] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P f052d5cbee65406593d8e44f74225bc2: No bootstrap required, opened a new log
I20260926 15:02:40.459352 9897 ts_tablet_manager.cc:1397] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P f052d5cbee65406593d8e44f74225bc2: Time spent bootstrapping tablet: real 0.114s user 0.005s sys 0.000s
I20260926 15:02:40.459376 9899 consensus_queue.cc:260] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P ae330d635b1848aaaecee2c7ce96520a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.460024 9900 ts_tablet_manager.cc:1905] T 570182804db146ccacf0150199c5d2be P 66ca148489d64deabd1dcbf7870c6d57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:40.460232 9901 pending_rounds.cc:68] T 570182804db146ccacf0150199c5d2be P f052d5cbee65406593d8e44f74225bc2: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333621596187783168 op_type: NO_OP noop_request { }
I20260926 15:02:40.461313 9901 raft_consensus.cc:2866] T 570182804db146ccacf0150199c5d2be P f052d5cbee65406593d8e44f74225bc2 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 15:02:40.461880 9082 catalog_manager.cc:5782] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42: Sending DeleteTablet for 3 replicas of tablet 2f0bbbe1ae394215b43195ad9662c9bb
I20260926 15:02:40.462095 9901 raft_consensus.cc:2258] T 570182804db146ccacf0150199c5d2be P f052d5cbee65406593d8e44f74225bc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:40.460986 9897 raft_consensus.cc:348] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.462679 9897 raft_consensus.cc:374] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:40.462777 9465 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f0bbbe1ae394215b43195ad9662c9bb with delete_type TABLET_DATA_DELETED (replaced by tablet 4bec0ca69b0e477281a232c1bf1121fb) from {username='slave'} at 127.0.0.1:58558
I20260926 15:02:40.463169 9324 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f0bbbe1ae394215b43195ad9662c9bb with delete_type TABLET_DATA_DELETED (replaced by tablet 4bec0ca69b0e477281a232c1bf1121fb) from {username='slave'} at 127.0.0.1:48112
I20260926 15:02:40.464022 9899 ts_tablet_manager.cc:1428] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P ae330d635b1848aaaecee2c7ce96520a: Time spent starting tablet: real 0.008s user 0.000s sys 0.002s
I20260926 15:02:40.464690 9899 tablet_bootstrap.cc:492] T 317305a7fdb54f0bb2bb569272550b7b P ae330d635b1848aaaecee2c7ce96520a: Bootstrap starting.
I20260926 15:02:40.469120 9900 ts_tablet_manager.cc:1918] T 570182804db146ccacf0150199c5d2be P 66ca148489d64deabd1dcbf7870c6d57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 15:02:40.469369 9897 raft_consensus.cc:729] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f052d5cbee65406593d8e44f74225bc2, State: Initialized, Role: FOLLOWER
I20260926 15:02:40.469444 9899 tablet_bootstrap.cc:654] T 317305a7fdb54f0bb2bb569272550b7b P ae330d635b1848aaaecee2c7ce96520a: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:40.469828 9897 consensus_queue.cc:260] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P f052d5cbee65406593d8e44f74225bc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.463660 9901 ts_tablet_manager.cc:1905] T 570182804db146ccacf0150199c5d2be P f052d5cbee65406593d8e44f74225bc2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:40.471544 9902 ts_tablet_manager.cc:1918] T 570182804db146ccacf0150199c5d2be P ae330d635b1848aaaecee2c7ce96520a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 15:02:40.463020 9611 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f0bbbe1ae394215b43195ad9662c9bb with delete_type TABLET_DATA_DELETED (replaced by tablet 4bec0ca69b0e477281a232c1bf1121fb) from {username='slave'} at 127.0.0.1:55968
I20260926 15:02:40.471701 9902 log.cc:1196] T 570182804db146ccacf0150199c5d2be P ae330d635b1848aaaecee2c7ce96520a: Deleting WAL directory at /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-1/wal/wals/570182804db146ccacf0150199c5d2be
I20260926 15:02:40.472555 9902 ts_tablet_manager.cc:1939] T 570182804db146ccacf0150199c5d2be P ae330d635b1848aaaecee2c7ce96520a: Deleting consensus metadata
I20260926 15:02:40.473606 9074 catalog_manager.cc:4841] TS ae330d635b1848aaaecee2c7ce96520a (127.8.138.130:44309): tablet 570182804db146ccacf0150199c5d2be (table default.table_d2243274908043e58210a197141af813 [id=5357ddcf21634f99bb4f9abc5b6c4d80]) successfully deleted
I20260926 15:02:40.473677 9902 tablet_replica.cc:330] T 2f0bbbe1ae394215b43195ad9662c9bb P ae330d635b1848aaaecee2c7ce96520a: stopping tablet replica
I20260926 15:02:40.473886 9902 raft_consensus.cc:2229] T 2f0bbbe1ae394215b43195ad9662c9bb P ae330d635b1848aaaecee2c7ce96520a [term 1 LEADER]: Raft consensus shutting down.
I20260926 15:02:40.473955 9897 ts_tablet_manager.cc:1428] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P f052d5cbee65406593d8e44f74225bc2: Time spent starting tablet: real 0.014s user 0.002s sys 0.000s
I20260926 15:02:40.474339 9902 pending_rounds.cc:62] T 2f0bbbe1ae394215b43195ad9662c9bb P ae330d635b1848aaaecee2c7ce96520a: Trying to abort 1 pending ops.
I20260926 15:02:40.474520 9902 pending_rounds.cc:68] T 2f0bbbe1ae394215b43195ad9662c9bb P ae330d635b1848aaaecee2c7ce96520a: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333621597446496256 op_type: NO_OP noop_request { }
I20260926 15:02:40.474753 9902 raft_consensus.cc:2866] T 2f0bbbe1ae394215b43195ad9662c9bb P ae330d635b1848aaaecee2c7ce96520a [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 15:02:40.474932 9902 raft_consensus.cc:2258] T 2f0bbbe1ae394215b43195ad9662c9bb P ae330d635b1848aaaecee2c7ce96520a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:40.476810 9902 ts_tablet_manager.cc:1905] T 2f0bbbe1ae394215b43195ad9662c9bb P ae330d635b1848aaaecee2c7ce96520a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:40.477197 9897 tablet_bootstrap.cc:492] T 41e641e28d1f48b2a7e0c8fb8964f678 P f052d5cbee65406593d8e44f74225bc2: Bootstrap starting.
I20260926 15:02:40.479298 9897 tablet_bootstrap.cc:654] T 41e641e28d1f48b2a7e0c8fb8964f678 P f052d5cbee65406593d8e44f74225bc2: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:40.481264 9898 tablet_bootstrap.cc:492] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P 66ca148489d64deabd1dcbf7870c6d57: No bootstrap required, opened a new log
I20260926 15:02:40.481472 9898 ts_tablet_manager.cc:1397] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P 66ca148489d64deabd1dcbf7870c6d57: Time spent bootstrapping tablet: real 0.031s user 0.000s sys 0.004s
I20260926 15:02:40.482364 9898 raft_consensus.cc:348] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.482705 9898 raft_consensus.cc:374] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:40.482856 9898 raft_consensus.cc:729] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ca148489d64deabd1dcbf7870c6d57, State: Initialized, Role: FOLLOWER
I20260926 15:02:40.483071 9899 tablet_bootstrap.cc:492] T 317305a7fdb54f0bb2bb569272550b7b P ae330d635b1848aaaecee2c7ce96520a: No bootstrap required, opened a new log
I20260926 15:02:40.483256 9899 ts_tablet_manager.cc:1397] T 317305a7fdb54f0bb2bb569272550b7b P ae330d635b1848aaaecee2c7ce96520a: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.002s
I20260926 15:02:40.483225 9898 consensus_queue.cc:260] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P 66ca148489d64deabd1dcbf7870c6d57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.485482 9899 raft_consensus.cc:348] T 317305a7fdb54f0bb2bb569272550b7b P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.485733 9899 raft_consensus.cc:374] T 317305a7fdb54f0bb2bb569272550b7b P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:40.485802 9899 raft_consensus.cc:729] T 317305a7fdb54f0bb2bb569272550b7b P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae330d635b1848aaaecee2c7ce96520a, State: Initialized, Role: FOLLOWER
I20260926 15:02:40.486052 9899 consensus_queue.cc:260] T 317305a7fdb54f0bb2bb569272550b7b P ae330d635b1848aaaecee2c7ce96520a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.486765 9324 tablet_service.cc:1467] Processing CreateTablet for tablet 4bec0ca69b0e477281a232c1bf1121fb (DEFAULT_TABLE table=default.table_41a3ce3c89684cbb9054c4251a0cc51c [id=7cc0c844d49c4ff08cef9e2c410991f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:40.487203 9324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bec0ca69b0e477281a232c1bf1121fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:40.488519 9898 ts_tablet_manager.cc:1428] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P 66ca148489d64deabd1dcbf7870c6d57: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20260926 15:02:40.489421 9082 catalog_manager.cc:6565] ReplaceTablet: tablet 2f0bbbe1ae394215b43195ad9662c9bb deleted and replaced by tablet 4bec0ca69b0e477281a232c1bf1121fb
I20260926 15:02:40.489573 9082 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:59430 (request call id 27) took 1069 ms. Trace:
I20260926 15:02:40.489671 9082 rpcz_store.cc:276] 0926 15:02:39.419737 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 15:02:40.448373 (+1028636us) service_pool.cc:227] Handling call
0926 15:02:40.489522 (+ 41149us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 15:02:40.449743 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 15:02:40.449896 (+ 153us) write_op.cc:432] Acquiring schema lock in shared mode
0926 15:02:40.449902 (+ 6us) write_op.cc:435] Acquired schema lock
0926 15:02:40.449904 (+ 2us) tablet.cc:660] Decoding operations
0926 15:02:40.450040 (+ 136us) write_op.cc:620] Acquiring the partition lock for write op
0926 15:02:40.450053 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0926 15:02:40.450058 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
0926 15:02:40.450150 (+ 92us) tablet.cc:699] Row locks acquired
0926 15:02:40.450154 (+ 4us) write_op.cc:260] PREPARE: finished
0926 15:02:40.450196 (+ 42us) write_op.cc:270] Start()
0926 15:02:40.450232 (+ 36us) write_op.cc:276] Timestamp: P: 1790434960450147 usec, L: 0
0926 15:02:40.450236 (+ 4us) op_driver.cc:331] REPLICATION: starting
0926 15:02:40.450438 (+ 202us) log.cc:842] Serialized 661 byte log entry
0926 15:02:40.456995 (+ 6557us) op_driver.cc:444] REPLICATION: finished
0926 15:02:40.458890 (+ 1895us) write_op.cc:301] APPLY: starting
0926 15:02:40.458908 (+ 18us) tablet.cc:1365] starting BulkCheckPresence
0926 15:02:40.458956 (+ 48us) tablet.cc:1368] finished BulkCheckPresence
0926 15:02:40.458982 (+ 26us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 15:02:40.459077 (+ 95us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 15:02:40.459143 (+ 66us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 15:02:40.459148 (+ 5us) write_op.cc:312] APPLY: finished
0926 15:02:40.459224 (+ 76us) log.cc:842] Serialized 28 byte log entry
0926 15:02:40.459277 (+ 53us) write_op.cc:489] Releasing partition, row and schema locks
0926 15:02:40.459296 (+ 19us) write_op.cc:454] Released schema lock
0926 15:02:40.459302 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0926 15:02:40.459358 (+ 56us) write_op.cc:489] Releasing partition, row and schema locks
0926 15:02:40.459362 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1819,"apply.run_cpu_time_us":546,"apply.run_wall_time_us":545,"num_ops":2,"prepare.queue_time_us":41,"prepare.run_cpu_time_us":812,"prepare.run_wall_time_us":813,"raft.queue_time_us":654,"raft.run_cpu_time_us":507,"raft.run_wall_time_us":507,"replication_time_us":6733}]]}
I20260926 15:02:40.491147 9465 tablet_service.cc:1467] Processing CreateTablet for tablet 4bec0ca69b0e477281a232c1bf1121fb (DEFAULT_TABLE table=default.table_41a3ce3c89684cbb9054c4251a0cc51c [id=7cc0c844d49c4ff08cef9e2c410991f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:40.492527 9898 tablet_bootstrap.cc:492] T 317305a7fdb54f0bb2bb569272550b7b P 66ca148489d64deabd1dcbf7870c6d57: Bootstrap starting.
I20260926 15:02:40.494374 9898 tablet_bootstrap.cc:654] T 317305a7fdb54f0bb2bb569272550b7b P 66ca148489d64deabd1dcbf7870c6d57: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:40.480603 9900 log.cc:1196] T 570182804db146ccacf0150199c5d2be P 66ca148489d64deabd1dcbf7870c6d57: Deleting WAL directory at /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-0/wal/wals/570182804db146ccacf0150199c5d2be
I20260926 15:02:40.496285 9611 tablet_service.cc:1467] Processing CreateTablet for tablet 4bec0ca69b0e477281a232c1bf1121fb (DEFAULT_TABLE table=default.table_41a3ce3c89684cbb9054c4251a0cc51c [id=7cc0c844d49c4ff08cef9e2c410991f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:40.496714 9611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bec0ca69b0e477281a232c1bf1121fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:40.497002 9900 ts_tablet_manager.cc:1939] T 570182804db146ccacf0150199c5d2be P 66ca148489d64deabd1dcbf7870c6d57: Deleting consensus metadata
I20260926 15:02:40.497238 9899 ts_tablet_manager.cc:1428] T 317305a7fdb54f0bb2bb569272550b7b P ae330d635b1848aaaecee2c7ce96520a: Time spent starting tablet: real 0.014s user 0.001s sys 0.001s
I20260926 15:02:40.498292 9073 catalog_manager.cc:4841] TS 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327): tablet 570182804db146ccacf0150199c5d2be (table default.table_d2243274908043e58210a197141af813 [id=5357ddcf21634f99bb4f9abc5b6c4d80]) successfully deleted
I20260926 15:02:40.499673 9897 tablet_bootstrap.cc:492] T 41e641e28d1f48b2a7e0c8fb8964f678 P f052d5cbee65406593d8e44f74225bc2: No bootstrap required, opened a new log
I20260926 15:02:40.499945 9897 ts_tablet_manager.cc:1397] T 41e641e28d1f48b2a7e0c8fb8964f678 P f052d5cbee65406593d8e44f74225bc2: Time spent bootstrapping tablet: real 0.023s user 0.004s sys 0.000s
I20260926 15:02:40.500056 9899 tablet_bootstrap.cc:492] T 41e641e28d1f48b2a7e0c8fb8964f678 P ae330d635b1848aaaecee2c7ce96520a: Bootstrap starting.
I20260926 15:02:40.500883 9897 raft_consensus.cc:348] T 41e641e28d1f48b2a7e0c8fb8964f678 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.501210 9897 raft_consensus.cc:374] T 41e641e28d1f48b2a7e0c8fb8964f678 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:40.501358 9897 raft_consensus.cc:729] T 41e641e28d1f48b2a7e0c8fb8964f678 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f052d5cbee65406593d8e44f74225bc2, State: Initialized, Role: FOLLOWER
I20260926 15:02:40.501648 9899 tablet_bootstrap.cc:654] T 41e641e28d1f48b2a7e0c8fb8964f678 P ae330d635b1848aaaecee2c7ce96520a: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:40.501793 9897 consensus_queue.cc:260] T 41e641e28d1f48b2a7e0c8fb8964f678 P f052d5cbee65406593d8e44f74225bc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.502796 9465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bec0ca69b0e477281a232c1bf1121fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:40.504523 9901 ts_tablet_manager.cc:1918] T 570182804db146ccacf0150199c5d2be P f052d5cbee65406593d8e44f74225bc2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 15:02:40.504662 9901 log.cc:1196] T 570182804db146ccacf0150199c5d2be P f052d5cbee65406593d8e44f74225bc2: Deleting WAL directory at /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-2/wal/wals/570182804db146ccacf0150199c5d2be
I20260926 15:02:40.505293 9897 ts_tablet_manager.cc:1428] T 41e641e28d1f48b2a7e0c8fb8964f678 P f052d5cbee65406593d8e44f74225bc2: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260926 15:02:40.507391 9901 ts_tablet_manager.cc:1939] T 570182804db146ccacf0150199c5d2be P f052d5cbee65406593d8e44f74225bc2: Deleting consensus metadata
I20260926 15:02:40.507974 9900 tablet_replica.cc:330] T 2f0bbbe1ae394215b43195ad9662c9bb P 66ca148489d64deabd1dcbf7870c6d57: stopping tablet replica
I20260926 15:02:40.508208 9900 raft_consensus.cc:2229] T 2f0bbbe1ae394215b43195ad9662c9bb P 66ca148489d64deabd1dcbf7870c6d57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:02:40.508405 9900 raft_consensus.cc:2258] T 2f0bbbe1ae394215b43195ad9662c9bb P 66ca148489d64deabd1dcbf7870c6d57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:40.508903 9897 tablet_bootstrap.cc:492] T 317305a7fdb54f0bb2bb569272550b7b P f052d5cbee65406593d8e44f74225bc2: Bootstrap starting.
I20260926 15:02:40.509210 9072 catalog_manager.cc:4841] TS f052d5cbee65406593d8e44f74225bc2 (127.8.138.131:35445): tablet 570182804db146ccacf0150199c5d2be (table default.table_d2243274908043e58210a197141af813 [id=5357ddcf21634f99bb4f9abc5b6c4d80]) successfully deleted
I20260926 15:02:40.509539 9901 tablet_replica.cc:330] T 2f0bbbe1ae394215b43195ad9662c9bb P f052d5cbee65406593d8e44f74225bc2: stopping tablet replica
I20260926 15:02:40.509755 9901 raft_consensus.cc:2229] T 2f0bbbe1ae394215b43195ad9662c9bb P f052d5cbee65406593d8e44f74225bc2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:02:40.509951 9901 raft_consensus.cc:2258] T 2f0bbbe1ae394215b43195ad9662c9bb P f052d5cbee65406593d8e44f74225bc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:40.510494 9901 ts_tablet_manager.cc:1905] T 2f0bbbe1ae394215b43195ad9662c9bb P f052d5cbee65406593d8e44f74225bc2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:40.510732 9897 tablet_bootstrap.cc:654] T 317305a7fdb54f0bb2bb569272550b7b P f052d5cbee65406593d8e44f74225bc2: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:40.512382 9900 ts_tablet_manager.cc:1905] T 2f0bbbe1ae394215b43195ad9662c9bb P 66ca148489d64deabd1dcbf7870c6d57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:40.513753 9082 master_service.cc:946] ReplaceTablet: received request to replace tablet 2f0bbbe1ae394215b43195ad9662c9bb from {username='slave'} at 127.0.0.1:59430
I20260926 15:02:40.514040 9902 ts_tablet_manager.cc:1918] T 2f0bbbe1ae394215b43195ad9662c9bb P ae330d635b1848aaaecee2c7ce96520a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 15:02:40.514533 9902 log.cc:1196] T 2f0bbbe1ae394215b43195ad9662c9bb P ae330d635b1848aaaecee2c7ce96520a: Deleting WAL directory at /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-1/wal/wals/2f0bbbe1ae394215b43195ad9662c9bb
I20260926 15:02:40.515080 9902 ts_tablet_manager.cc:1939] T 2f0bbbe1ae394215b43195ad9662c9bb P ae330d635b1848aaaecee2c7ce96520a: Deleting consensus metadata
I20260926 15:02:40.516126 9899 tablet_bootstrap.cc:492] T 41e641e28d1f48b2a7e0c8fb8964f678 P ae330d635b1848aaaecee2c7ce96520a: No bootstrap required, opened a new log
I20260926 15:02:40.516494 9899 ts_tablet_manager.cc:1397] T 41e641e28d1f48b2a7e0c8fb8964f678 P ae330d635b1848aaaecee2c7ce96520a: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.001s
I20260926 15:02:40.517506 9898 tablet_bootstrap.cc:492] T 317305a7fdb54f0bb2bb569272550b7b P 66ca148489d64deabd1dcbf7870c6d57: No bootstrap required, opened a new log
I20260926 15:02:40.517692 9898 ts_tablet_manager.cc:1397] T 317305a7fdb54f0bb2bb569272550b7b P 66ca148489d64deabd1dcbf7870c6d57: Time spent bootstrapping tablet: real 0.025s user 0.000s sys 0.004s
I20260926 15:02:40.517491 9899 raft_consensus.cc:348] T 41e641e28d1f48b2a7e0c8fb8964f678 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.517930 9899 raft_consensus.cc:374] T 41e641e28d1f48b2a7e0c8fb8964f678 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:40.518078 9899 raft_consensus.cc:729] T 41e641e28d1f48b2a7e0c8fb8964f678 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae330d635b1848aaaecee2c7ce96520a, State: Initialized, Role: FOLLOWER
I20260926 15:02:40.518419 9899 consensus_queue.cc:260] T 41e641e28d1f48b2a7e0c8fb8964f678 P ae330d635b1848aaaecee2c7ce96520a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.518466 9898 raft_consensus.cc:348] T 317305a7fdb54f0bb2bb569272550b7b P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.518734 9898 raft_consensus.cc:374] T 317305a7fdb54f0bb2bb569272550b7b P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:40.518811 9898 raft_consensus.cc:729] T 317305a7fdb54f0bb2bb569272550b7b P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ca148489d64deabd1dcbf7870c6d57, State: Initialized, Role: FOLLOWER
I20260926 15:02:40.519093 9898 consensus_queue.cc:260] T 317305a7fdb54f0bb2bb569272550b7b P 66ca148489d64deabd1dcbf7870c6d57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.519232 9082 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:59430 (request call id 28) took 1096 ms. Trace:
I20260926 15:02:40.522262 9899 ts_tablet_manager.cc:1428] T 41e641e28d1f48b2a7e0c8fb8964f678 P ae330d635b1848aaaecee2c7ce96520a: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260926 15:02:40.523343 9899 tablet_bootstrap.cc:492] T 4bec0ca69b0e477281a232c1bf1121fb P ae330d635b1848aaaecee2c7ce96520a: Bootstrap starting.
I20260926 15:02:40.523415 9082 rpcz_store.cc:276] 0926 15:02:39.422544 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 15:02:40.513628 (+1091084us) service_pool.cc:227] Handling call
0926 15:02:40.519182 (+ 5554us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260926 15:02:40.525290 9898 ts_tablet_manager.cc:1428] T 317305a7fdb54f0bb2bb569272550b7b P 66ca148489d64deabd1dcbf7870c6d57: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260926 15:02:40.526297 9074 catalog_manager.cc:4841] TS ae330d635b1848aaaecee2c7ce96520a (127.8.138.130:44309): tablet 2f0bbbe1ae394215b43195ad9662c9bb (table default.table_41a3ce3c89684cbb9054c4251a0cc51c [id=7cc0c844d49c4ff08cef9e2c410991f8]) successfully deleted
I20260926 15:02:40.526626 9899 tablet_bootstrap.cc:654] T 4bec0ca69b0e477281a232c1bf1121fb P ae330d635b1848aaaecee2c7ce96520a: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:40.528963 9898 tablet_bootstrap.cc:492] T 41e641e28d1f48b2a7e0c8fb8964f678 P 66ca148489d64deabd1dcbf7870c6d57: Bootstrap starting.
I20260926 15:02:40.531148 9082 master_service.cc:946] ReplaceTablet: received request to replace tablet 2f0bbbe1ae394215b43195ad9662c9bb from {username='slave'} at 127.0.0.1:59430
I20260926 15:02:40.531594 9082 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:59430 (request call id 29) took 1108 ms. Trace:
I20260926 15:02:40.531704 9082 rpcz_store.cc:276] 0926 15:02:39.422793 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 15:02:40.531079 (+1108286us) service_pool.cc:227] Handling call
0926 15:02:40.531538 (+ 459us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260926 15:02:40.532603 9897 tablet_bootstrap.cc:492] T 317305a7fdb54f0bb2bb569272550b7b P f052d5cbee65406593d8e44f74225bc2: No bootstrap required, opened a new log
I20260926 15:02:40.532526 9082 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:59430:
table { table_name: "default.Table_95f48725b6ce47fab5f96e18ccb69dbc" } new_table_name: "default.Table_85e820ec1f9f4f17be9385e54532213a" modify_external_catalogs: true
I20260926 15:02:40.532763 9897 ts_tablet_manager.cc:1397] T 317305a7fdb54f0bb2bb569272550b7b P f052d5cbee65406593d8e44f74225bc2: Time spent bootstrapping tablet: real 0.025s user 0.003s sys 0.001s
I20260926 15:02:40.533504 9897 raft_consensus.cc:348] T 317305a7fdb54f0bb2bb569272550b7b P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.533720 9897 raft_consensus.cc:374] T 317305a7fdb54f0bb2bb569272550b7b P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:40.533806 9897 raft_consensus.cc:729] T 317305a7fdb54f0bb2bb569272550b7b P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f052d5cbee65406593d8e44f74225bc2, State: Initialized, Role: FOLLOWER
I20260926 15:02:40.534077 9897 consensus_queue.cc:260] T 317305a7fdb54f0bb2bb569272550b7b P f052d5cbee65406593d8e44f74225bc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.535327 9898 tablet_bootstrap.cc:654] T 41e641e28d1f48b2a7e0c8fb8964f678 P 66ca148489d64deabd1dcbf7870c6d57: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:40.538229 9900 ts_tablet_manager.cc:1918] T 2f0bbbe1ae394215b43195ad9662c9bb P 66ca148489d64deabd1dcbf7870c6d57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 15:02:40.538415 9900 log.cc:1196] T 2f0bbbe1ae394215b43195ad9662c9bb P 66ca148489d64deabd1dcbf7870c6d57: Deleting WAL directory at /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-0/wal/wals/2f0bbbe1ae394215b43195ad9662c9bb
I20260926 15:02:40.538746 9900 ts_tablet_manager.cc:1939] T 2f0bbbe1ae394215b43195ad9662c9bb P 66ca148489d64deabd1dcbf7870c6d57: Deleting consensus metadata
I20260926 15:02:40.539722 9073 catalog_manager.cc:4841] TS 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327): tablet 2f0bbbe1ae394215b43195ad9662c9bb (table default.table_41a3ce3c89684cbb9054c4251a0cc51c [id=7cc0c844d49c4ff08cef9e2c410991f8]) successfully deleted
I20260926 15:02:40.542740 9897 ts_tablet_manager.cc:1428] T 317305a7fdb54f0bb2bb569272550b7b P f052d5cbee65406593d8e44f74225bc2: Time spent starting tablet: real 0.010s user 0.000s sys 0.002s
I20260926 15:02:40.546387 9901 ts_tablet_manager.cc:1918] T 2f0bbbe1ae394215b43195ad9662c9bb P f052d5cbee65406593d8e44f74225bc2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 15:02:40.546487 9897 tablet_bootstrap.cc:492] T 4bec0ca69b0e477281a232c1bf1121fb P f052d5cbee65406593d8e44f74225bc2: Bootstrap starting.
I20260926 15:02:40.546540 9901 log.cc:1196] T 2f0bbbe1ae394215b43195ad9662c9bb P f052d5cbee65406593d8e44f74225bc2: Deleting WAL directory at /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-2/wal/wals/2f0bbbe1ae394215b43195ad9662c9bb
I20260926 15:02:40.547021 9901 ts_tablet_manager.cc:1939] T 2f0bbbe1ae394215b43195ad9662c9bb P f052d5cbee65406593d8e44f74225bc2: Deleting consensus metadata
I20260926 15:02:40.548353 9897 tablet_bootstrap.cc:654] T 4bec0ca69b0e477281a232c1bf1121fb P f052d5cbee65406593d8e44f74225bc2: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:40.549817 9072 catalog_manager.cc:4841] TS f052d5cbee65406593d8e44f74225bc2 (127.8.138.131:35445): tablet 2f0bbbe1ae394215b43195ad9662c9bb (table default.table_41a3ce3c89684cbb9054c4251a0cc51c [id=7cc0c844d49c4ff08cef9e2c410991f8]) successfully deleted
I20260926 15:02:40.550202 9898 tablet_bootstrap.cc:492] T 41e641e28d1f48b2a7e0c8fb8964f678 P 66ca148489d64deabd1dcbf7870c6d57: No bootstrap required, opened a new log
I20260926 15:02:40.550221 9899 tablet_bootstrap.cc:492] T 4bec0ca69b0e477281a232c1bf1121fb P ae330d635b1848aaaecee2c7ce96520a: No bootstrap required, opened a new log
I20260926 15:02:40.550379 9899 ts_tablet_manager.cc:1397] T 4bec0ca69b0e477281a232c1bf1121fb P ae330d635b1848aaaecee2c7ce96520a: Time spent bootstrapping tablet: real 0.027s user 0.011s sys 0.012s
I20260926 15:02:40.550550 9898 ts_tablet_manager.cc:1397] T 41e641e28d1f48b2a7e0c8fb8964f678 P 66ca148489d64deabd1dcbf7870c6d57: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.001s
I20260926 15:02:40.551304 9898 raft_consensus.cc:348] T 41e641e28d1f48b2a7e0c8fb8964f678 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.551627 9898 raft_consensus.cc:374] T 41e641e28d1f48b2a7e0c8fb8964f678 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:40.551599 9899 raft_consensus.cc:348] T 4bec0ca69b0e477281a232c1bf1121fb P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.551760 9898 raft_consensus.cc:729] T 41e641e28d1f48b2a7e0c8fb8964f678 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ca148489d64deabd1dcbf7870c6d57, State: Initialized, Role: FOLLOWER
I20260926 15:02:40.551918 9899 raft_consensus.cc:374] T 4bec0ca69b0e477281a232c1bf1121fb P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:40.552002 9899 raft_consensus.cc:729] T 4bec0ca69b0e477281a232c1bf1121fb P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae330d635b1848aaaecee2c7ce96520a, State: Initialized, Role: FOLLOWER
I20260926 15:02:40.552237 9899 consensus_queue.cc:260] T 4bec0ca69b0e477281a232c1bf1121fb P ae330d635b1848aaaecee2c7ce96520a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.552269 9898 consensus_queue.cc:260] T 41e641e28d1f48b2a7e0c8fb8964f678 P 66ca148489d64deabd1dcbf7870c6d57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.553905 9899 ts_tablet_manager.cc:1428] T 4bec0ca69b0e477281a232c1bf1121fb P ae330d635b1848aaaecee2c7ce96520a: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260926 15:02:40.568169 9898 ts_tablet_manager.cc:1428] T 41e641e28d1f48b2a7e0c8fb8964f678 P 66ca148489d64deabd1dcbf7870c6d57: Time spent starting tablet: real 0.017s user 0.002s sys 0.000s
I20260926 15:02:40.574031 9898 tablet_bootstrap.cc:492] T 4bec0ca69b0e477281a232c1bf1121fb P 66ca148489d64deabd1dcbf7870c6d57: Bootstrap starting.
I20260926 15:02:40.576547 9898 tablet_bootstrap.cc:654] T 4bec0ca69b0e477281a232c1bf1121fb P 66ca148489d64deabd1dcbf7870c6d57: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:40.578073 9897 tablet_bootstrap.cc:492] T 4bec0ca69b0e477281a232c1bf1121fb P f052d5cbee65406593d8e44f74225bc2: No bootstrap required, opened a new log
I20260926 15:02:40.578465 9897 ts_tablet_manager.cc:1397] T 4bec0ca69b0e477281a232c1bf1121fb P f052d5cbee65406593d8e44f74225bc2: Time spent bootstrapping tablet: real 0.032s user 0.010s sys 0.014s
I20260926 15:02:40.580489 9897 raft_consensus.cc:348] T 4bec0ca69b0e477281a232c1bf1121fb P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.580865 9897 raft_consensus.cc:374] T 4bec0ca69b0e477281a232c1bf1121fb P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:40.580986 9897 raft_consensus.cc:729] T 4bec0ca69b0e477281a232c1bf1121fb P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f052d5cbee65406593d8e44f74225bc2, State: Initialized, Role: FOLLOWER
I20260926 15:02:40.581284 9897 consensus_queue.cc:260] T 4bec0ca69b0e477281a232c1bf1121fb P f052d5cbee65406593d8e44f74225bc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.581881 9897 ts_tablet_manager.cc:1428] T 4bec0ca69b0e477281a232c1bf1121fb P f052d5cbee65406593d8e44f74225bc2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 15:02:40.588774 9898 tablet_bootstrap.cc:492] T 4bec0ca69b0e477281a232c1bf1121fb P 66ca148489d64deabd1dcbf7870c6d57: No bootstrap required, opened a new log
I20260926 15:02:40.588985 9898 ts_tablet_manager.cc:1397] T 4bec0ca69b0e477281a232c1bf1121fb P 66ca148489d64deabd1dcbf7870c6d57: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.000s
I20260926 15:02:40.589887 9898 raft_consensus.cc:348] T 4bec0ca69b0e477281a232c1bf1121fb P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.590145 9898 raft_consensus.cc:374] T 4bec0ca69b0e477281a232c1bf1121fb P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:40.590229 9898 raft_consensus.cc:729] T 4bec0ca69b0e477281a232c1bf1121fb P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ca148489d64deabd1dcbf7870c6d57, State: Initialized, Role: FOLLOWER
I20260926 15:02:40.590406 9895 raft_consensus.cc:482] T 4bec0ca69b0e477281a232c1bf1121fb P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:02:40.590478 9898 consensus_queue.cc:260] T 4bec0ca69b0e477281a232c1bf1121fb P 66ca148489d64deabd1dcbf7870c6d57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.590583 9895 raft_consensus.cc:504] T 4bec0ca69b0e477281a232c1bf1121fb P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.591073 9898 ts_tablet_manager.cc:1428] T 4bec0ca69b0e477281a232c1bf1121fb P 66ca148489d64deabd1dcbf7870c6d57: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260926 15:02:40.591337 9895 leader_election.cc:288] T 4bec0ca69b0e477281a232c1bf1121fb P ae330d635b1848aaaecee2c7ce96520a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327), f052d5cbee65406593d8e44f74225bc2 (127.8.138.131:35445)
I20260926 15:02:40.592115 9632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bec0ca69b0e477281a232c1bf1121fb" candidate_uuid: "ae330d635b1848aaaecee2c7ce96520a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f052d5cbee65406593d8e44f74225bc2" is_pre_election: true
I20260926 15:02:40.592365 9632 raft_consensus.cc:2454] T 4bec0ca69b0e477281a232c1bf1121fb P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae330d635b1848aaaecee2c7ce96520a in term 0.
I20260926 15:02:40.593351 9345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bec0ca69b0e477281a232c1bf1121fb" candidate_uuid: "ae330d635b1848aaaecee2c7ce96520a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ca148489d64deabd1dcbf7870c6d57" is_pre_election: true
I20260926 15:02:40.593596 9345 raft_consensus.cc:2454] T 4bec0ca69b0e477281a232c1bf1121fb P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae330d635b1848aaaecee2c7ce96520a in term 0.
I20260926 15:02:40.594630 9421 leader_election.cc:302] T 4bec0ca69b0e477281a232c1bf1121fb P ae330d635b1848aaaecee2c7ce96520a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ca148489d64deabd1dcbf7870c6d57, ae330d635b1848aaaecee2c7ce96520a; no voters:
I20260926 15:02:40.595145 9895 raft_consensus.cc:2781] T 4bec0ca69b0e477281a232c1bf1121fb P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:02:40.595266 9895 raft_consensus.cc:482] T 4bec0ca69b0e477281a232c1bf1121fb P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:02:40.595376 9895 raft_consensus.cc:3037] T 4bec0ca69b0e477281a232c1bf1121fb P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:40.600222 9895 raft_consensus.cc:504] T 4bec0ca69b0e477281a232c1bf1121fb P ae330d635b1848aaaecee2c7ce96520a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.601572 9345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bec0ca69b0e477281a232c1bf1121fb" candidate_uuid: "ae330d635b1848aaaecee2c7ce96520a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ca148489d64deabd1dcbf7870c6d57"
I20260926 15:02:40.601574 9632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bec0ca69b0e477281a232c1bf1121fb" candidate_uuid: "ae330d635b1848aaaecee2c7ce96520a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f052d5cbee65406593d8e44f74225bc2"
I20260926 15:02:40.601843 9632 raft_consensus.cc:3037] T 4bec0ca69b0e477281a232c1bf1121fb P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:40.601876 9345 raft_consensus.cc:3037] T 4bec0ca69b0e477281a232c1bf1121fb P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:40.604940 9345 raft_consensus.cc:2454] T 4bec0ca69b0e477281a232c1bf1121fb P 66ca148489d64deabd1dcbf7870c6d57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae330d635b1848aaaecee2c7ce96520a in term 1.
I20260926 15:02:40.604955 9632 raft_consensus.cc:2454] T 4bec0ca69b0e477281a232c1bf1121fb P f052d5cbee65406593d8e44f74225bc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae330d635b1848aaaecee2c7ce96520a in term 1.
I20260926 15:02:40.605479 9421 leader_election.cc:302] T 4bec0ca69b0e477281a232c1bf1121fb P ae330d635b1848aaaecee2c7ce96520a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ca148489d64deabd1dcbf7870c6d57, ae330d635b1848aaaecee2c7ce96520a; no voters:
I20260926 15:02:40.605881 9887 raft_consensus.cc:2781] T 4bec0ca69b0e477281a232c1bf1121fb P ae330d635b1848aaaecee2c7ce96520a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:02:40.606081 9895 leader_election.cc:288] T 4bec0ca69b0e477281a232c1bf1121fb P ae330d635b1848aaaecee2c7ce96520a [CANDIDATE]: Term 1 election: Requested vote from peers 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327), f052d5cbee65406593d8e44f74225bc2 (127.8.138.131:35445)
I20260926 15:02:40.607331 9887 raft_consensus.cc:686] T 4bec0ca69b0e477281a232c1bf1121fb P ae330d635b1848aaaecee2c7ce96520a [term 1 LEADER]: Becoming Leader. State: Replica: ae330d635b1848aaaecee2c7ce96520a, State: Running, Role: LEADER
I20260926 15:02:40.607676 9887 consensus_queue.cc:237] T 4bec0ca69b0e477281a232c1bf1121fb P ae330d635b1848aaaecee2c7ce96520a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
15:02:40.670 [HMS - ERROR - pool-6-thread-3] (RetryingHMSHandler.java:201) AlreadyExistsException(message:Table hive.default.table_f356bc0dfc6a47f98430002dccbb873f already exists)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1839)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1786)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:2035)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
at com.sun.proxy.$Proxy25.create_table_with_environment_context(Unknown Source)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14800)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14784)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:104)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
W20260926 15:02:40.672767 9083 catalog_manager.cc:2144] Already present: failed to create HMS catalog entry for table [id=3032fe9e2e5348658ad2b5c4ae8a7c59]: failed to create Hive MetaStore table: TException - service has thrown: AlreadyExistsException(message=Table hive.default.table_f356bc0dfc6a47f98430002dccbb873f already exists)
I20260926 15:02:40.673327 9083 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:59430 (request call id 10) took 2421 ms. Trace:
I20260926 15:02:40.673466 9083 rpcz_store.cc:276] 0926 15:02:38.251892 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 15:02:38.255238 (+ 3346us) service_pool.cc:227] Handling call
0926 15:02:38.273737 (+ 18499us) catalog_manager.cc:2075] Acquired catalog manager lock
0926 15:02:38.274120 (+ 383us) catalog_manager.cc:2121] Created new table and tablet info
0926 15:02:40.673281 (+2399161us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":2049627,"Hive Metastore.run_cpu_time_us":1053,"Hive Metastore.run_wall_time_us":348662}
I20260926 15:02:40.679423 9083 catalog_manager.cc:5495] T 4bec0ca69b0e477281a232c1bf1121fb P ae330d635b1848aaaecee2c7ce96520a reported cstate change: term changed from 0 to 1, leader changed from <none> to ae330d635b1848aaaecee2c7ce96520a (127.8.138.130). New cstate: current_term: 1 leader_uuid: "ae330d635b1848aaaecee2c7ce96520a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } health_report { overall_health: UNKNOWN } } }
I20260926 15:02:40.689867 9083 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59430:
name: "default.Table_e26e1e9fab934eab9d1b22663b28c712"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:40.703472 9080 catalog_manager.cc:2148] created HMS catalog entry for table [id=51dea274f4bf4903baaddbbd8c257b27]
I20260926 15:02:40.709955 9080 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:59430 (request call id 13) took 2457 ms. Trace:
I20260926 15:02:40.710127 9080 rpcz_store.cc:276] 0926 15:02:38.252183 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 15:02:38.254024 (+ 1841us) service_pool.cc:227] Handling call
0926 15:02:38.274091 (+ 20067us) catalog_manager.cc:2075] Acquired catalog manager lock
0926 15:02:38.274590 (+ 499us) catalog_manager.cc:2121] Created new table and tablet info
0926 15:02:40.703402 (+2428812us) catalog_manager.cc:2147] Created new table in HMS catalog
0926 15:02:40.709644 (+ 6242us) catalog_manager.cc:2181] Wrote table and tablets to system table
0926 15:02:40.709794 (+ 150us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0926 15:02:40.709899 (+ 105us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 15:02:40.704512 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 15:02:40.704641 (+ 129us) write_op.cc:432] Acquiring schema lock in shared mode
0926 15:02:40.704649 (+ 8us) write_op.cc:435] Acquired schema lock
0926 15:02:40.704652 (+ 3us) tablet.cc:660] Decoding operations
0926 15:02:40.704754 (+ 102us) write_op.cc:620] Acquiring the partition lock for write op
0926 15:02:40.704770 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0926 15:02:40.704775 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
0926 15:02:40.704873 (+ 98us) tablet.cc:699] Row locks acquired
0926 15:02:40.704875 (+ 2us) write_op.cc:260] PREPARE: finished
0926 15:02:40.704916 (+ 41us) write_op.cc:270] Start()
0926 15:02:40.704951 (+ 35us) write_op.cc:276] Timestamp: P: 1790434960704842 usec, L: 0
0926 15:02:40.704955 (+ 4us) op_driver.cc:331] REPLICATION: starting
0926 15:02:40.705169 (+ 214us) log.cc:842] Serialized 432 byte log entry
0926 15:02:40.709003 (+ 3834us) op_driver.cc:444] REPLICATION: finished
0926 15:02:40.709130 (+ 127us) write_op.cc:301] APPLY: starting
0926 15:02:40.709146 (+ 16us) tablet.cc:1365] starting BulkCheckPresence
0926 15:02:40.709193 (+ 47us) tablet.cc:1368] finished BulkCheckPresence
0926 15:02:40.709228 (+ 35us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 15:02:40.709335 (+ 107us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 15:02:40.709374 (+ 39us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0926 15:02:40.709381 (+ 7us) write_op.cc:312] APPLY: finished
0926 15:02:40.709462 (+ 81us) log.cc:842] Serialized 28 byte log entry
0926 15:02:40.709527 (+ 65us) write_op.cc:489] Releasing partition, row and schema locks
0926 15:02:40.709550 (+ 23us) write_op.cc:454] Released schema lock
0926 15:02:40.709557 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0926 15:02:40.709623 (+ 66us) write_op.cc:489] Releasing partition, row and schema locks
0926 15:02:40.709627 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":2397859,"Hive Metastore.run_cpu_time_us":343,"Hive Metastore.run_wall_time_us":30756,"child_traces":[["op",{"apply.queue_time_us":76,"apply.run_cpu_time_us":555,"apply.run_wall_time_us":554,"num_ops":2,"prepare.queue_time_us":67,"prepare.run_cpu_time_us":798,"prepare.run_wall_time_us":798,"raft.queue_time_us":1339,"raft.run_cpu_time_us":549,"raft.run_wall_time_us":550,"replication_time_us":4023}]]}
I20260926 15:02:40.711051 9080 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59430:
name: "default.Table_a20572133e32441ab0bfe9e3fceed04a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:40.721148 9324 tablet_service.cc:1467] Processing CreateTablet for tablet 0d837276d648440cbebe1f88c281cb11 (DEFAULT_TABLE table=default.table_96be20374f7e43e7a865549e39bd64f6 [id=51dea274f4bf4903baaddbbd8c257b27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:40.721457 9611 tablet_service.cc:1467] Processing CreateTablet for tablet 0d837276d648440cbebe1f88c281cb11 (DEFAULT_TABLE table=default.table_96be20374f7e43e7a865549e39bd64f6 [id=51dea274f4bf4903baaddbbd8c257b27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:40.721684 9324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d837276d648440cbebe1f88c281cb11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:40.721897 9611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d837276d648440cbebe1f88c281cb11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:40.724545 9465 tablet_service.cc:1467] Processing CreateTablet for tablet 0d837276d648440cbebe1f88c281cb11 (DEFAULT_TABLE table=default.table_96be20374f7e43e7a865549e39bd64f6 [id=51dea274f4bf4903baaddbbd8c257b27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:40.725015 9465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d837276d648440cbebe1f88c281cb11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:40.728427 9898 tablet_bootstrap.cc:492] T 0d837276d648440cbebe1f88c281cb11 P 66ca148489d64deabd1dcbf7870c6d57: Bootstrap starting.
I20260926 15:02:40.730113 9898 tablet_bootstrap.cc:654] T 0d837276d648440cbebe1f88c281cb11 P 66ca148489d64deabd1dcbf7870c6d57: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:40.733369 9897 tablet_bootstrap.cc:492] T 0d837276d648440cbebe1f88c281cb11 P f052d5cbee65406593d8e44f74225bc2: Bootstrap starting.
I20260926 15:02:40.735524 9897 tablet_bootstrap.cc:654] T 0d837276d648440cbebe1f88c281cb11 P f052d5cbee65406593d8e44f74225bc2: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:40.736848 9899 tablet_bootstrap.cc:492] T 0d837276d648440cbebe1f88c281cb11 P ae330d635b1848aaaecee2c7ce96520a: Bootstrap starting.
I20260926 15:02:40.738490 9899 tablet_bootstrap.cc:654] T 0d837276d648440cbebe1f88c281cb11 P ae330d635b1848aaaecee2c7ce96520a: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:40.741899 9897 tablet_bootstrap.cc:492] T 0d837276d648440cbebe1f88c281cb11 P f052d5cbee65406593d8e44f74225bc2: No bootstrap required, opened a new log
I20260926 15:02:40.742064 9897 ts_tablet_manager.cc:1397] T 0d837276d648440cbebe1f88c281cb11 P f052d5cbee65406593d8e44f74225bc2: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.002s
I20260926 15:02:40.744235 9897 raft_consensus.cc:348] T 0d837276d648440cbebe1f88c281cb11 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.744603 9897 raft_consensus.cc:374] T 0d837276d648440cbebe1f88c281cb11 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:40.744809 9897 raft_consensus.cc:729] T 0d837276d648440cbebe1f88c281cb11 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f052d5cbee65406593d8e44f74225bc2, State: Initialized, Role: FOLLOWER
I20260926 15:02:40.745096 9897 consensus_queue.cc:260] T 0d837276d648440cbebe1f88c281cb11 P f052d5cbee65406593d8e44f74225bc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.745659 9898 tablet_bootstrap.cc:492] T 0d837276d648440cbebe1f88c281cb11 P 66ca148489d64deabd1dcbf7870c6d57: No bootstrap required, opened a new log
I20260926 15:02:40.745863 9898 ts_tablet_manager.cc:1397] T 0d837276d648440cbebe1f88c281cb11 P 66ca148489d64deabd1dcbf7870c6d57: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.000s
I20260926 15:02:40.746995 9898 raft_consensus.cc:348] T 0d837276d648440cbebe1f88c281cb11 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.747236 9898 raft_consensus.cc:374] T 0d837276d648440cbebe1f88c281cb11 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:40.747327 9898 raft_consensus.cc:729] T 0d837276d648440cbebe1f88c281cb11 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ca148489d64deabd1dcbf7870c6d57, State: Initialized, Role: FOLLOWER
I20260926 15:02:40.751436 9897 ts_tablet_manager.cc:1428] T 0d837276d648440cbebe1f88c281cb11 P f052d5cbee65406593d8e44f74225bc2: Time spent starting tablet: real 0.009s user 0.002s sys 0.001s
I20260926 15:02:40.751935 9898 consensus_queue.cc:260] T 0d837276d648440cbebe1f88c281cb11 P 66ca148489d64deabd1dcbf7870c6d57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.761490 9898 ts_tablet_manager.cc:1428] T 0d837276d648440cbebe1f88c281cb11 P 66ca148489d64deabd1dcbf7870c6d57: Time spent starting tablet: real 0.015s user 0.003s sys 0.000s
I20260926 15:02:40.766438 9079 catalog_manager.cc:2148] created HMS catalog entry for table [id=7ba7b39b84394421b88a6ce7616c3c91]
I20260926 15:02:40.766795 9899 tablet_bootstrap.cc:492] T 0d837276d648440cbebe1f88c281cb11 P ae330d635b1848aaaecee2c7ce96520a: No bootstrap required, opened a new log
I20260926 15:02:40.766958 9899 ts_tablet_manager.cc:1397] T 0d837276d648440cbebe1f88c281cb11 P ae330d635b1848aaaecee2c7ce96520a: Time spent bootstrapping tablet: real 0.030s user 0.001s sys 0.003s
I20260926 15:02:40.767753 9899 raft_consensus.cc:348] T 0d837276d648440cbebe1f88c281cb11 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.768014 9899 raft_consensus.cc:374] T 0d837276d648440cbebe1f88c281cb11 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:40.769170 9899 raft_consensus.cc:729] T 0d837276d648440cbebe1f88c281cb11 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae330d635b1848aaaecee2c7ce96520a, State: Initialized, Role: FOLLOWER
I20260926 15:02:40.769464 9899 consensus_queue.cc:260] T 0d837276d648440cbebe1f88c281cb11 P ae330d635b1848aaaecee2c7ce96520a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.771929 9899 ts_tablet_manager.cc:1428] T 0d837276d648440cbebe1f88c281cb11 P ae330d635b1848aaaecee2c7ce96520a: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260926 15:02:40.783273 9079 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:59430 (request call id 14) took 2530 ms. Trace:
I20260926 15:02:40.783442 9079 rpcz_store.cc:276] 0926 15:02:38.252309 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 15:02:38.253557 (+ 1248us) service_pool.cc:227] Handling call
0926 15:02:38.274280 (+ 20723us) catalog_manager.cc:2075] Acquired catalog manager lock
0926 15:02:38.274632 (+ 352us) catalog_manager.cc:2121] Created new table and tablet info
0926 15:02:40.766357 (+2491725us) catalog_manager.cc:2147] Created new table in HMS catalog
0926 15:02:40.782981 (+ 16624us) catalog_manager.cc:2181] Wrote table and tablets to system table
0926 15:02:40.783102 (+ 121us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0926 15:02:40.783214 (+ 112us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 15:02:40.767262 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 15:02:40.767431 (+ 169us) write_op.cc:432] Acquiring schema lock in shared mode
0926 15:02:40.767438 (+ 7us) write_op.cc:435] Acquired schema lock
0926 15:02:40.767440 (+ 2us) tablet.cc:660] Decoding operations
0926 15:02:40.767548 (+ 108us) write_op.cc:620] Acquiring the partition lock for write op
0926 15:02:40.767582 (+ 34us) write_op.cc:641] Partition lock acquired for write op
0926 15:02:40.767595 (+ 13us) tablet.cc:683] Acquiring locks for 2 operations
0926 15:02:40.767687 (+ 92us) tablet.cc:699] Row locks acquired
0926 15:02:40.767691 (+ 4us) write_op.cc:260] PREPARE: finished
0926 15:02:40.767759 (+ 68us) write_op.cc:270] Start()
0926 15:02:40.767795 (+ 36us) write_op.cc:276] Timestamp: P: 1790434960767679 usec, L: 0
0926 15:02:40.767801 (+ 6us) op_driver.cc:331] REPLICATION: starting
0926 15:02:40.768048 (+ 247us) log.cc:842] Serialized 432 byte log entry
0926 15:02:40.781974 (+ 13926us) op_driver.cc:444] REPLICATION: finished
0926 15:02:40.782441 (+ 467us) write_op.cc:301] APPLY: starting
0926 15:02:40.782459 (+ 18us) tablet.cc:1365] starting BulkCheckPresence
0926 15:02:40.782507 (+ 48us) tablet.cc:1368] finished BulkCheckPresence
0926 15:02:40.782535 (+ 28us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 15:02:40.782656 (+ 121us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 15:02:40.782695 (+ 39us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0926 15:02:40.782702 (+ 7us) write_op.cc:312] APPLY: finished
0926 15:02:40.782792 (+ 90us) log.cc:842] Serialized 28 byte log entry
0926 15:02:40.782863 (+ 71us) write_op.cc:489] Releasing partition, row and schema locks
0926 15:02:40.782886 (+ 23us) write_op.cc:454] Released schema lock
0926 15:02:40.782893 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0926 15:02:40.782965 (+ 72us) write_op.cc:489] Releasing partition, row and schema locks
0926 15:02:40.782969 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":2428669,"Hive Metastore.run_cpu_time_us":285,"Hive Metastore.run_wall_time_us":62886,"spinlock_wait_cycles":8960,"child_traces":[["op",{"apply.queue_time_us":403,"apply.run_cpu_time_us":585,"apply.run_wall_time_us":585,"num_ops":2,"prepare.queue_time_us":71,"prepare.run_cpu_time_us":930,"prepare.run_wall_time_us":931,"raft.queue_time_us":477,"raft.run_cpu_time_us":716,"raft.run_wall_time_us":716,"replication_time_us":14141,"spinlock_wait_cycles":125696}]]}
I20260926 15:02:40.797864 9465 tablet_service.cc:1467] Processing CreateTablet for tablet c8d090f2e0384924a72f288eff2281e7 (DEFAULT_TABLE table=default.table_22fd4957e3d14964803428457f8490ac [id=7ba7b39b84394421b88a6ce7616c3c91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:40.798269 9465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8d090f2e0384924a72f288eff2281e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:40.798779 9887 raft_consensus.cc:482] T 317305a7fdb54f0bb2bb569272550b7b P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:02:40.798955 9887 raft_consensus.cc:504] T 317305a7fdb54f0bb2bb569272550b7b P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.799607 9887 leader_election.cc:288] T 317305a7fdb54f0bb2bb569272550b7b P ae330d635b1848aaaecee2c7ce96520a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327), f052d5cbee65406593d8e44f74225bc2 (127.8.138.131:35445)
I20260926 15:02:40.800276 9345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "317305a7fdb54f0bb2bb569272550b7b" candidate_uuid: "ae330d635b1848aaaecee2c7ce96520a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ca148489d64deabd1dcbf7870c6d57" is_pre_election: true
I20260926 15:02:40.800503 9345 raft_consensus.cc:2454] T 317305a7fdb54f0bb2bb569272550b7b P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae330d635b1848aaaecee2c7ce96520a in term 0.
I20260926 15:02:40.800956 9421 leader_election.cc:302] T 317305a7fdb54f0bb2bb569272550b7b P ae330d635b1848aaaecee2c7ce96520a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ca148489d64deabd1dcbf7870c6d57, ae330d635b1848aaaecee2c7ce96520a; no voters:
I20260926 15:02:40.801242 9887 raft_consensus.cc:2781] T 317305a7fdb54f0bb2bb569272550b7b P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:02:40.801326 9887 raft_consensus.cc:482] T 317305a7fdb54f0bb2bb569272550b7b P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:02:40.801440 9887 raft_consensus.cc:3037] T 317305a7fdb54f0bb2bb569272550b7b P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:40.803031 9324 tablet_service.cc:1467] Processing CreateTablet for tablet c8d090f2e0384924a72f288eff2281e7 (DEFAULT_TABLE table=default.table_22fd4957e3d14964803428457f8490ac [id=7ba7b39b84394421b88a6ce7616c3c91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:40.803433 9324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8d090f2e0384924a72f288eff2281e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:40.804800 9887 raft_consensus.cc:504] T 317305a7fdb54f0bb2bb569272550b7b P ae330d635b1848aaaecee2c7ce96520a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.805847 9345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "317305a7fdb54f0bb2bb569272550b7b" candidate_uuid: "ae330d635b1848aaaecee2c7ce96520a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ca148489d64deabd1dcbf7870c6d57"
I20260926 15:02:40.806051 9345 raft_consensus.cc:3037] T 317305a7fdb54f0bb2bb569272550b7b P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:40.808938 9345 raft_consensus.cc:2454] T 317305a7fdb54f0bb2bb569272550b7b P 66ca148489d64deabd1dcbf7870c6d57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae330d635b1848aaaecee2c7ce96520a in term 1.
I20260926 15:02:40.809338 9421 leader_election.cc:302] T 317305a7fdb54f0bb2bb569272550b7b P ae330d635b1848aaaecee2c7ce96520a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ca148489d64deabd1dcbf7870c6d57, ae330d635b1848aaaecee2c7ce96520a; no voters:
I20260926 15:02:40.809710 9915 raft_consensus.cc:2781] T 317305a7fdb54f0bb2bb569272550b7b P ae330d635b1848aaaecee2c7ce96520a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:02:40.809859 9915 raft_consensus.cc:686] T 317305a7fdb54f0bb2bb569272550b7b P ae330d635b1848aaaecee2c7ce96520a [term 1 LEADER]: Becoming Leader. State: Replica: ae330d635b1848aaaecee2c7ce96520a, State: Running, Role: LEADER
I20260926 15:02:40.810123 9915 consensus_queue.cc:237] T 317305a7fdb54f0bb2bb569272550b7b P ae330d635b1848aaaecee2c7ce96520a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.811614 9887 leader_election.cc:288] T 317305a7fdb54f0bb2bb569272550b7b P ae330d635b1848aaaecee2c7ce96520a [CANDIDATE]: Term 1 election: Requested vote from peers 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327), f052d5cbee65406593d8e44f74225bc2 (127.8.138.131:35445)
I20260926 15:02:40.795696 9079 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:59430:
table { table_name: "default.Table_e3b713aac3c94e449bb5882310d55a40" } new_table_name: "default.Table_fd56d79387ed41638b65164801eaa9ab" modify_external_catalogs: true
I20260926 15:02:40.815608 9081 catalog_manager.cc:2148] created HMS catalog entry for table [id=3295fc8a0a004843be045b8bf6a4f2cf]
I20260926 15:02:40.815943 9611 tablet_service.cc:1467] Processing CreateTablet for tablet c8d090f2e0384924a72f288eff2281e7 (DEFAULT_TABLE table=default.table_22fd4957e3d14964803428457f8490ac [id=7ba7b39b84394421b88a6ce7616c3c91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:40.816462 9611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8d090f2e0384924a72f288eff2281e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:40.819133 9632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "317305a7fdb54f0bb2bb569272550b7b" candidate_uuid: "ae330d635b1848aaaecee2c7ce96520a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f052d5cbee65406593d8e44f74225bc2" is_pre_election: true
I20260926 15:02:40.819541 9632 raft_consensus.cc:2454] T 317305a7fdb54f0bb2bb569272550b7b P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae330d635b1848aaaecee2c7ce96520a in term 0.
I20260926 15:02:40.820499 9632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "317305a7fdb54f0bb2bb569272550b7b" candidate_uuid: "ae330d635b1848aaaecee2c7ce96520a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f052d5cbee65406593d8e44f74225bc2"
I20260926 15:02:40.820699 9632 raft_consensus.cc:3037] T 317305a7fdb54f0bb2bb569272550b7b P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Advancing to term 1
F20260926 15:02:40.824744 9249 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_f356bc0dfc6a47f98430002dccbb873f
*** Check failure stack trace: ***
I20260926 15:02:40.826583 9632 raft_consensus.cc:2454] T 317305a7fdb54f0bb2bb569272550b7b P f052d5cbee65406593d8e44f74225bc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae330d635b1848aaaecee2c7ce96520a in term 1.
I20260926 15:02:40.827234 9081 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:59430 (request call id 12) took 2575 ms. Trace:
@ 0x7fc76bb4f9ad google::LogMessage::Fail() at ??:0
I20260926 15:02:40.827404 9081 rpcz_store.cc:276] 0926 15:02:38.252081 (+ 0us) service_pool.cc:170] Inserting onto call queue
0926 15:02:38.254459 (+ 2378us) service_pool.cc:227] Handling call
0926 15:02:38.275049 (+ 20590us) catalog_manager.cc:2075] Acquired catalog manager lock
0926 15:02:38.275538 (+ 489us) catalog_manager.cc:2121] Created new table and tablet info
0926 15:02:40.815529 (+2539991us) catalog_manager.cc:2147] Created new table in HMS catalog
0926 15:02:40.826893 (+ 11364us) catalog_manager.cc:2181] Wrote table and tablets to system table
0926 15:02:40.827058 (+ 165us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0926 15:02:40.827185 (+ 127us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0926 15:02:40.816639 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 15:02:40.816749 (+ 110us) write_op.cc:432] Acquiring schema lock in shared mode
0926 15:02:40.816755 (+ 6us) write_op.cc:435] Acquired schema lock
0926 15:02:40.816757 (+ 2us) tablet.cc:660] Decoding operations
0926 15:02:40.816854 (+ 97us) write_op.cc:620] Acquiring the partition lock for write op
0926 15:02:40.816870 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0926 15:02:40.816875 (+ 5us) tablet.cc:683] Acquiring locks for 2 operations
0926 15:02:40.816991 (+ 116us) tablet.cc:699] Row locks acquired
0926 15:02:40.816995 (+ 4us) write_op.cc:260] PREPARE: finished
0926 15:02:40.817037 (+ 42us) write_op.cc:270] Start()
0926 15:02:40.817075 (+ 38us) write_op.cc:276] Timestamp: P: 1790434960816402 usec, L: 0
0926 15:02:40.817079 (+ 4us) op_driver.cc:331] REPLICATION: starting
0926 15:02:40.817288 (+ 209us) log.cc:842] Serialized 432 byte log entry
0926 15:02:40.823300 (+ 6012us) op_driver.cc:444] REPLICATION: finished
0926 15:02:40.823397 (+ 97us) write_op.cc:301] APPLY: starting
0926 15:02:40.823412 (+ 15us) tablet.cc:1365] starting BulkCheckPresence
0926 15:02:40.823458 (+ 46us) tablet.cc:1368] finished BulkCheckPresence
0926 15:02:40.823499 (+ 41us) tablet.cc:1370] starting ApplyRowOperation cycle
0926 15:02:40.823637 (+ 138us) tablet.cc:1381] finished ApplyRowOperation cycle
0926 15:02:40.823672 (+ 35us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0926 15:02:40.823679 (+ 7us) write_op.cc:312] APPLY: finished
0926 15:02:40.823756 (+ 77us) log.cc:842] Serialized 28 byte log entry
0926 15:02:40.823812 (+ 56us) write_op.cc:489] Releasing partition, row and schema locks
0926 15:02:40.823833 (+ 21us) write_op.cc:454] Released schema lock
0926 15:02:40.823840 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0926 15:02:40.824792 (+ 952us) write_op.cc:489] Releasing partition, row and schema locks
0926 15:02:40.824798 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":2490675,"Hive Metastore.run_cpu_time_us":312,"Hive Metastore.run_wall_time_us":49114,"child_traces":[["op",{"apply.queue_time_us":55,"apply.run_cpu_time_us":604,"apply.run_wall_time_us":1453,"num_ops":2,"prepare.queue_time_us":115,"prepare.run_cpu_time_us":763,"prepare.run_wall_time_us":763,"raft.queue_time_us":1296,"raft.run_cpu_time_us":516,"raft.run_wall_time_us":515,"replication_time_us":6198}]]}
@ 0x7fc76bb5288a google::LogMessage::SendToLog() at ??:0
@ 0x7fc76bb4f4de google::LogMessage::Flush() at ??:0
@ 0x7fc76bb4f6d1 google::LogMessage::~LogMessage() at ??:0
I20260926 15:02:40.835440 9897 tablet_bootstrap.cc:492] T c8d090f2e0384924a72f288eff2281e7 P f052d5cbee65406593d8e44f74225bc2: Bootstrap starting.
I20260926 15:02:40.836149 9899 tablet_bootstrap.cc:492] T c8d090f2e0384924a72f288eff2281e7 P ae330d635b1848aaaecee2c7ce96520a: Bootstrap starting.
I20260926 15:02:40.837638 9897 tablet_bootstrap.cc:654] T c8d090f2e0384924a72f288eff2281e7 P f052d5cbee65406593d8e44f74225bc2: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:40.837770 9899 tablet_bootstrap.cc:654] T c8d090f2e0384924a72f288eff2281e7 P ae330d635b1848aaaecee2c7ce96520a: Neither blocks nor log segments found. Creating new log.
@ 0x7fc7816edba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260926 15:02:40.844074 9898 tablet_bootstrap.cc:492] T c8d090f2e0384924a72f288eff2281e7 P 66ca148489d64deabd1dcbf7870c6d57: Bootstrap starting.
I20260926 15:02:40.845302 9325 tablet_service.cc:1467] Processing CreateTablet for tablet 51ff1e9e47864b24b7c7a869ff37e06a (DEFAULT_TABLE table=default.table_854219551f9048c6925c711ce33d5256 [id=3295fc8a0a004843be045b8bf6a4f2cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:40.845738 9325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51ff1e9e47864b24b7c7a869ff37e06a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:40.845849 9898 tablet_bootstrap.cc:654] T c8d090f2e0384924a72f288eff2281e7 P 66ca148489d64deabd1dcbf7870c6d57: Neither blocks nor log segments found. Creating new log.
@ 0x7fc7816ec901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260926 15:02:40.853953 9465 tablet_service.cc:1467] Processing CreateTablet for tablet 51ff1e9e47864b24b7c7a869ff37e06a (DEFAULT_TABLE table=default.table_854219551f9048c6925c711ce33d5256 [id=3295fc8a0a004843be045b8bf6a4f2cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:40.854408 9465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51ff1e9e47864b24b7c7a869ff37e06a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:40.857961 9611 tablet_service.cc:1467] Processing CreateTablet for tablet 51ff1e9e47864b24b7c7a869ff37e06a (DEFAULT_TABLE table=default.table_854219551f9048c6925c711ce33d5256 [id=3295fc8a0a004843be045b8bf6a4f2cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:40.858428 9611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51ff1e9e47864b24b7c7a869ff37e06a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:40.858569 9897 tablet_bootstrap.cc:492] T c8d090f2e0384924a72f288eff2281e7 P f052d5cbee65406593d8e44f74225bc2: No bootstrap required, opened a new log
I20260926 15:02:40.858712 9897 ts_tablet_manager.cc:1397] T c8d090f2e0384924a72f288eff2281e7 P f052d5cbee65406593d8e44f74225bc2: Time spent bootstrapping tablet: real 0.023s user 0.005s sys 0.005s
I20260926 15:02:40.859609 9897 raft_consensus.cc:348] T c8d090f2e0384924a72f288eff2281e7 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.859900 9897 raft_consensus.cc:374] T c8d090f2e0384924a72f288eff2281e7 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:40.859990 9897 raft_consensus.cc:729] T c8d090f2e0384924a72f288eff2281e7 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f052d5cbee65406593d8e44f74225bc2, State: Initialized, Role: FOLLOWER
I20260926 15:02:40.860257 9897 consensus_queue.cc:260] T c8d090f2e0384924a72f288eff2281e7 P f052d5cbee65406593d8e44f74225bc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.864437 9897 ts_tablet_manager.cc:1428] T c8d090f2e0384924a72f288eff2281e7 P f052d5cbee65406593d8e44f74225bc2: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260926 15:02:40.865458 9898 tablet_bootstrap.cc:492] T c8d090f2e0384924a72f288eff2281e7 P 66ca148489d64deabd1dcbf7870c6d57: No bootstrap required, opened a new log
I20260926 15:02:40.865635 9898 ts_tablet_manager.cc:1397] T c8d090f2e0384924a72f288eff2281e7 P 66ca148489d64deabd1dcbf7870c6d57: Time spent bootstrapping tablet: real 0.022s user 0.006s sys 0.000s
I20260926 15:02:40.866407 9898 raft_consensus.cc:348] T c8d090f2e0384924a72f288eff2281e7 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.866657 9898 raft_consensus.cc:374] T c8d090f2e0384924a72f288eff2281e7 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:40.866735 9898 raft_consensus.cc:729] T c8d090f2e0384924a72f288eff2281e7 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ca148489d64deabd1dcbf7870c6d57, State: Initialized, Role: FOLLOWER
I20260926 15:02:40.866956 9897 tablet_bootstrap.cc:492] T 51ff1e9e47864b24b7c7a869ff37e06a P f052d5cbee65406593d8e44f74225bc2: Bootstrap starting.
I20260926 15:02:40.867012 9898 consensus_queue.cc:260] T c8d090f2e0384924a72f288eff2281e7 P 66ca148489d64deabd1dcbf7870c6d57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.868420 9081 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59430:
table { table_name: "default.Table_7b56039466d14dcab4dc63192de3b08e" } modify_external_catalogs: true
@ 0x7fc7816f366c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260926 15:02:40.869889 9081 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59430:
table { table_name: "default.Table_7b56039466d14dcab4dc63192de3b08e" } modify_external_catalogs: true
I20260926 15:02:40.871248 9899 tablet_bootstrap.cc:492] T c8d090f2e0384924a72f288eff2281e7 P ae330d635b1848aaaecee2c7ce96520a: No bootstrap required, opened a new log
I20260926 15:02:40.871371 9899 ts_tablet_manager.cc:1397] T c8d090f2e0384924a72f288eff2281e7 P ae330d635b1848aaaecee2c7ce96520a: Time spent bootstrapping tablet: real 0.035s user 0.006s sys 0.003s
I20260926 15:02:40.871591 9898 ts_tablet_manager.cc:1428] T c8d090f2e0384924a72f288eff2281e7 P 66ca148489d64deabd1dcbf7870c6d57: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260926 15:02:40.871971 9899 raft_consensus.cc:348] T c8d090f2e0384924a72f288eff2281e7 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.872150 9899 raft_consensus.cc:374] T c8d090f2e0384924a72f288eff2281e7 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:40.872196 9899 raft_consensus.cc:729] T c8d090f2e0384924a72f288eff2281e7 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae330d635b1848aaaecee2c7ce96520a, State: Initialized, Role: FOLLOWER
I20260926 15:02:40.872424 9899 consensus_queue.cc:260] T c8d090f2e0384924a72f288eff2281e7 P ae330d635b1848aaaecee2c7ce96520a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.872905 9899 ts_tablet_manager.cc:1428] T c8d090f2e0384924a72f288eff2281e7 P ae330d635b1848aaaecee2c7ce96520a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 15:02:40.874154 9899 tablet_bootstrap.cc:492] T 51ff1e9e47864b24b7c7a869ff37e06a P ae330d635b1848aaaecee2c7ce96520a: Bootstrap starting.
@ 0x7fc7816f34e9 std::_Function_handler<>::_M_invoke() at ??:0
I20260926 15:02:40.874919 9898 tablet_bootstrap.cc:492] T 51ff1e9e47864b24b7c7a869ff37e06a P 66ca148489d64deabd1dcbf7870c6d57: Bootstrap starting.
I20260926 15:02:40.875782 9899 tablet_bootstrap.cc:654] T 51ff1e9e47864b24b7c7a869ff37e06a P ae330d635b1848aaaecee2c7ce96520a: Neither blocks nor log segments found. Creating new log.
@ 0x9edb7d std::function<>::operator()() at ??:?
I20260926 15:02:40.877103 9898 tablet_bootstrap.cc:654] T 51ff1e9e47864b24b7c7a869ff37e06a P 66ca148489d64deabd1dcbf7870c6d57: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:40.881795 9897 tablet_bootstrap.cc:654] T 51ff1e9e47864b24b7c7a869ff37e06a P f052d5cbee65406593d8e44f74225bc2: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:40.882905 9898 tablet_bootstrap.cc:492] T 51ff1e9e47864b24b7c7a869ff37e06a P 66ca148489d64deabd1dcbf7870c6d57: No bootstrap required, opened a new log
I20260926 15:02:40.883049 9898 ts_tablet_manager.cc:1397] T 51ff1e9e47864b24b7c7a869ff37e06a P 66ca148489d64deabd1dcbf7870c6d57: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.004s
I20260926 15:02:40.883961 9898 raft_consensus.cc:348] T 51ff1e9e47864b24b7c7a869ff37e06a P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.884167 9898 raft_consensus.cc:374] T 51ff1e9e47864b24b7c7a869ff37e06a P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:40.884256 9898 raft_consensus.cc:729] T 51ff1e9e47864b24b7c7a869ff37e06a P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ca148489d64deabd1dcbf7870c6d57, State: Initialized, Role: FOLLOWER
I20260926 15:02:40.884478 9898 consensus_queue.cc:260] T 51ff1e9e47864b24b7c7a869ff37e06a P 66ca148489d64deabd1dcbf7870c6d57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
@ 0x7fc76e779eaf kudu::Thread::SuperviseThread() at ??:0
I20260926 15:02:40.889894 9898 ts_tablet_manager.cc:1428] T 51ff1e9e47864b24b7c7a869ff37e06a P 66ca148489d64deabd1dcbf7870c6d57: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260926 15:02:40.890074 9899 tablet_bootstrap.cc:492] T 51ff1e9e47864b24b7c7a869ff37e06a P ae330d635b1848aaaecee2c7ce96520a: No bootstrap required, opened a new log
I20260926 15:02:40.890228 9899 ts_tablet_manager.cc:1397] T 51ff1e9e47864b24b7c7a869ff37e06a P ae330d635b1848aaaecee2c7ce96520a: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.000s
I20260926 15:02:40.891000 9899 raft_consensus.cc:348] T 51ff1e9e47864b24b7c7a869ff37e06a P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.891220 9899 raft_consensus.cc:374] T 51ff1e9e47864b24b7c7a869ff37e06a P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
@ 0x7fc771a536db start_thread at ??:0
I20260926 15:02:40.891310 9899 raft_consensus.cc:729] T 51ff1e9e47864b24b7c7a869ff37e06a P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae330d635b1848aaaecee2c7ce96520a, State: Initialized, Role: FOLLOWER
I20260926 15:02:40.891537 9899 consensus_queue.cc:260] T 51ff1e9e47864b24b7c7a869ff37e06a P ae330d635b1848aaaecee2c7ce96520a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
@ 0x7fc76a9e961f clone at ??:0
I20260926 15:02:40.895614 9899 ts_tablet_manager.cc:1428] T 51ff1e9e47864b24b7c7a869ff37e06a P ae330d635b1848aaaecee2c7ce96520a: Time spent starting tablet: real 0.005s user 0.002s sys 0.001s
I20260926 15:02:40.896409 9758 raft_consensus.cc:482] T 41e641e28d1f48b2a7e0c8fb8964f678 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:02:40.896610 9758 raft_consensus.cc:504] T 41e641e28d1f48b2a7e0c8fb8964f678 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.897303 9758 leader_election.cc:288] T 41e641e28d1f48b2a7e0c8fb8964f678 P f052d5cbee65406593d8e44f74225bc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327), ae330d635b1848aaaecee2c7ce96520a (127.8.138.130:44309)
I20260926 15:02:40.897727 9345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41e641e28d1f48b2a7e0c8fb8964f678" candidate_uuid: "f052d5cbee65406593d8e44f74225bc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ca148489d64deabd1dcbf7870c6d57" is_pre_election: true
I20260926 15:02:40.897971 9345 raft_consensus.cc:2454] T 41e641e28d1f48b2a7e0c8fb8964f678 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f052d5cbee65406593d8e44f74225bc2 in term 0.
I20260926 15:02:40.898371 9566 leader_election.cc:302] T 41e641e28d1f48b2a7e0c8fb8964f678 P f052d5cbee65406593d8e44f74225bc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ca148489d64deabd1dcbf7870c6d57, f052d5cbee65406593d8e44f74225bc2; no voters:
I20260926 15:02:40.898603 9487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41e641e28d1f48b2a7e0c8fb8964f678" candidate_uuid: "f052d5cbee65406593d8e44f74225bc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae330d635b1848aaaecee2c7ce96520a" is_pre_election: true
I20260926 15:02:40.898890 9487 raft_consensus.cc:2454] T 41e641e28d1f48b2a7e0c8fb8964f678 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f052d5cbee65406593d8e44f74225bc2 in term 0.
I20260926 15:02:40.900490 9758 raft_consensus.cc:2781] T 41e641e28d1f48b2a7e0c8fb8964f678 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:02:40.900607 9758 raft_consensus.cc:482] T 41e641e28d1f48b2a7e0c8fb8964f678 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:02:40.900733 9758 raft_consensus.cc:3037] T 41e641e28d1f48b2a7e0c8fb8964f678 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:40.904201 9758 raft_consensus.cc:504] T 41e641e28d1f48b2a7e0c8fb8964f678 P f052d5cbee65406593d8e44f74225bc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.905458 9345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41e641e28d1f48b2a7e0c8fb8964f678" candidate_uuid: "f052d5cbee65406593d8e44f74225bc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ca148489d64deabd1dcbf7870c6d57"
I20260926 15:02:40.905603 9897 tablet_bootstrap.cc:492] T 51ff1e9e47864b24b7c7a869ff37e06a P f052d5cbee65406593d8e44f74225bc2: No bootstrap required, opened a new log
I20260926 15:02:40.905498 9487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41e641e28d1f48b2a7e0c8fb8964f678" candidate_uuid: "f052d5cbee65406593d8e44f74225bc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae330d635b1848aaaecee2c7ce96520a"
I20260926 15:02:40.905686 9345 raft_consensus.cc:3037] T 41e641e28d1f48b2a7e0c8fb8964f678 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:40.905754 9897 ts_tablet_manager.cc:1397] T 51ff1e9e47864b24b7c7a869ff37e06a P f052d5cbee65406593d8e44f74225bc2: Time spent bootstrapping tablet: real 0.039s user 0.000s sys 0.004s
I20260926 15:02:40.905797 9487 raft_consensus.cc:3037] T 41e641e28d1f48b2a7e0c8fb8964f678 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:40.906551 9897 raft_consensus.cc:348] T 51ff1e9e47864b24b7c7a869ff37e06a P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.906759 9897 raft_consensus.cc:374] T 51ff1e9e47864b24b7c7a869ff37e06a P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:40.906840 9897 raft_consensus.cc:729] T 51ff1e9e47864b24b7c7a869ff37e06a P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f052d5cbee65406593d8e44f74225bc2, State: Initialized, Role: FOLLOWER
I20260926 15:02:40.907038 9897 consensus_queue.cc:260] T 51ff1e9e47864b24b7c7a869ff37e06a P f052d5cbee65406593d8e44f74225bc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.908718 9345 raft_consensus.cc:2454] T 41e641e28d1f48b2a7e0c8fb8964f678 P 66ca148489d64deabd1dcbf7870c6d57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f052d5cbee65406593d8e44f74225bc2 in term 1.
I20260926 15:02:40.909253 9566 leader_election.cc:302] T 41e641e28d1f48b2a7e0c8fb8964f678 P f052d5cbee65406593d8e44f74225bc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ca148489d64deabd1dcbf7870c6d57, f052d5cbee65406593d8e44f74225bc2; no voters:
I20260926 15:02:40.909545 9930 raft_consensus.cc:2781] T 41e641e28d1f48b2a7e0c8fb8964f678 P f052d5cbee65406593d8e44f74225bc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:02:40.909977 9897 ts_tablet_manager.cc:1428] T 51ff1e9e47864b24b7c7a869ff37e06a P f052d5cbee65406593d8e44f74225bc2: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260926 15:02:40.910352 9880 raft_consensus.cc:482] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:02:40.910511 9880 raft_consensus.cc:504] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.911056 9880 leader_election.cc:288] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P 66ca148489d64deabd1dcbf7870c6d57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae330d635b1848aaaecee2c7ce96520a (127.8.138.130:44309), f052d5cbee65406593d8e44f74225bc2 (127.8.138.131:35445)
I20260926 15:02:40.911723 9632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9134d14fae4d46b6bd8ef1c05dc5b1c2" candidate_uuid: "66ca148489d64deabd1dcbf7870c6d57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f052d5cbee65406593d8e44f74225bc2" is_pre_election: true
I20260926 15:02:40.911963 9632 raft_consensus.cc:2454] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66ca148489d64deabd1dcbf7870c6d57 in term 0.
I20260926 15:02:40.912568 9278 leader_election.cc:302] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P 66ca148489d64deabd1dcbf7870c6d57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ca148489d64deabd1dcbf7870c6d57, f052d5cbee65406593d8e44f74225bc2; no voters:
I20260926 15:02:40.912575 9486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9134d14fae4d46b6bd8ef1c05dc5b1c2" candidate_uuid: "66ca148489d64deabd1dcbf7870c6d57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae330d635b1848aaaecee2c7ce96520a" is_pre_election: true
I20260926 15:02:40.912827 9880 raft_consensus.cc:2781] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:02:40.912914 9880 raft_consensus.cc:482] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:02:40.912873 9486 raft_consensus.cc:2454] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66ca148489d64deabd1dcbf7870c6d57 in term 0.
I20260926 15:02:40.912997 9880 raft_consensus.cc:3037] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:40.915650 9880 raft_consensus.cc:504] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P 66ca148489d64deabd1dcbf7870c6d57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.916800 9632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9134d14fae4d46b6bd8ef1c05dc5b1c2" candidate_uuid: "66ca148489d64deabd1dcbf7870c6d57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f052d5cbee65406593d8e44f74225bc2"
I20260926 15:02:40.916997 9632 raft_consensus.cc:3037] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:40.917316 9486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9134d14fae4d46b6bd8ef1c05dc5b1c2" candidate_uuid: "66ca148489d64deabd1dcbf7870c6d57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae330d635b1848aaaecee2c7ce96520a"
I20260926 15:02:40.917538 9486 raft_consensus.cc:3037] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:40.919695 9632 raft_consensus.cc:2454] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P f052d5cbee65406593d8e44f74225bc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ca148489d64deabd1dcbf7870c6d57 in term 1.
I20260926 15:02:40.920126 9278 leader_election.cc:302] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P 66ca148489d64deabd1dcbf7870c6d57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ca148489d64deabd1dcbf7870c6d57, f052d5cbee65406593d8e44f74225bc2; no voters:
I20260926 15:02:40.920912 9880 leader_election.cc:288] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P 66ca148489d64deabd1dcbf7870c6d57 [CANDIDATE]: Term 1 election: Requested vote from peers ae330d635b1848aaaecee2c7ce96520a (127.8.138.130:44309), f052d5cbee65406593d8e44f74225bc2 (127.8.138.131:35445)
I20260926 15:02:40.921067 9880 raft_consensus.cc:2781] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P 66ca148489d64deabd1dcbf7870c6d57 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:02:40.921175 9880 raft_consensus.cc:686] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P 66ca148489d64deabd1dcbf7870c6d57 [term 1 LEADER]: Becoming Leader. State: Replica: 66ca148489d64deabd1dcbf7870c6d57, State: Running, Role: LEADER
I20260926 15:02:40.921437 9880 consensus_queue.cc:237] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P 66ca148489d64deabd1dcbf7870c6d57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:40.926352 9486 raft_consensus.cc:2454] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P ae330d635b1848aaaecee2c7ce96520a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ca148489d64deabd1dcbf7870c6d57 in term 1.
I20260926 15:02:40.928435 9487 raft_consensus.cc:2454] T 41e641e28d1f48b2a7e0c8fb8964f678 P ae330d635b1848aaaecee2c7ce96520a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f052d5cbee65406593d8e44f74225bc2 in term 1.
I20260926 15:02:40.929214 9758 leader_election.cc:288] T 41e641e28d1f48b2a7e0c8fb8964f678 P f052d5cbee65406593d8e44f74225bc2 [CANDIDATE]: Term 1 election: Requested vote from peers 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327), ae330d635b1848aaaecee2c7ce96520a (127.8.138.130:44309)
I20260926 15:02:40.930548 9930 raft_consensus.cc:686] T 41e641e28d1f48b2a7e0c8fb8964f678 P f052d5cbee65406593d8e44f74225bc2 [term 1 LEADER]: Becoming Leader. State: Replica: f052d5cbee65406593d8e44f74225bc2, State: Running, Role: LEADER
I20260926 15:02:40.930922 9930 consensus_queue.cc:237] T 41e641e28d1f48b2a7e0c8fb8964f678 P f052d5cbee65406593d8e44f74225bc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:41.023556 9915 raft_consensus.cc:482] T 51ff1e9e47864b24b7c7a869ff37e06a P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:02:41.023813 9915 raft_consensus.cc:504] T 51ff1e9e47864b24b7c7a869ff37e06a P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:41.024541 9915 leader_election.cc:288] T 51ff1e9e47864b24b7c7a869ff37e06a P ae330d635b1848aaaecee2c7ce96520a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327), f052d5cbee65406593d8e44f74225bc2 (127.8.138.131:35445)
I20260926 15:02:41.025218 9345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51ff1e9e47864b24b7c7a869ff37e06a" candidate_uuid: "ae330d635b1848aaaecee2c7ce96520a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ca148489d64deabd1dcbf7870c6d57" is_pre_election: true
I20260926 15:02:41.025252 9632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51ff1e9e47864b24b7c7a869ff37e06a" candidate_uuid: "ae330d635b1848aaaecee2c7ce96520a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f052d5cbee65406593d8e44f74225bc2" is_pre_election: true
I20260926 15:02:41.025496 9345 raft_consensus.cc:2454] T 51ff1e9e47864b24b7c7a869ff37e06a P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae330d635b1848aaaecee2c7ce96520a in term 0.
I20260926 15:02:41.025530 9632 raft_consensus.cc:2454] T 51ff1e9e47864b24b7c7a869ff37e06a P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae330d635b1848aaaecee2c7ce96520a in term 0.
I20260926 15:02:41.025983 9421 leader_election.cc:302] T 51ff1e9e47864b24b7c7a869ff37e06a P ae330d635b1848aaaecee2c7ce96520a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ca148489d64deabd1dcbf7870c6d57, ae330d635b1848aaaecee2c7ce96520a; no voters:
I20260926 15:02:41.026309 9915 raft_consensus.cc:2781] T 51ff1e9e47864b24b7c7a869ff37e06a P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:02:41.026392 9915 raft_consensus.cc:482] T 51ff1e9e47864b24b7c7a869ff37e06a P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:02:41.026505 9915 raft_consensus.cc:3037] T 51ff1e9e47864b24b7c7a869ff37e06a P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:41.028543 9915 raft_consensus.cc:504] T 51ff1e9e47864b24b7c7a869ff37e06a P ae330d635b1848aaaecee2c7ce96520a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:41.029107 9915 leader_election.cc:288] T 51ff1e9e47864b24b7c7a869ff37e06a P ae330d635b1848aaaecee2c7ce96520a [CANDIDATE]: Term 1 election: Requested vote from peers 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327), f052d5cbee65406593d8e44f74225bc2 (127.8.138.131:35445)
I20260926 15:02:41.029891 9632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51ff1e9e47864b24b7c7a869ff37e06a" candidate_uuid: "ae330d635b1848aaaecee2c7ce96520a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f052d5cbee65406593d8e44f74225bc2"
I20260926 15:02:41.029907 9345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51ff1e9e47864b24b7c7a869ff37e06a" candidate_uuid: "ae330d635b1848aaaecee2c7ce96520a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ca148489d64deabd1dcbf7870c6d57"
I20260926 15:02:41.030160 9345 raft_consensus.cc:3037] T 51ff1e9e47864b24b7c7a869ff37e06a P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:41.030190 9632 raft_consensus.cc:3037] T 51ff1e9e47864b24b7c7a869ff37e06a P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:41.032266 9632 raft_consensus.cc:2454] T 51ff1e9e47864b24b7c7a869ff37e06a P f052d5cbee65406593d8e44f74225bc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae330d635b1848aaaecee2c7ce96520a in term 1.
I20260926 15:02:41.032735 9420 leader_election.cc:302] T 51ff1e9e47864b24b7c7a869ff37e06a P ae330d635b1848aaaecee2c7ce96520a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae330d635b1848aaaecee2c7ce96520a, f052d5cbee65406593d8e44f74225bc2; no voters:
I20260926 15:02:41.033008 9915 raft_consensus.cc:2781] T 51ff1e9e47864b24b7c7a869ff37e06a P ae330d635b1848aaaecee2c7ce96520a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:02:41.033209 9915 raft_consensus.cc:686] T 51ff1e9e47864b24b7c7a869ff37e06a P ae330d635b1848aaaecee2c7ce96520a [term 1 LEADER]: Becoming Leader. State: Replica: ae330d635b1848aaaecee2c7ce96520a, State: Running, Role: LEADER
I20260926 15:02:41.033190 9345 raft_consensus.cc:2454] T 51ff1e9e47864b24b7c7a869ff37e06a P 66ca148489d64deabd1dcbf7870c6d57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae330d635b1848aaaecee2c7ce96520a in term 1.
I20260926 15:02:41.033468 9915 consensus_queue.cc:237] T 51ff1e9e47864b24b7c7a869ff37e06a P ae330d635b1848aaaecee2c7ce96520a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:41.068418 9933 consensus_queue.cc:1035] T c5d07d3a7a164d098e61f8f7f2ac4a67 P 66ca148489d64deabd1dcbf7870c6d57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:02:41.077805 9880 consensus_queue.cc:1035] T c5d07d3a7a164d098e61f8f7f2ac4a67 P 66ca148489d64deabd1dcbf7870c6d57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:02:41.124059 9930 consensus_queue.cc:1035] T 3fae8a76002849de9bb26610a0c33406 P f052d5cbee65406593d8e44f74225bc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:02:41.130976 9931 consensus_queue.cc:1035] T 3fae8a76002849de9bb26610a0c33406 P f052d5cbee65406593d8e44f74225bc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 15:02:41.182099 9690 connection.cc:535] client connection to 127.8.138.189:37105 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 15:02:41.182957 9690 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.8.138.189:37105)
W20260926 15:02:41.183583 9690 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.8.138.189:37105)
W20260926 15:02:41.183980 9690 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.8.138.189:37105)
W20260926 15:02:41.185120 9690 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.8.138.189:37105)
I20260926 15:02:41.186405 9124 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: recv got EOF from 127.8.138.189:37105 (error 108),
I20260926 15:02:41.186604 9124 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:02:41.182646 9740 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: recv got EOF from 127.8.138.189:37105 (error 108),
I20260926 15:02:41.196195 9740 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:02:41.275808 9933 raft_consensus.cc:482] T 0d837276d648440cbebe1f88c281cb11 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:02:41.276098 9933 raft_consensus.cc:504] T 0d837276d648440cbebe1f88c281cb11 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:41.276854 9933 leader_election.cc:288] T 0d837276d648440cbebe1f88c281cb11 P 66ca148489d64deabd1dcbf7870c6d57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae330d635b1848aaaecee2c7ce96520a (127.8.138.130:44309), f052d5cbee65406593d8e44f74225bc2 (127.8.138.131:35445)
I20260926 15:02:41.277663 9487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d837276d648440cbebe1f88c281cb11" candidate_uuid: "66ca148489d64deabd1dcbf7870c6d57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae330d635b1848aaaecee2c7ce96520a" is_pre_election: true
I20260926 15:02:41.277781 9632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d837276d648440cbebe1f88c281cb11" candidate_uuid: "66ca148489d64deabd1dcbf7870c6d57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f052d5cbee65406593d8e44f74225bc2" is_pre_election: true
I20260926 15:02:41.277904 9487 raft_consensus.cc:2454] T 0d837276d648440cbebe1f88c281cb11 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66ca148489d64deabd1dcbf7870c6d57 in term 0.
I20260926 15:02:41.278023 9632 raft_consensus.cc:2454] T 0d837276d648440cbebe1f88c281cb11 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66ca148489d64deabd1dcbf7870c6d57 in term 0.
I20260926 15:02:41.278368 9280 leader_election.cc:302] T 0d837276d648440cbebe1f88c281cb11 P 66ca148489d64deabd1dcbf7870c6d57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ca148489d64deabd1dcbf7870c6d57, ae330d635b1848aaaecee2c7ce96520a; no voters:
I20260926 15:02:41.278748 9933 raft_consensus.cc:2781] T 0d837276d648440cbebe1f88c281cb11 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:02:41.278838 9933 raft_consensus.cc:482] T 0d837276d648440cbebe1f88c281cb11 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:02:41.278936 9933 raft_consensus.cc:3037] T 0d837276d648440cbebe1f88c281cb11 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:41.281834 9933 raft_consensus.cc:504] T 0d837276d648440cbebe1f88c281cb11 P 66ca148489d64deabd1dcbf7870c6d57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:41.282975 9487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d837276d648440cbebe1f88c281cb11" candidate_uuid: "66ca148489d64deabd1dcbf7870c6d57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae330d635b1848aaaecee2c7ce96520a"
I20260926 15:02:41.283177 9487 raft_consensus.cc:3037] T 0d837276d648440cbebe1f88c281cb11 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:41.284878 9933 leader_election.cc:288] T 0d837276d648440cbebe1f88c281cb11 P 66ca148489d64deabd1dcbf7870c6d57 [CANDIDATE]: Term 1 election: Requested vote from peers ae330d635b1848aaaecee2c7ce96520a (127.8.138.130:44309), f052d5cbee65406593d8e44f74225bc2 (127.8.138.131:35445)
I20260926 15:02:41.286000 9487 raft_consensus.cc:2454] T 0d837276d648440cbebe1f88c281cb11 P ae330d635b1848aaaecee2c7ce96520a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ca148489d64deabd1dcbf7870c6d57 in term 1.
I20260926 15:02:41.286506 9280 leader_election.cc:302] T 0d837276d648440cbebe1f88c281cb11 P 66ca148489d64deabd1dcbf7870c6d57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ca148489d64deabd1dcbf7870c6d57, ae330d635b1848aaaecee2c7ce96520a; no voters:
I20260926 15:02:41.286860 9933 raft_consensus.cc:2781] T 0d837276d648440cbebe1f88c281cb11 P 66ca148489d64deabd1dcbf7870c6d57 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:02:41.286989 9933 raft_consensus.cc:686] T 0d837276d648440cbebe1f88c281cb11 P 66ca148489d64deabd1dcbf7870c6d57 [term 1 LEADER]: Becoming Leader. State: Replica: 66ca148489d64deabd1dcbf7870c6d57, State: Running, Role: LEADER
I20260926 15:02:41.287211 9933 consensus_queue.cc:237] T 0d837276d648440cbebe1f88c281cb11 P 66ca148489d64deabd1dcbf7870c6d57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:41.292356 9632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d837276d648440cbebe1f88c281cb11" candidate_uuid: "66ca148489d64deabd1dcbf7870c6d57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f052d5cbee65406593d8e44f74225bc2"
I20260926 15:02:41.292595 9632 raft_consensus.cc:3037] T 0d837276d648440cbebe1f88c281cb11 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:41.296999 9632 raft_consensus.cc:2454] T 0d837276d648440cbebe1f88c281cb11 P f052d5cbee65406593d8e44f74225bc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66ca148489d64deabd1dcbf7870c6d57 in term 1.
I20260926 15:02:41.355589 9961 raft_consensus.cc:482] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader be9958f2085242f7ab82ae933ceb1c42)
I20260926 15:02:41.355913 9961 raft_consensus.cc:504] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:41.356364 9962 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader be9958f2085242f7ab82ae933ceb1c42)
I20260926 15:02:41.356608 9962 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:41.356814 9961 leader_election.cc:288] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105)
I20260926 15:02:41.357466 9813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "320cd09624624312bf1bcc0856c2874f" candidate_term: 5 candidate_status { last_received { term: 4 index: 34 } } ignore_live_leader: false dest_uuid: "6315fb5099e1420b95b8d83b3e19375c" is_pre_election: true
I20260926 15:02:41.357897 9813 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 320cd09624624312bf1bcc0856c2874f in term 4.
I20260926 15:02:41.358599 9142 leader_election.cc:302] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f, 6315fb5099e1420b95b8d83b3e19375c; no voters:
I20260926 15:02:41.358979 9961 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260926 15:02:41.359128 9961 raft_consensus.cc:482] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 4 FOLLOWER]: Starting leader election (detected failure of leader be9958f2085242f7ab82ae933ceb1c42)
I20260926 15:02:41.359277 9961 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 4 FOLLOWER]: Advancing to term 5
W20260926 15:02:41.359211 9145 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:02:41.363758 9961 raft_consensus.cc:504] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:41.364836 9962 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
I20260926 15:02:41.365588 9961 leader_election.cc:288] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 5 election: Requested vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105)
I20260926 15:02:41.365949 9813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "320cd09624624312bf1bcc0856c2874f" candidate_term: 5 candidate_status { last_received { term: 4 index: 34 } } ignore_live_leader: false dest_uuid: "6315fb5099e1420b95b8d83b3e19375c"
I20260926 15:02:41.366201 9813 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 4 FOLLOWER]: Advancing to term 5
W20260926 15:02:41.368021 9145 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
W20260926 15:02:41.369321 9145 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:02:41.369477 9813 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 320cd09624624312bf1bcc0856c2874f in term 5.
I20260926 15:02:41.370352 9142 leader_election.cc:302] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f, 6315fb5099e1420b95b8d83b3e19375c; no voters: be9958f2085242f7ab82ae933ceb1c42
W20260926 15:02:41.370585 9791 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:02:41.370882 9961 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 5 FOLLOWER]: Leader election won for term 5
I20260926 15:02:41.371460 9961 raft_consensus.cc:686] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 5 LEADER]: Becoming Leader. State: Replica: 320cd09624624312bf1bcc0856c2874f, State: Running, Role: LEADER
I20260926 15:02:41.371809 9961 consensus_queue.cc:237] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 4.34, Last appended by leader: 34, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
W20260926 15:02:41.372502 9791 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:02:41.372965 9969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 320cd09624624312bf1bcc0856c2874f. Latest consensus state: current_term: 5 leader_uuid: "320cd09624624312bf1bcc0856c2874f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:41.373378 9969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: This master's current role is: LEADER
I20260926 15:02:41.374104 9970 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:02:41.377005 9970 catalog_manager.cc:669] Loaded metadata for table default.table_43176f08d9e24974abdd53171465a15c [id=08b331ce5f9c44389fae209bac801f42]
I20260926 15:02:41.377460 9970 catalog_manager.cc:669] Loaded metadata for table default.table_854219551f9048c6925c711ce33d5256 [id=3295fc8a0a004843be045b8bf6a4f2cf]
I20260926 15:02:41.377830 9970 catalog_manager.cc:669] Loaded metadata for table default.table_96be20374f7e43e7a865549e39bd64f6 [id=51dea274f4bf4903baaddbbd8c257b27]
I20260926 15:02:41.378170 9970 catalog_manager.cc:669] Loaded metadata for table default.table_d2243274908043e58210a197141af813 [id=5357ddcf21634f99bb4f9abc5b6c4d80]
I20260926 15:02:41.378521 9970 catalog_manager.cc:669] Loaded metadata for table default.table_95f48725b6ce47fab5f96e18ccb69dbc [id=630deebc15b04878ae13844ff2808f52]
I20260926 15:02:41.378901 9970 catalog_manager.cc:669] Loaded metadata for table default.table_22fd4957e3d14964803428457f8490ac [id=7ba7b39b84394421b88a6ce7616c3c91]
I20260926 15:02:41.379207 9970 catalog_manager.cc:669] Loaded metadata for table default.table_41a3ce3c89684cbb9054c4251a0cc51c [id=7cc0c844d49c4ff08cef9e2c410991f8]
I20260926 15:02:41.379530 9970 catalog_manager.cc:669] Loaded metadata for table default.table_7b56039466d14dcab4dc63192de3b08e [id=b84ce6d15e4246dbbb299468138add55]
I20260926 15:02:41.379837 9970 catalog_manager.cc:669] Loaded metadata for table default.table_e3b713aac3c94e449bb5882310d55a40 [id=e9e378126d2649dab722c82f99d654b7]
I20260926 15:02:41.381506 9970 tablet_loader.cc:96] loaded metadata for tablet 0d837276d648440cbebe1f88c281cb11 (table default.table_96be20374f7e43e7a865549e39bd64f6 [id=51dea274f4bf4903baaddbbd8c257b27])
I20260926 15:02:41.382032 9970 tablet_loader.cc:96] loaded metadata for tablet 317305a7fdb54f0bb2bb569272550b7b (table default.table_7b56039466d14dcab4dc63192de3b08e [id=b84ce6d15e4246dbbb299468138add55])
I20260926 15:02:41.382329 9970 tablet_loader.cc:96] loaded metadata for tablet 3fae8a76002849de9bb26610a0c33406 (table default.table_95f48725b6ce47fab5f96e18ccb69dbc [id=630deebc15b04878ae13844ff2808f52])
I20260926 15:02:41.382632 9970 tablet_loader.cc:96] loaded metadata for tablet 41e641e28d1f48b2a7e0c8fb8964f678 (table default.table_d2243274908043e58210a197141af813 [id=5357ddcf21634f99bb4f9abc5b6c4d80])
I20260926 15:02:41.382920 9970 tablet_loader.cc:96] loaded metadata for tablet 4bec0ca69b0e477281a232c1bf1121fb (table default.table_41a3ce3c89684cbb9054c4251a0cc51c [id=7cc0c844d49c4ff08cef9e2c410991f8])
I20260926 15:02:41.383203 9970 tablet_loader.cc:96] loaded metadata for tablet 51ff1e9e47864b24b7c7a869ff37e06a (table default.table_854219551f9048c6925c711ce33d5256 [id=3295fc8a0a004843be045b8bf6a4f2cf])
I20260926 15:02:41.383607 9970 tablet_loader.cc:96] loaded metadata for tablet 9134d14fae4d46b6bd8ef1c05dc5b1c2 (table default.table_e3b713aac3c94e449bb5882310d55a40 [id=e9e378126d2649dab722c82f99d654b7])
I20260926 15:02:41.384114 9970 tablet_loader.cc:96] loaded metadata for tablet c5d07d3a7a164d098e61f8f7f2ac4a67 (table default.table_43176f08d9e24974abdd53171465a15c [id=08b331ce5f9c44389fae209bac801f42])
I20260926 15:02:41.384410 9970 tablet_loader.cc:96] loaded metadata for tablet c8d090f2e0384924a72f288eff2281e7 (table default.table_22fd4957e3d14964803428457f8490ac [id=7ba7b39b84394421b88a6ce7616c3c91])
I20260926 15:02:41.384651 9970 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:02:41.384689 9168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6315fb5099e1420b95b8d83b3e19375c" candidate_term: 5 candidate_status { last_received { term: 4 index: 34 } } ignore_live_leader: false dest_uuid: "320cd09624624312bf1bcc0856c2874f" is_pre_election: true
I20260926 15:02:41.385660 9790 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:02:41.386034 9970 catalog_manager.cc:1259] Loaded cluster ID: 497ed160bc39477e99bf3beb637a4391
I20260926 15:02:41.386106 9970 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:02:41.387120 9962 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260926 15:02:41.388129 9970 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:02:41.389032 9970 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: Loaded TSK: 0
I20260926 15:02:41.389648 9970 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:02:41.390501 9970 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260926 15:02:41.391259 9970 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260926 15:02:41.410073 9158 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51204:
name: "default.Table_6c170e0fd90e42f08500c2a89216a4d3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:41.411689 9152 master_service.cc:946] ReplaceTablet: received request to replace tablet 4bec0ca69b0e477281a232c1bf1121fb from {username='slave'} at 127.0.0.1:51204
I20260926 15:02:41.410305 9156 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51204:
name: "default.Table_0e42d4f90b4e4529bb0a2ea0909cbadb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:41.414088 9150 master_service.cc:946] ReplaceTablet: received request to replace tablet 317305a7fdb54f0bb2bb569272550b7b from {username='slave'} at 127.0.0.1:51204
I20260926 15:02:41.413938 9153 master_service.cc:946] ReplaceTablet: received request to replace tablet 4bec0ca69b0e477281a232c1bf1121fb from {username='slave'} at 127.0.0.1:51204
I20260926 15:02:41.410487 9157 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51204:
name: "default.Table_5bfdd1b3c4a247d58d08b75039b6124f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:41.414243 9155 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51204:
table { table_name: "default.Table_854219551f9048c6925c711ce33d5256" } modify_external_catalogs: true
I20260926 15:02:41.411099 9154 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51204:
table { table_name: "default.Table_22fd4957e3d14964803428457f8490ac" } modify_external_catalogs: true
I20260926 15:02:41.416996 9155 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51204:
table { table_name: "default.Table_854219551f9048c6925c711ce33d5256" } modify_external_catalogs: true
I20260926 15:02:41.417061 9154 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51204:
table { table_name: "default.Table_22fd4957e3d14964803428457f8490ac" } modify_external_catalogs: true
I20260926 15:02:41.414214 9151 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51204:
table { table_name: "default.Table_96be20374f7e43e7a865549e39bd64f6" } modify_external_catalogs: true
I20260926 15:02:41.417331 9151 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51204:
table { table_name: "default.Table_96be20374f7e43e7a865549e39bd64f6" } modify_external_catalogs: true
I20260926 15:02:41.413898 9149 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51204:
name: "default.Table_aa60eda53a3f4f338866d309cbde44a9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260926 15:02:41.418845 9213 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_f356bc0dfc6a47f98430002dccbb873f
*** Check failure stack trace: ***
@ 0x7fec3bcf39ad google::LogMessage::Fail() at ??:0
I20260926 15:02:41.422328 9813 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 5 FOLLOWER]: Refusing update from remote peer 320cd09624624312bf1bcc0856c2874f: Log matching property violated. Preceding OpId in replica: term: 4 index: 34. Preceding OpId from leader: term: 5 index: 36. (index mismatch)
I20260926 15:02:41.423066 9969 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
W20260926 15:02:41.423314 9145 consensus_peers.cc:487] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f -> Peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Couldn't send request to peer be9958f2085242f7ab82ae933ceb1c42. Status: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
@ 0x7fec3bcf688a google::LogMessage::SendToLog() at ??:0
@ 0x7fec3bcf34de google::LogMessage::Flush() at ??:0
I20260926 15:02:41.426527 9962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 320cd09624624312bf1bcc0856c2874f. Latest consensus state: current_term: 5 leader_uuid: "320cd09624624312bf1bcc0856c2874f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:41.426787 9962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:02:41.427459 9971 mvcc.cc:205] Tried to move back new op lower bound from 7333621601977913344 to 7333621601781800960. Current Snapshot: MvccSnapshot[applied={T|T < 7333621601977913344}]
@ 0x7fec3bcf36d1 google::LogMessage::~LogMessage() at ??:0
I20260926 15:02:41.429152 9150 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: Sending DeleteTablet for 3 replicas of tablet 317305a7fdb54f0bb2bb569272550b7b
I20260926 15:02:41.431689 9150 catalog_manager.cc:6565] ReplaceTablet: tablet 317305a7fdb54f0bb2bb569272550b7b deleted and replaced by tablet d4c76c4e1678474c92af007a7b5f7596
I20260926 15:02:41.432189 9150 master_service.cc:946] ReplaceTablet: received request to replace tablet 317305a7fdb54f0bb2bb569272550b7b from {username='slave'} at 127.0.0.1:51204
I20260926 15:02:41.444613 9152 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: Sending DeleteTablet for 3 replicas of tablet 4bec0ca69b0e477281a232c1bf1121fb
I20260926 15:02:41.445174 9152 catalog_manager.cc:6565] ReplaceTablet: tablet 4bec0ca69b0e477281a232c1bf1121fb deleted and replaced by tablet 7fa51439f5324161991e8dbbd169851e
I20260926 15:02:41.445446 9152 master_service.cc:946] ReplaceTablet: received request to replace tablet 9134d14fae4d46b6bd8ef1c05dc5b1c2 from {username='slave'} at 127.0.0.1:51204
I20260926 15:02:41.445339 9962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "320cd09624624312bf1bcc0856c2874f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:41.445586 9962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:02:41.460218 9153 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51204:
name: "default.Table_7257407810154f2aa325ed506a8a62d5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:41.462770 9610 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bec0ca69b0e477281a232c1bf1121fb with delete_type TABLET_DATA_DELETED (replaced by tablet 7fa51439f5324161991e8dbbd169851e) from {username='slave'} at 127.0.0.1:48320
I20260926 15:02:41.462714 9611 tablet_service.cc:1514] Processing DeleteTablet for tablet 317305a7fdb54f0bb2bb569272550b7b with delete_type TABLET_DATA_DELETED (replaced by tablet d4c76c4e1678474c92af007a7b5f7596) from {username='slave'} at 127.0.0.1:48320
I20260926 15:02:41.463333 9986 tablet_replica.cc:330] T 4bec0ca69b0e477281a232c1bf1121fb P f052d5cbee65406593d8e44f74225bc2: stopping tablet replica
I20260926 15:02:41.463547 9986 raft_consensus.cc:2229] T 4bec0ca69b0e477281a232c1bf1121fb P f052d5cbee65406593d8e44f74225bc2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:02:41.463768 9986 raft_consensus.cc:2258] T 4bec0ca69b0e477281a232c1bf1121fb P f052d5cbee65406593d8e44f74225bc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:41.464300 9986 ts_tablet_manager.cc:1905] T 4bec0ca69b0e477281a232c1bf1121fb P f052d5cbee65406593d8e44f74225bc2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:41.461529 9961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 5 leader_uuid: "320cd09624624312bf1bcc0856c2874f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:41.468899 9325 tablet_service.cc:1514] Processing DeleteTablet for tablet 317305a7fdb54f0bb2bb569272550b7b with delete_type TABLET_DATA_DELETED (replaced by tablet d4c76c4e1678474c92af007a7b5f7596) from {username='slave'} at 127.0.0.1:33164
I20260926 15:02:41.469527 9324 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bec0ca69b0e477281a232c1bf1121fb with delete_type TABLET_DATA_DELETED (replaced by tablet 7fa51439f5324161991e8dbbd169851e) from {username='slave'} at 127.0.0.1:33164
I20260926 15:02:41.469519 9988 tablet_replica.cc:330] T 317305a7fdb54f0bb2bb569272550b7b P 66ca148489d64deabd1dcbf7870c6d57: stopping tablet replica
I20260926 15:02:41.469931 9988 raft_consensus.cc:2229] T 317305a7fdb54f0bb2bb569272550b7b P 66ca148489d64deabd1dcbf7870c6d57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:02:41.470252 9988 raft_consensus.cc:2258] T 317305a7fdb54f0bb2bb569272550b7b P 66ca148489d64deabd1dcbf7870c6d57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:41.470417 9986 ts_tablet_manager.cc:1918] T 4bec0ca69b0e477281a232c1bf1121fb P f052d5cbee65406593d8e44f74225bc2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 15:02:41.470566 9986 log.cc:1196] T 4bec0ca69b0e477281a232c1bf1121fb P f052d5cbee65406593d8e44f74225bc2: Deleting WAL directory at /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-2/wal/wals/4bec0ca69b0e477281a232c1bf1121fb
I20260926 15:02:41.470945 9986 ts_tablet_manager.cc:1939] T 4bec0ca69b0e477281a232c1bf1121fb P f052d5cbee65406593d8e44f74225bc2: Deleting consensus metadata
I20260926 15:02:41.471094 9988 ts_tablet_manager.cc:1905] T 317305a7fdb54f0bb2bb569272550b7b P 66ca148489d64deabd1dcbf7870c6d57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:41.471810 9986 tablet_replica.cc:330] T 317305a7fdb54f0bb2bb569272550b7b P f052d5cbee65406593d8e44f74225bc2: stopping tablet replica
I20260926 15:02:41.472127 9986 raft_consensus.cc:2229] T 317305a7fdb54f0bb2bb569272550b7b P f052d5cbee65406593d8e44f74225bc2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:02:41.471997 9961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: This master's current role is: LEADER
I20260926 15:02:41.472043 9142 catalog_manager.cc:4841] TS f052d5cbee65406593d8e44f74225bc2 (127.8.138.131:35445): tablet 4bec0ca69b0e477281a232c1bf1121fb (table default.table_41a3ce3c89684cbb9054c4251a0cc51c [id=7cc0c844d49c4ff08cef9e2c410991f8]) successfully deleted
I20260926 15:02:41.472397 9986 raft_consensus.cc:2258] T 317305a7fdb54f0bb2bb569272550b7b P f052d5cbee65406593d8e44f74225bc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:41.472891 9986 ts_tablet_manager.cc:1905] T 317305a7fdb54f0bb2bb569272550b7b P f052d5cbee65406593d8e44f74225bc2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:41.477666 9986 ts_tablet_manager.cc:1918] T 317305a7fdb54f0bb2bb569272550b7b P f052d5cbee65406593d8e44f74225bc2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 15:02:41.477831 9986 log.cc:1196] T 317305a7fdb54f0bb2bb569272550b7b P f052d5cbee65406593d8e44f74225bc2: Deleting WAL directory at /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-2/wal/wals/317305a7fdb54f0bb2bb569272550b7b
I20260926 15:02:41.478160 9986 ts_tablet_manager.cc:1939] T 317305a7fdb54f0bb2bb569272550b7b P f052d5cbee65406593d8e44f74225bc2: Deleting consensus metadata
I20260926 15:02:41.480118 9150 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51204:
name: "default.Table_04fd049074c940dba0495d97eac1d272"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:41.480810 9142 catalog_manager.cc:4841] TS f052d5cbee65406593d8e44f74225bc2 (127.8.138.131:35445): tablet 317305a7fdb54f0bb2bb569272550b7b (table default.table_7b56039466d14dcab4dc63192de3b08e [id=b84ce6d15e4246dbbb299468138add55]) successfully deleted
I20260926 15:02:41.481261 9988 ts_tablet_manager.cc:1918] T 317305a7fdb54f0bb2bb569272550b7b P 66ca148489d64deabd1dcbf7870c6d57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 15:02:41.481432 9988 log.cc:1196] T 317305a7fdb54f0bb2bb569272550b7b P 66ca148489d64deabd1dcbf7870c6d57: Deleting WAL directory at /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-0/wal/wals/317305a7fdb54f0bb2bb569272550b7b
I20260926 15:02:41.481891 9988 ts_tablet_manager.cc:1939] T 317305a7fdb54f0bb2bb569272550b7b P 66ca148489d64deabd1dcbf7870c6d57: Deleting consensus metadata
I20260926 15:02:41.482992 9988 tablet_replica.cc:330] T 4bec0ca69b0e477281a232c1bf1121fb P 66ca148489d64deabd1dcbf7870c6d57: stopping tablet replica
I20260926 15:02:41.483249 9143 catalog_manager.cc:4841] TS 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327): tablet 317305a7fdb54f0bb2bb569272550b7b (table default.table_7b56039466d14dcab4dc63192de3b08e [id=b84ce6d15e4246dbbb299468138add55]) successfully deleted
I20260926 15:02:41.483593 9988 raft_consensus.cc:2229] T 4bec0ca69b0e477281a232c1bf1121fb P 66ca148489d64deabd1dcbf7870c6d57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:02:41.483984 9988 raft_consensus.cc:2258] T 4bec0ca69b0e477281a232c1bf1121fb P 66ca148489d64deabd1dcbf7870c6d57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:41.484606 9988 ts_tablet_manager.cc:1905] T 4bec0ca69b0e477281a232c1bf1121fb P 66ca148489d64deabd1dcbf7870c6d57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:41.486403 9610 tablet_service.cc:1467] Processing CreateTablet for tablet 7fa51439f5324161991e8dbbd169851e (DEFAULT_TABLE table=default.table_41a3ce3c89684cbb9054c4251a0cc51c [id=7cc0c844d49c4ff08cef9e2c410991f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:41.486680 9324 tablet_service.cc:1467] Processing CreateTablet for tablet 7fa51439f5324161991e8dbbd169851e (DEFAULT_TABLE table=default.table_41a3ce3c89684cbb9054c4251a0cc51c [id=7cc0c844d49c4ff08cef9e2c410991f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:41.486867 9610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fa51439f5324161991e8dbbd169851e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:41.487103 9324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fa51439f5324161991e8dbbd169851e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:41.488262 9325 tablet_service.cc:1467] Processing CreateTablet for tablet d4c76c4e1678474c92af007a7b5f7596 (DEFAULT_TABLE table=default.table_7b56039466d14dcab4dc63192de3b08e [id=b84ce6d15e4246dbbb299468138add55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:41.488379 9611 tablet_service.cc:1467] Processing CreateTablet for tablet d4c76c4e1678474c92af007a7b5f7596 (DEFAULT_TABLE table=default.table_7b56039466d14dcab4dc63192de3b08e [id=b84ce6d15e4246dbbb299468138add55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:41.488656 9325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4c76c4e1678474c92af007a7b5f7596. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:41.488786 9611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4c76c4e1678474c92af007a7b5f7596. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:41.495607 9990 tablet_bootstrap.cc:492] T 7fa51439f5324161991e8dbbd169851e P 66ca148489d64deabd1dcbf7870c6d57: Bootstrap starting.
I20260926 15:02:41.495604 9465 tablet_service.cc:1514] Processing DeleteTablet for tablet 317305a7fdb54f0bb2bb569272550b7b with delete_type TABLET_DATA_DELETED (replaced by tablet d4c76c4e1678474c92af007a7b5f7596) from {username='slave'} at 127.0.0.1:50634
I20260926 15:02:41.495990 9466 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bec0ca69b0e477281a232c1bf1121fb with delete_type TABLET_DATA_DELETED (replaced by tablet 7fa51439f5324161991e8dbbd169851e) from {username='slave'} at 127.0.0.1:50634
I20260926 15:02:41.496428 9467 tablet_service.cc:1467] Processing CreateTablet for tablet 7fa51439f5324161991e8dbbd169851e (DEFAULT_TABLE table=default.table_41a3ce3c89684cbb9054c4251a0cc51c [id=7cc0c844d49c4ff08cef9e2c410991f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:41.496848 9467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fa51439f5324161991e8dbbd169851e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:41.497846 9990 tablet_bootstrap.cc:654] T 7fa51439f5324161991e8dbbd169851e P 66ca148489d64deabd1dcbf7870c6d57: Neither blocks nor log segments found. Creating new log.
@ 0x7fec51891ba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260926 15:02:41.500300 9464 tablet_service.cc:1467] Processing CreateTablet for tablet d4c76c4e1678474c92af007a7b5f7596 (DEFAULT_TABLE table=default.table_7b56039466d14dcab4dc63192de3b08e [id=b84ce6d15e4246dbbb299468138add55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:41.500797 9464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4c76c4e1678474c92af007a7b5f7596. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:41.502610 9989 tablet_bootstrap.cc:492] T 7fa51439f5324161991e8dbbd169851e P f052d5cbee65406593d8e44f74225bc2: Bootstrap starting.
I20260926 15:02:41.504938 9989 tablet_bootstrap.cc:654] T 7fa51439f5324161991e8dbbd169851e P f052d5cbee65406593d8e44f74225bc2: Neither blocks nor log segments found. Creating new log.
@ 0x7fec51890901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260926 15:02:41.508302 9990 tablet_bootstrap.cc:492] T 7fa51439f5324161991e8dbbd169851e P 66ca148489d64deabd1dcbf7870c6d57: No bootstrap required, opened a new log
I20260926 15:02:41.508455 9990 ts_tablet_manager.cc:1397] T 7fa51439f5324161991e8dbbd169851e P 66ca148489d64deabd1dcbf7870c6d57: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.000s
I20260926 15:02:41.508611 9988 ts_tablet_manager.cc:1918] T 4bec0ca69b0e477281a232c1bf1121fb P 66ca148489d64deabd1dcbf7870c6d57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 15:02:41.508803 9988 log.cc:1196] T 4bec0ca69b0e477281a232c1bf1121fb P 66ca148489d64deabd1dcbf7870c6d57: Deleting WAL directory at /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-0/wal/wals/4bec0ca69b0e477281a232c1bf1121fb
I20260926 15:02:41.509233 9988 ts_tablet_manager.cc:1939] T 4bec0ca69b0e477281a232c1bf1121fb P 66ca148489d64deabd1dcbf7870c6d57: Deleting consensus metadata
@ 0x7fec5189766c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260926 15:02:41.509413 9990 raft_consensus.cc:348] T 7fa51439f5324161991e8dbbd169851e P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } }
I20260926 15:02:41.509682 9990 raft_consensus.cc:374] T 7fa51439f5324161991e8dbbd169851e P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:41.509773 9990 raft_consensus.cc:729] T 7fa51439f5324161991e8dbbd169851e P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ca148489d64deabd1dcbf7870c6d57, State: Initialized, Role: FOLLOWER
I20260926 15:02:41.510504 9143 catalog_manager.cc:4841] TS 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327): tablet 4bec0ca69b0e477281a232c1bf1121fb (table default.table_41a3ce3c89684cbb9054c4251a0cc51c [id=7cc0c844d49c4ff08cef9e2c410991f8]) successfully deleted
I20260926 15:02:41.510044 9990 consensus_queue.cc:260] T 7fa51439f5324161991e8dbbd169851e P 66ca148489d64deabd1dcbf7870c6d57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } }
I20260926 15:02:41.512662 9989 tablet_bootstrap.cc:492] T 7fa51439f5324161991e8dbbd169851e P f052d5cbee65406593d8e44f74225bc2: No bootstrap required, opened a new log
I20260926 15:02:41.512812 9989 ts_tablet_manager.cc:1397] T 7fa51439f5324161991e8dbbd169851e P f052d5cbee65406593d8e44f74225bc2: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260926 15:02:41.513752 9989 raft_consensus.cc:348] T 7fa51439f5324161991e8dbbd169851e P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } }
@ 0x7fec518974e9 std::_Function_handler<>::_M_invoke() at ??:0
I20260926 15:02:41.514074 9989 raft_consensus.cc:374] T 7fa51439f5324161991e8dbbd169851e P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:41.514171 9989 raft_consensus.cc:729] T 7fa51439f5324161991e8dbbd169851e P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f052d5cbee65406593d8e44f74225bc2, State: Initialized, Role: FOLLOWER
I20260926 15:02:41.514626 9990 ts_tablet_manager.cc:1428] T 7fa51439f5324161991e8dbbd169851e P 66ca148489d64deabd1dcbf7870c6d57: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260926 15:02:41.514484 9989 consensus_queue.cc:260] T 7fa51439f5324161991e8dbbd169851e P f052d5cbee65406593d8e44f74225bc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } }
I20260926 15:02:41.515918 9989 ts_tablet_manager.cc:1428] T 7fa51439f5324161991e8dbbd169851e P f052d5cbee65406593d8e44f74225bc2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 15:02:41.516212 9989 tablet_bootstrap.cc:492] T d4c76c4e1678474c92af007a7b5f7596 P f052d5cbee65406593d8e44f74225bc2: Bootstrap starting.
I20260926 15:02:41.517853 9989 tablet_bootstrap.cc:654] T d4c76c4e1678474c92af007a7b5f7596 P f052d5cbee65406593d8e44f74225bc2: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:41.520143 9990 tablet_bootstrap.cc:492] T d4c76c4e1678474c92af007a7b5f7596 P 66ca148489d64deabd1dcbf7870c6d57: Bootstrap starting.
I20260926 15:02:41.522815 9990 tablet_bootstrap.cc:654] T d4c76c4e1678474c92af007a7b5f7596 P 66ca148489d64deabd1dcbf7870c6d57: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:41.524560 9152 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: Sending DeleteTablet for 3 replicas of tablet 9134d14fae4d46b6bd8ef1c05dc5b1c2
I20260926 15:02:41.525094 9152 catalog_manager.cc:6565] ReplaceTablet: tablet 9134d14fae4d46b6bd8ef1c05dc5b1c2 deleted and replaced by tablet 252e506a97564739bc26268653efa864
@ 0x9edb7d std::function<>::operator()() at ??:?
I20260926 15:02:41.525348 9324 tablet_service.cc:1514] Processing DeleteTablet for tablet 9134d14fae4d46b6bd8ef1c05dc5b1c2 with delete_type TABLET_DATA_DELETED (replaced by tablet 252e506a97564739bc26268653efa864) from {username='slave'} at 127.0.0.1:33164
I20260926 15:02:41.526665 9988 tablet_replica.cc:330] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P 66ca148489d64deabd1dcbf7870c6d57: stopping tablet replica
I20260926 15:02:41.526909 9988 raft_consensus.cc:2229] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P 66ca148489d64deabd1dcbf7870c6d57 [term 1 LEADER]: Raft consensus shutting down.
I20260926 15:02:41.527391 9988 pending_rounds.cc:62] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P 66ca148489d64deabd1dcbf7870c6d57: Trying to abort 1 pending ops.
I20260926 15:02:41.527587 9988 pending_rounds.cc:68] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P 66ca148489d64deabd1dcbf7870c6d57: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333621599941574656 op_type: NO_OP noop_request { }
I20260926 15:02:41.527823 9988 raft_consensus.cc:2866] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P 66ca148489d64deabd1dcbf7870c6d57 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 15:02:41.528012 9988 raft_consensus.cc:2258] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P 66ca148489d64deabd1dcbf7870c6d57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:41.529385 9610 tablet_service.cc:1514] Processing DeleteTablet for tablet 9134d14fae4d46b6bd8ef1c05dc5b1c2 with delete_type TABLET_DATA_DELETED (replaced by tablet 252e506a97564739bc26268653efa864) from {username='slave'} at 127.0.0.1:48320
I20260926 15:02:41.529798 9988 ts_tablet_manager.cc:1905] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P 66ca148489d64deabd1dcbf7870c6d57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:41.530027 9986 tablet_replica.cc:330] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P f052d5cbee65406593d8e44f74225bc2: stopping tablet replica
I20260926 15:02:41.530119 9464 tablet_service.cc:1514] Processing DeleteTablet for tablet 9134d14fae4d46b6bd8ef1c05dc5b1c2 with delete_type TABLET_DATA_DELETED (replaced by tablet 252e506a97564739bc26268653efa864) from {username='slave'} at 127.0.0.1:50634
I20260926 15:02:41.530229 9986 raft_consensus.cc:2229] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P f052d5cbee65406593d8e44f74225bc2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:02:41.530416 9986 raft_consensus.cc:2258] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P f052d5cbee65406593d8e44f74225bc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:41.530948 9986 ts_tablet_manager.cc:1905] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P f052d5cbee65406593d8e44f74225bc2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:41.532972 9995 tablet_bootstrap.cc:492] T 7fa51439f5324161991e8dbbd169851e P ae330d635b1848aaaecee2c7ce96520a: Bootstrap starting.
I20260926 15:02:41.535233 9995 tablet_bootstrap.cc:654] T 7fa51439f5324161991e8dbbd169851e P ae330d635b1848aaaecee2c7ce96520a: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:41.536262 9988 ts_tablet_manager.cc:1918] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P 66ca148489d64deabd1dcbf7870c6d57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 15:02:41.536415 9988 log.cc:1196] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P 66ca148489d64deabd1dcbf7870c6d57: Deleting WAL directory at /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-0/wal/wals/9134d14fae4d46b6bd8ef1c05dc5b1c2
I20260926 15:02:41.536816 9988 ts_tablet_manager.cc:1939] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P 66ca148489d64deabd1dcbf7870c6d57: Deleting consensus metadata
I20260926 15:02:41.537856 9324 tablet_service.cc:1467] Processing CreateTablet for tablet 252e506a97564739bc26268653efa864 (DEFAULT_TABLE table=default.table_e3b713aac3c94e449bb5882310d55a40 [id=e9e378126d2649dab722c82f99d654b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:41.538777 9610 tablet_service.cc:1467] Processing CreateTablet for tablet 252e506a97564739bc26268653efa864 (DEFAULT_TABLE table=default.table_e3b713aac3c94e449bb5882310d55a40 [id=e9e378126d2649dab722c82f99d654b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:41.539216 9610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 252e506a97564739bc26268653efa864. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:41.542254 9677 heartbeater.cc:503] Master 127.8.138.188:45473 was elected leader, sending a full tablet report...
I20260926 15:02:41.543661 9986 ts_tablet_manager.cc:1918] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P f052d5cbee65406593d8e44f74225bc2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 15:02:41.543790 9986 log.cc:1196] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P f052d5cbee65406593d8e44f74225bc2: Deleting WAL directory at /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-2/wal/wals/9134d14fae4d46b6bd8ef1c05dc5b1c2
I20260926 15:02:41.544459 9986 ts_tablet_manager.cc:1939] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P f052d5cbee65406593d8e44f74225bc2: Deleting consensus metadata
I20260926 15:02:41.544500 9390 heartbeater.cc:503] Master 127.8.138.188:45473 was elected leader, sending a full tablet report...
@ 0x7fec3e91deaf kudu::Thread::SuperviseThread() at ??:0
I20260926 15:02:41.545449 9142 catalog_manager.cc:4841] TS f052d5cbee65406593d8e44f74225bc2 (127.8.138.131:35445): tablet 9134d14fae4d46b6bd8ef1c05dc5b1c2 (table default.table_e3b713aac3c94e449bb5882310d55a40 [id=e9e378126d2649dab722c82f99d654b7]) successfully deleted
I20260926 15:02:41.547748 9324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 252e506a97564739bc26268653efa864. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7fec41bf76db start_thread at ??:0
I20260926 15:02:41.547919 9152 catalog_manager.cc:5495] T 41e641e28d1f48b2a7e0c8fb8964f678 P f052d5cbee65406593d8e44f74225bc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to f052d5cbee65406593d8e44f74225bc2 (127.8.138.131). New cstate: current_term: 1 leader_uuid: "f052d5cbee65406593d8e44f74225bc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } health_report { overall_health: HEALTHY } } }
@ 0x7fec3ab8d61f clone at ??:0
I20260926 15:02:41.550959 9464 tablet_service.cc:1467] Processing CreateTablet for tablet 252e506a97564739bc26268653efa864 (DEFAULT_TABLE table=default.table_e3b713aac3c94e449bb5882310d55a40 [id=e9e378126d2649dab722c82f99d654b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:41.551465 9464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 252e506a97564739bc26268653efa864. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:41.556244 9989 tablet_bootstrap.cc:492] T d4c76c4e1678474c92af007a7b5f7596 P f052d5cbee65406593d8e44f74225bc2: No bootstrap required, opened a new log
I20260926 15:02:41.556373 9989 ts_tablet_manager.cc:1397] T d4c76c4e1678474c92af007a7b5f7596 P f052d5cbee65406593d8e44f74225bc2: Time spent bootstrapping tablet: real 0.040s user 0.000s sys 0.004s
I20260926 15:02:41.556525 9993 tablet_replica.cc:330] T 317305a7fdb54f0bb2bb569272550b7b P ae330d635b1848aaaecee2c7ce96520a: stopping tablet replica
I20260926 15:02:41.556799 9993 raft_consensus.cc:2229] T 317305a7fdb54f0bb2bb569272550b7b P ae330d635b1848aaaecee2c7ce96520a [term 1 LEADER]: Raft consensus shutting down.
I20260926 15:02:41.556914 9989 raft_consensus.cc:348] T d4c76c4e1678474c92af007a7b5f7596 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:41.557090 9989 raft_consensus.cc:374] T d4c76c4e1678474c92af007a7b5f7596 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:41.557111 9993 pending_rounds.cc:62] T 317305a7fdb54f0bb2bb569272550b7b P ae330d635b1848aaaecee2c7ce96520a: Trying to abort 1 pending ops.
I20260926 15:02:41.557144 9989 raft_consensus.cc:729] T d4c76c4e1678474c92af007a7b5f7596 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f052d5cbee65406593d8e44f74225bc2, State: Initialized, Role: FOLLOWER
I20260926 15:02:41.557170 9993 pending_rounds.cc:68] T 317305a7fdb54f0bb2bb569272550b7b P ae330d635b1848aaaecee2c7ce96520a: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333621599480877056 op_type: NO_OP noop_request { }
I20260926 15:02:41.557348 9993 raft_consensus.cc:2866] T 317305a7fdb54f0bb2bb569272550b7b P ae330d635b1848aaaecee2c7ce96520a [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 15:02:41.557442 9993 raft_consensus.cc:2258] T 317305a7fdb54f0bb2bb569272550b7b P ae330d635b1848aaaecee2c7ce96520a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:41.557371 9989 consensus_queue.cc:260] T d4c76c4e1678474c92af007a7b5f7596 P f052d5cbee65406593d8e44f74225bc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:41.557987 9989 ts_tablet_manager.cc:1428] T d4c76c4e1678474c92af007a7b5f7596 P f052d5cbee65406593d8e44f74225bc2: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260926 15:02:41.559268 9989 tablet_bootstrap.cc:492] T 252e506a97564739bc26268653efa864 P f052d5cbee65406593d8e44f74225bc2: Bootstrap starting.
I20260926 15:02:41.561375 9989 tablet_bootstrap.cc:654] T 252e506a97564739bc26268653efa864 P f052d5cbee65406593d8e44f74225bc2: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:41.562849 9993 ts_tablet_manager.cc:1905] T 317305a7fdb54f0bb2bb569272550b7b P ae330d635b1848aaaecee2c7ce96520a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:41.571528 9989 tablet_bootstrap.cc:492] T 252e506a97564739bc26268653efa864 P f052d5cbee65406593d8e44f74225bc2: No bootstrap required, opened a new log
I20260926 15:02:41.571724 9989 ts_tablet_manager.cc:1397] T 252e506a97564739bc26268653efa864 P f052d5cbee65406593d8e44f74225bc2: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.003s
I20260926 15:02:41.572784 9989 raft_consensus.cc:348] T 252e506a97564739bc26268653efa864 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:41.573071 9989 raft_consensus.cc:374] T 252e506a97564739bc26268653efa864 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:41.573170 9989 raft_consensus.cc:729] T 252e506a97564739bc26268653efa864 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f052d5cbee65406593d8e44f74225bc2, State: Initialized, Role: FOLLOWER
I20260926 15:02:41.573499 9989 consensus_queue.cc:260] T 252e506a97564739bc26268653efa864 P f052d5cbee65406593d8e44f74225bc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:41.574424 9989 ts_tablet_manager.cc:1428] T 252e506a97564739bc26268653efa864 P f052d5cbee65406593d8e44f74225bc2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 15:02:41.577605 9990 tablet_bootstrap.cc:492] T d4c76c4e1678474c92af007a7b5f7596 P 66ca148489d64deabd1dcbf7870c6d57: No bootstrap required, opened a new log
I20260926 15:02:41.577769 9990 ts_tablet_manager.cc:1397] T d4c76c4e1678474c92af007a7b5f7596 P 66ca148489d64deabd1dcbf7870c6d57: Time spent bootstrapping tablet: real 0.058s user 0.012s sys 0.015s
I20260926 15:02:41.578285 9995 tablet_bootstrap.cc:492] T 7fa51439f5324161991e8dbbd169851e P ae330d635b1848aaaecee2c7ce96520a: No bootstrap required, opened a new log
I20260926 15:02:41.578449 9995 ts_tablet_manager.cc:1397] T 7fa51439f5324161991e8dbbd169851e P ae330d635b1848aaaecee2c7ce96520a: Time spent bootstrapping tablet: real 0.046s user 0.004s sys 0.000s
I20260926 15:02:41.578573 9990 raft_consensus.cc:348] T d4c76c4e1678474c92af007a7b5f7596 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:41.578810 9990 raft_consensus.cc:374] T d4c76c4e1678474c92af007a7b5f7596 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:41.578903 9990 raft_consensus.cc:729] T d4c76c4e1678474c92af007a7b5f7596 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ca148489d64deabd1dcbf7870c6d57, State: Initialized, Role: FOLLOWER
I20260926 15:02:41.579150 9995 raft_consensus.cc:348] T 7fa51439f5324161991e8dbbd169851e P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } }
I20260926 15:02:41.579176 9990 consensus_queue.cc:260] T d4c76c4e1678474c92af007a7b5f7596 P 66ca148489d64deabd1dcbf7870c6d57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:41.579361 9995 raft_consensus.cc:374] T 7fa51439f5324161991e8dbbd169851e P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:41.579450 9995 raft_consensus.cc:729] T 7fa51439f5324161991e8dbbd169851e P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae330d635b1848aaaecee2c7ce96520a, State: Initialized, Role: FOLLOWER
I20260926 15:02:41.580022 9990 ts_tablet_manager.cc:1428] T d4c76c4e1678474c92af007a7b5f7596 P 66ca148489d64deabd1dcbf7870c6d57: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 15:02:41.580327 9990 tablet_bootstrap.cc:492] T 252e506a97564739bc26268653efa864 P 66ca148489d64deabd1dcbf7870c6d57: Bootstrap starting.
I20260926 15:02:41.580523 9995 consensus_queue.cc:260] T 7fa51439f5324161991e8dbbd169851e P ae330d635b1848aaaecee2c7ce96520a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } }
I20260926 15:02:41.582679 9995 ts_tablet_manager.cc:1428] T 7fa51439f5324161991e8dbbd169851e P ae330d635b1848aaaecee2c7ce96520a: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260926 15:02:41.584435 9993 ts_tablet_manager.cc:1918] T 317305a7fdb54f0bb2bb569272550b7b P ae330d635b1848aaaecee2c7ce96520a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 15:02:41.584596 9993 log.cc:1196] T 317305a7fdb54f0bb2bb569272550b7b P ae330d635b1848aaaecee2c7ce96520a: Deleting WAL directory at /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-1/wal/wals/317305a7fdb54f0bb2bb569272550b7b
I20260926 15:02:41.585060 9993 ts_tablet_manager.cc:1939] T 317305a7fdb54f0bb2bb569272550b7b P ae330d635b1848aaaecee2c7ce96520a: Deleting consensus metadata
I20260926 15:02:41.585690 9995 tablet_bootstrap.cc:492] T d4c76c4e1678474c92af007a7b5f7596 P ae330d635b1848aaaecee2c7ce96520a: Bootstrap starting.
I20260926 15:02:41.587285 9990 tablet_bootstrap.cc:654] T 252e506a97564739bc26268653efa864 P 66ca148489d64deabd1dcbf7870c6d57: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:41.587710 9995 tablet_bootstrap.cc:654] T d4c76c4e1678474c92af007a7b5f7596 P ae330d635b1848aaaecee2c7ce96520a: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:41.588344 9993 tablet_replica.cc:330] T 4bec0ca69b0e477281a232c1bf1121fb P ae330d635b1848aaaecee2c7ce96520a: stopping tablet replica
I20260926 15:02:41.588539 9993 raft_consensus.cc:2229] T 4bec0ca69b0e477281a232c1bf1121fb P ae330d635b1848aaaecee2c7ce96520a [term 1 LEADER]: Raft consensus shutting down.
I20260926 15:02:41.588812 9993 pending_rounds.cc:62] T 4bec0ca69b0e477281a232c1bf1121fb P ae330d635b1848aaaecee2c7ce96520a: Trying to abort 1 pending ops.
I20260926 15:02:41.588887 9993 pending_rounds.cc:68] T 4bec0ca69b0e477281a232c1bf1121fb P ae330d635b1848aaaecee2c7ce96520a: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333621598651924480 op_type: NO_OP noop_request { }
I20260926 15:02:41.589031 9993 raft_consensus.cc:2866] T 4bec0ca69b0e477281a232c1bf1121fb P ae330d635b1848aaaecee2c7ce96520a [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 15:02:41.589145 9993 raft_consensus.cc:2258] T 4bec0ca69b0e477281a232c1bf1121fb P ae330d635b1848aaaecee2c7ce96520a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:41.590133 9990 tablet_bootstrap.cc:492] T 252e506a97564739bc26268653efa864 P 66ca148489d64deabd1dcbf7870c6d57: No bootstrap required, opened a new log
I20260926 15:02:41.590282 9990 ts_tablet_manager.cc:1397] T 252e506a97564739bc26268653efa864 P 66ca148489d64deabd1dcbf7870c6d57: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20260926 15:02:41.590960 9990 raft_consensus.cc:348] T 252e506a97564739bc26268653efa864 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:41.591193 9990 raft_consensus.cc:374] T 252e506a97564739bc26268653efa864 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:41.591282 9990 raft_consensus.cc:729] T 252e506a97564739bc26268653efa864 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ca148489d64deabd1dcbf7870c6d57, State: Initialized, Role: FOLLOWER
I20260926 15:02:41.591508 9990 consensus_queue.cc:260] T 252e506a97564739bc26268653efa864 P 66ca148489d64deabd1dcbf7870c6d57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:41.592165 9990 ts_tablet_manager.cc:1428] T 252e506a97564739bc26268653efa864 P 66ca148489d64deabd1dcbf7870c6d57: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 15:02:41.592743 9993 ts_tablet_manager.cc:1905] T 4bec0ca69b0e477281a232c1bf1121fb P ae330d635b1848aaaecee2c7ce96520a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:41.597290 9995 tablet_bootstrap.cc:492] T d4c76c4e1678474c92af007a7b5f7596 P ae330d635b1848aaaecee2c7ce96520a: No bootstrap required, opened a new log
I20260926 15:02:41.597539 9995 ts_tablet_manager.cc:1397] T d4c76c4e1678474c92af007a7b5f7596 P ae330d635b1848aaaecee2c7ce96520a: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.003s
I20260926 15:02:41.598308 9995 raft_consensus.cc:348] T d4c76c4e1678474c92af007a7b5f7596 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:41.598537 9995 raft_consensus.cc:374] T d4c76c4e1678474c92af007a7b5f7596 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:41.598641 9995 raft_consensus.cc:729] T d4c76c4e1678474c92af007a7b5f7596 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae330d635b1848aaaecee2c7ce96520a, State: Initialized, Role: FOLLOWER
I20260926 15:02:41.598943 9995 consensus_queue.cc:260] T d4c76c4e1678474c92af007a7b5f7596 P ae330d635b1848aaaecee2c7ce96520a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:41.599552 9995 ts_tablet_manager.cc:1428] T d4c76c4e1678474c92af007a7b5f7596 P ae330d635b1848aaaecee2c7ce96520a: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260926 15:02:41.599592 9993 ts_tablet_manager.cc:1918] T 4bec0ca69b0e477281a232c1bf1121fb P ae330d635b1848aaaecee2c7ce96520a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 15:02:41.599819 9993 log.cc:1196] T 4bec0ca69b0e477281a232c1bf1121fb P ae330d635b1848aaaecee2c7ce96520a: Deleting WAL directory at /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-1/wal/wals/4bec0ca69b0e477281a232c1bf1121fb
I20260926 15:02:41.600003 9995 tablet_bootstrap.cc:492] T 252e506a97564739bc26268653efa864 P ae330d635b1848aaaecee2c7ce96520a: Bootstrap starting.
I20260926 15:02:41.600322 9993 ts_tablet_manager.cc:1939] T 4bec0ca69b0e477281a232c1bf1121fb P ae330d635b1848aaaecee2c7ce96520a: Deleting consensus metadata
I20260926 15:02:41.601532 9993 tablet_replica.cc:330] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P ae330d635b1848aaaecee2c7ce96520a: stopping tablet replica
I20260926 15:02:41.601763 9993 raft_consensus.cc:2229] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P ae330d635b1848aaaecee2c7ce96520a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:02:41.601909 9995 tablet_bootstrap.cc:654] T 252e506a97564739bc26268653efa864 P ae330d635b1848aaaecee2c7ce96520a: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:41.601974 9993 raft_consensus.cc:2258] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P ae330d635b1848aaaecee2c7ce96520a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:41.602548 9993 ts_tablet_manager.cc:1905] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P ae330d635b1848aaaecee2c7ce96520a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:41.607383 9995 tablet_bootstrap.cc:492] T 252e506a97564739bc26268653efa864 P ae330d635b1848aaaecee2c7ce96520a: No bootstrap required, opened a new log
I20260926 15:02:41.607594 9995 ts_tablet_manager.cc:1397] T 252e506a97564739bc26268653efa864 P ae330d635b1848aaaecee2c7ce96520a: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20260926 15:02:41.608397 9995 raft_consensus.cc:348] T 252e506a97564739bc26268653efa864 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:41.608650 9995 raft_consensus.cc:374] T 252e506a97564739bc26268653efa864 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:41.608767 9995 raft_consensus.cc:729] T 252e506a97564739bc26268653efa864 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae330d635b1848aaaecee2c7ce96520a, State: Initialized, Role: FOLLOWER
I20260926 15:02:41.609066 9995 consensus_queue.cc:260] T 252e506a97564739bc26268653efa864 P ae330d635b1848aaaecee2c7ce96520a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:41.609544 9993 ts_tablet_manager.cc:1918] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P ae330d635b1848aaaecee2c7ce96520a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 15:02:41.609686 9995 ts_tablet_manager.cc:1428] T 252e506a97564739bc26268653efa864 P ae330d635b1848aaaecee2c7ce96520a: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260926 15:02:41.609711 9993 log.cc:1196] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P ae330d635b1848aaaecee2c7ce96520a: Deleting WAL directory at /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/ts-1/wal/wals/9134d14fae4d46b6bd8ef1c05dc5b1c2
I20260926 15:02:41.610425 9993 ts_tablet_manager.cc:1939] T 9134d14fae4d46b6bd8ef1c05dc5b1c2 P ae330d635b1848aaaecee2c7ce96520a: Deleting consensus metadata
I20260926 15:02:41.629449 9915 raft_consensus.cc:482] T c8d090f2e0384924a72f288eff2281e7 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:02:41.629690 9915 raft_consensus.cc:504] T c8d090f2e0384924a72f288eff2281e7 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:41.630501 9915 leader_election.cc:288] T c8d090f2e0384924a72f288eff2281e7 P ae330d635b1848aaaecee2c7ce96520a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327), f052d5cbee65406593d8e44f74225bc2 (127.8.138.131:35445)
I20260926 15:02:41.631084 9345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8d090f2e0384924a72f288eff2281e7" candidate_uuid: "ae330d635b1848aaaecee2c7ce96520a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ca148489d64deabd1dcbf7870c6d57" is_pre_election: true
I20260926 15:02:41.631090 9632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8d090f2e0384924a72f288eff2281e7" candidate_uuid: "ae330d635b1848aaaecee2c7ce96520a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f052d5cbee65406593d8e44f74225bc2" is_pre_election: true
I20260926 15:02:41.631345 9345 raft_consensus.cc:2454] T c8d090f2e0384924a72f288eff2281e7 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae330d635b1848aaaecee2c7ce96520a in term 0.
I20260926 15:02:41.631363 9632 raft_consensus.cc:2454] T c8d090f2e0384924a72f288eff2281e7 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae330d635b1848aaaecee2c7ce96520a in term 0.
I20260926 15:02:41.631888 9421 leader_election.cc:302] T c8d090f2e0384924a72f288eff2281e7 P ae330d635b1848aaaecee2c7ce96520a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ca148489d64deabd1dcbf7870c6d57, ae330d635b1848aaaecee2c7ce96520a; no voters:
I20260926 15:02:41.632193 9915 raft_consensus.cc:2781] T c8d090f2e0384924a72f288eff2281e7 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:02:41.632308 9915 raft_consensus.cc:482] T c8d090f2e0384924a72f288eff2281e7 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:02:41.632395 9915 raft_consensus.cc:3037] T c8d090f2e0384924a72f288eff2281e7 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:41.634347 9915 raft_consensus.cc:504] T c8d090f2e0384924a72f288eff2281e7 P ae330d635b1848aaaecee2c7ce96520a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:41.635519 9632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8d090f2e0384924a72f288eff2281e7" candidate_uuid: "ae330d635b1848aaaecee2c7ce96520a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f052d5cbee65406593d8e44f74225bc2"
I20260926 15:02:41.635522 9345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8d090f2e0384924a72f288eff2281e7" candidate_uuid: "ae330d635b1848aaaecee2c7ce96520a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ca148489d64deabd1dcbf7870c6d57"
I20260926 15:02:41.635751 9632 raft_consensus.cc:3037] T c8d090f2e0384924a72f288eff2281e7 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:41.635764 9345 raft_consensus.cc:3037] T c8d090f2e0384924a72f288eff2281e7 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:41.638707 9345 raft_consensus.cc:2454] T c8d090f2e0384924a72f288eff2281e7 P 66ca148489d64deabd1dcbf7870c6d57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae330d635b1848aaaecee2c7ce96520a in term 1.
I20260926 15:02:41.638720 9632 raft_consensus.cc:2454] T c8d090f2e0384924a72f288eff2281e7 P f052d5cbee65406593d8e44f74225bc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae330d635b1848aaaecee2c7ce96520a in term 1.
I20260926 15:02:41.639312 9421 leader_election.cc:302] T c8d090f2e0384924a72f288eff2281e7 P ae330d635b1848aaaecee2c7ce96520a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ca148489d64deabd1dcbf7870c6d57, ae330d635b1848aaaecee2c7ce96520a; no voters:
I20260926 15:02:41.639312 9915 leader_election.cc:288] T c8d090f2e0384924a72f288eff2281e7 P ae330d635b1848aaaecee2c7ce96520a [CANDIDATE]: Term 1 election: Requested vote from peers 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327), f052d5cbee65406593d8e44f74225bc2 (127.8.138.131:35445)
I20260926 15:02:41.640100 10009 raft_consensus.cc:2781] T c8d090f2e0384924a72f288eff2281e7 P ae330d635b1848aaaecee2c7ce96520a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:02:41.640290 10009 raft_consensus.cc:686] T c8d090f2e0384924a72f288eff2281e7 P ae330d635b1848aaaecee2c7ce96520a [term 1 LEADER]: Becoming Leader. State: Replica: ae330d635b1848aaaecee2c7ce96520a, State: Running, Role: LEADER
I20260926 15:02:41.640632 10009 consensus_queue.cc:237] T c8d090f2e0384924a72f288eff2281e7 P ae330d635b1848aaaecee2c7ce96520a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:41.698529 9930 consensus_queue.cc:1035] T 41e641e28d1f48b2a7e0c8fb8964f678 P f052d5cbee65406593d8e44f74225bc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:02:41.701138 9930 raft_consensus.cc:482] T 7fa51439f5324161991e8dbbd169851e P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:02:41.701398 9930 raft_consensus.cc:504] T 7fa51439f5324161991e8dbbd169851e P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } }
I20260926 15:02:41.702061 9930 leader_election.cc:288] T 7fa51439f5324161991e8dbbd169851e P f052d5cbee65406593d8e44f74225bc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327), ae330d635b1848aaaecee2c7ce96520a (127.8.138.130:44309)
I20260926 15:02:41.702597 9487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fa51439f5324161991e8dbbd169851e" candidate_uuid: "f052d5cbee65406593d8e44f74225bc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae330d635b1848aaaecee2c7ce96520a" is_pre_election: true
I20260926 15:02:41.702893 9487 raft_consensus.cc:2454] T 7fa51439f5324161991e8dbbd169851e P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f052d5cbee65406593d8e44f74225bc2 in term 0.
I20260926 15:02:41.702913 9345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fa51439f5324161991e8dbbd169851e" candidate_uuid: "f052d5cbee65406593d8e44f74225bc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ca148489d64deabd1dcbf7870c6d57" is_pre_election: true
I20260926 15:02:41.703145 9345 raft_consensus.cc:2454] T 7fa51439f5324161991e8dbbd169851e P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f052d5cbee65406593d8e44f74225bc2 in term 0.
I20260926 15:02:41.703544 9567 leader_election.cc:302] T 7fa51439f5324161991e8dbbd169851e P f052d5cbee65406593d8e44f74225bc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae330d635b1848aaaecee2c7ce96520a, f052d5cbee65406593d8e44f74225bc2; no voters:
I20260926 15:02:41.703946 9930 raft_consensus.cc:2781] T 7fa51439f5324161991e8dbbd169851e P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:02:41.704049 9930 raft_consensus.cc:482] T 7fa51439f5324161991e8dbbd169851e P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:02:41.704177 9930 raft_consensus.cc:3037] T 7fa51439f5324161991e8dbbd169851e P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:41.707278 10012 consensus_queue.cc:1035] T 41e641e28d1f48b2a7e0c8fb8964f678 P f052d5cbee65406593d8e44f74225bc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260926 15:02:41.709506 9930 raft_consensus.cc:504] T 7fa51439f5324161991e8dbbd169851e P f052d5cbee65406593d8e44f74225bc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } }
I20260926 15:02:41.710232 9930 leader_election.cc:288] T 7fa51439f5324161991e8dbbd169851e P f052d5cbee65406593d8e44f74225bc2 [CANDIDATE]: Term 1 election: Requested vote from peers 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327), ae330d635b1848aaaecee2c7ce96520a (127.8.138.130:44309)
I20260926 15:02:41.710654 9344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fa51439f5324161991e8dbbd169851e" candidate_uuid: "f052d5cbee65406593d8e44f74225bc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ca148489d64deabd1dcbf7870c6d57"
I20260926 15:02:41.710870 9344 raft_consensus.cc:3037] T 7fa51439f5324161991e8dbbd169851e P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:41.710896 9487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fa51439f5324161991e8dbbd169851e" candidate_uuid: "f052d5cbee65406593d8e44f74225bc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae330d635b1848aaaecee2c7ce96520a"
I20260926 15:02:41.711099 9487 raft_consensus.cc:3037] T 7fa51439f5324161991e8dbbd169851e P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:41.713918 9487 raft_consensus.cc:2454] T 7fa51439f5324161991e8dbbd169851e P ae330d635b1848aaaecee2c7ce96520a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f052d5cbee65406593d8e44f74225bc2 in term 1.
I20260926 15:02:41.713932 9344 raft_consensus.cc:2454] T 7fa51439f5324161991e8dbbd169851e P 66ca148489d64deabd1dcbf7870c6d57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f052d5cbee65406593d8e44f74225bc2 in term 1.
I20260926 15:02:41.714331 9567 leader_election.cc:302] T 7fa51439f5324161991e8dbbd169851e P f052d5cbee65406593d8e44f74225bc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae330d635b1848aaaecee2c7ce96520a, f052d5cbee65406593d8e44f74225bc2; no voters:
I20260926 15:02:41.714627 10013 raft_consensus.cc:2781] T 7fa51439f5324161991e8dbbd169851e P f052d5cbee65406593d8e44f74225bc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:02:41.714773 10013 raft_consensus.cc:686] T 7fa51439f5324161991e8dbbd169851e P f052d5cbee65406593d8e44f74225bc2 [term 1 LEADER]: Becoming Leader. State: Replica: f052d5cbee65406593d8e44f74225bc2, State: Running, Role: LEADER
I20260926 15:02:41.715047 10013 consensus_queue.cc:237] T 7fa51439f5324161991e8dbbd169851e P f052d5cbee65406593d8e44f74225bc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } }
W20260926 15:02:41.771255 9278 connection.cc:535] server connection from 127.0.0.1:33164 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 15:02:41.771281 9421 connection.cc:535] server connection from 127.0.0.1:50634 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 15:02:41.771471 9689 connection.cc:535] client connection to 127.8.138.188:45473 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 15:02:41.771543 9422 connection.cc:535] client connection to 127.8.138.188:45473 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 15:02:41.771989 9280 connection.cc:535] client connection to 127.8.138.188:45473 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 15:02:41.848078 10009 raft_consensus.cc:482] T d4c76c4e1678474c92af007a7b5f7596 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:02:41.848318 10009 raft_consensus.cc:504] T d4c76c4e1678474c92af007a7b5f7596 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:41.848984 10009 leader_election.cc:288] T d4c76c4e1678474c92af007a7b5f7596 P ae330d635b1848aaaecee2c7ce96520a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327), f052d5cbee65406593d8e44f74225bc2 (127.8.138.131:35445)
I20260926 15:02:41.849483 9344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4c76c4e1678474c92af007a7b5f7596" candidate_uuid: "ae330d635b1848aaaecee2c7ce96520a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ca148489d64deabd1dcbf7870c6d57" is_pre_election: true
I20260926 15:02:41.849627 9632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4c76c4e1678474c92af007a7b5f7596" candidate_uuid: "ae330d635b1848aaaecee2c7ce96520a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f052d5cbee65406593d8e44f74225bc2" is_pre_election: true
I20260926 15:02:41.849795 9344 raft_consensus.cc:2454] T d4c76c4e1678474c92af007a7b5f7596 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae330d635b1848aaaecee2c7ce96520a in term 0.
I20260926 15:02:41.849913 9632 raft_consensus.cc:2454] T d4c76c4e1678474c92af007a7b5f7596 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae330d635b1848aaaecee2c7ce96520a in term 0.
I20260926 15:02:41.850281 9421 leader_election.cc:302] T d4c76c4e1678474c92af007a7b5f7596 P ae330d635b1848aaaecee2c7ce96520a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ca148489d64deabd1dcbf7870c6d57, ae330d635b1848aaaecee2c7ce96520a; no voters:
I20260926 15:02:41.850553 10009 raft_consensus.cc:2781] T d4c76c4e1678474c92af007a7b5f7596 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:02:41.850664 10009 raft_consensus.cc:482] T d4c76c4e1678474c92af007a7b5f7596 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:02:41.850742 10009 raft_consensus.cc:3037] T d4c76c4e1678474c92af007a7b5f7596 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:41.852769 10009 raft_consensus.cc:504] T d4c76c4e1678474c92af007a7b5f7596 P ae330d635b1848aaaecee2c7ce96520a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:41.853318 10009 leader_election.cc:288] T d4c76c4e1678474c92af007a7b5f7596 P ae330d635b1848aaaecee2c7ce96520a [CANDIDATE]: Term 1 election: Requested vote from peers 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327), f052d5cbee65406593d8e44f74225bc2 (127.8.138.131:35445)
I20260926 15:02:41.853760 9344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4c76c4e1678474c92af007a7b5f7596" candidate_uuid: "ae330d635b1848aaaecee2c7ce96520a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ca148489d64deabd1dcbf7870c6d57"
I20260926 15:02:41.853848 9632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4c76c4e1678474c92af007a7b5f7596" candidate_uuid: "ae330d635b1848aaaecee2c7ce96520a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f052d5cbee65406593d8e44f74225bc2"
I20260926 15:02:41.853978 9344 raft_consensus.cc:3037] T d4c76c4e1678474c92af007a7b5f7596 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:41.854087 9632 raft_consensus.cc:3037] T d4c76c4e1678474c92af007a7b5f7596 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:41.855947 9344 raft_consensus.cc:2454] T d4c76c4e1678474c92af007a7b5f7596 P 66ca148489d64deabd1dcbf7870c6d57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae330d635b1848aaaecee2c7ce96520a in term 1.
I20260926 15:02:41.856266 9632 raft_consensus.cc:2454] T d4c76c4e1678474c92af007a7b5f7596 P f052d5cbee65406593d8e44f74225bc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae330d635b1848aaaecee2c7ce96520a in term 1.
I20260926 15:02:41.856335 9421 leader_election.cc:302] T d4c76c4e1678474c92af007a7b5f7596 P ae330d635b1848aaaecee2c7ce96520a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ca148489d64deabd1dcbf7870c6d57, ae330d635b1848aaaecee2c7ce96520a; no voters:
I20260926 15:02:41.856644 10009 raft_consensus.cc:2781] T d4c76c4e1678474c92af007a7b5f7596 P ae330d635b1848aaaecee2c7ce96520a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:02:41.856833 10009 raft_consensus.cc:686] T d4c76c4e1678474c92af007a7b5f7596 P ae330d635b1848aaaecee2c7ce96520a [term 1 LEADER]: Becoming Leader. State: Replica: ae330d635b1848aaaecee2c7ce96520a, State: Running, Role: LEADER
I20260926 15:02:41.857388 10009 consensus_queue.cc:237] T d4c76c4e1678474c92af007a7b5f7596 P ae330d635b1848aaaecee2c7ce96520a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:41.869026 10013 raft_consensus.cc:482] T 252e506a97564739bc26268653efa864 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:02:41.869221 10013 raft_consensus.cc:504] T 252e506a97564739bc26268653efa864 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:41.869844 10013 leader_election.cc:288] T 252e506a97564739bc26268653efa864 P f052d5cbee65406593d8e44f74225bc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327), ae330d635b1848aaaecee2c7ce96520a (127.8.138.130:44309)
I20260926 15:02:41.870446 9487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "252e506a97564739bc26268653efa864" candidate_uuid: "f052d5cbee65406593d8e44f74225bc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae330d635b1848aaaecee2c7ce96520a" is_pre_election: true
I20260926 15:02:41.870460 9344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "252e506a97564739bc26268653efa864" candidate_uuid: "f052d5cbee65406593d8e44f74225bc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ca148489d64deabd1dcbf7870c6d57" is_pre_election: true
I20260926 15:02:41.870683 9487 raft_consensus.cc:2454] T 252e506a97564739bc26268653efa864 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f052d5cbee65406593d8e44f74225bc2 in term 0.
I20260926 15:02:41.870702 9344 raft_consensus.cc:2454] T 252e506a97564739bc26268653efa864 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f052d5cbee65406593d8e44f74225bc2 in term 0.
I20260926 15:02:41.871120 9566 leader_election.cc:302] T 252e506a97564739bc26268653efa864 P f052d5cbee65406593d8e44f74225bc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66ca148489d64deabd1dcbf7870c6d57, f052d5cbee65406593d8e44f74225bc2; no voters:
I20260926 15:02:41.871490 10013 raft_consensus.cc:2781] T 252e506a97564739bc26268653efa864 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:02:41.871594 10013 raft_consensus.cc:482] T 252e506a97564739bc26268653efa864 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:02:41.871770 10013 raft_consensus.cc:3037] T 252e506a97564739bc26268653efa864 P f052d5cbee65406593d8e44f74225bc2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:41.873965 10013 raft_consensus.cc:504] T 252e506a97564739bc26268653efa864 P f052d5cbee65406593d8e44f74225bc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:41.874567 10013 leader_election.cc:288] T 252e506a97564739bc26268653efa864 P f052d5cbee65406593d8e44f74225bc2 [CANDIDATE]: Term 1 election: Requested vote from peers 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327), ae330d635b1848aaaecee2c7ce96520a (127.8.138.130:44309)
I20260926 15:02:41.875129 9487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "252e506a97564739bc26268653efa864" candidate_uuid: "f052d5cbee65406593d8e44f74225bc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae330d635b1848aaaecee2c7ce96520a"
I20260926 15:02:41.875131 9344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "252e506a97564739bc26268653efa864" candidate_uuid: "f052d5cbee65406593d8e44f74225bc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66ca148489d64deabd1dcbf7870c6d57"
I20260926 15:02:41.875339 9487 raft_consensus.cc:3037] T 252e506a97564739bc26268653efa864 P ae330d635b1848aaaecee2c7ce96520a [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:41.875439 9344 raft_consensus.cc:3037] T 252e506a97564739bc26268653efa864 P 66ca148489d64deabd1dcbf7870c6d57 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:41.877609 9487 raft_consensus.cc:2454] T 252e506a97564739bc26268653efa864 P ae330d635b1848aaaecee2c7ce96520a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f052d5cbee65406593d8e44f74225bc2 in term 1.
I20260926 15:02:41.877681 9344 raft_consensus.cc:2454] T 252e506a97564739bc26268653efa864 P 66ca148489d64deabd1dcbf7870c6d57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f052d5cbee65406593d8e44f74225bc2 in term 1.
I20260926 15:02:41.878054 9567 leader_election.cc:302] T 252e506a97564739bc26268653efa864 P f052d5cbee65406593d8e44f74225bc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae330d635b1848aaaecee2c7ce96520a, f052d5cbee65406593d8e44f74225bc2; no voters:
I20260926 15:02:41.878361 10013 raft_consensus.cc:2781] T 252e506a97564739bc26268653efa864 P f052d5cbee65406593d8e44f74225bc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:02:41.878540 10013 raft_consensus.cc:686] T 252e506a97564739bc26268653efa864 P f052d5cbee65406593d8e44f74225bc2 [term 1 LEADER]: Becoming Leader. State: Replica: f052d5cbee65406593d8e44f74225bc2, State: Running, Role: LEADER
I20260926 15:02:41.879127 10013 consensus_queue.cc:237] T 252e506a97564739bc26268653efa864 P f052d5cbee65406593d8e44f74225bc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 } } peers { permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 } } peers { permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 } }
I20260926 15:02:41.893290 10009 consensus_queue.cc:1035] T 51ff1e9e47864b24b7c7a869ff37e06a P ae330d635b1848aaaecee2c7ce96520a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:02:41.901245 10009 consensus_queue.cc:1035] T 51ff1e9e47864b24b7c7a869ff37e06a P ae330d635b1848aaaecee2c7ce96520a [LEADER]: Connected to new peer: Peer: permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:02:42.038125 10028 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:02:42.038357 10028 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:42.039086 10028 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:02:42.041220 9791 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
W20260926 15:02:42.041433 9790 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
I20260926 15:02:42.041566 9790 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:02:42.041930 10028 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260926 15:02:42.243113 10033 consensus_queue.cc:1035] T 0d837276d648440cbebe1f88c281cb11 P 66ca148489d64deabd1dcbf7870c6d57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:02:42.248569 9933 consensus_queue.cc:1035] T 0d837276d648440cbebe1f88c281cb11 P 66ca148489d64deabd1dcbf7870c6d57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:02:42.348272 8746 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu
/tmp/dist-test-taskLGH8i2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.138.188:45473
--webserver_interface=127.8.138.188
--webserver_port=38457
--builtin_ntp_servers=127.8.138.148:33835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473
--hive_metastore_uris=thrift://127.0.0.1:40047
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260926 15:02:42.511234 10009 consensus_queue.cc:1035] T c8d090f2e0384924a72f288eff2281e7 P ae330d635b1848aaaecee2c7ce96520a [LEADER]: Connected to new peer: Peer: permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:02:42.517920 10040 consensus_queue.cc:1035] T c8d090f2e0384924a72f288eff2281e7 P ae330d635b1848aaaecee2c7ce96520a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 15:02:42.545233 10042 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:02:42.545473 10042 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:42.546283 10042 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:02:42.548648 9790 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
W20260926 15:02:42.549576 9791 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:02:42.549722 9791 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:02:42.549979 10042 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260926 15:02:42.552076 10013 consensus_queue.cc:1035] T 7fa51439f5324161991e8dbbd169851e P f052d5cbee65406593d8e44f74225bc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:02:42.563239 10016 consensus_queue.cc:1035] T 7fa51439f5324161991e8dbbd169851e P f052d5cbee65406593d8e44f74225bc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:02:42.618172 10039 consensus_queue.cc:1035] T d4c76c4e1678474c92af007a7b5f7596 P ae330d635b1848aaaecee2c7ce96520a [LEADER]: Connected to new peer: Peer: permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:02:42.624416 10040 consensus_queue.cc:1035] T d4c76c4e1678474c92af007a7b5f7596 P ae330d635b1848aaaecee2c7ce96520a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" member_type: VOTER last_known_addr { host: "127.8.138.131" port: 35445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:02:42.637346 10016 consensus_queue.cc:1035] T 252e506a97564739bc26268653efa864 P f052d5cbee65406593d8e44f74225bc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" member_type: VOTER last_known_addr { host: "127.8.138.129" port: 40327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:02:42.647023 10016 consensus_queue.cc:1035] T 252e506a97564739bc26268653efa864 P f052d5cbee65406593d8e44f74225bc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" member_type: VOTER last_known_addr { host: "127.8.138.130" port: 44309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 15:02:42.731030 10036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:42.731355 10036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:42.731436 10036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:42.739440 10036 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:02:42.739559 10036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:42.739600 10036 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:02:42.739650 10036 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:02:42.752058 10036 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.138.148:33835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:40047
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.138.188:45473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.138.188
--webserver_port=38457
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:42.753602 10036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:42.755540 10036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:42.768903 10057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:42.768901 10056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:42.768927 10059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:42.769523 10036 server_base.cc:1034] running on GCE node
I20260926 15:02:42.770344 10036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:42.771795 10036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:42.773255 10036 hybrid_clock.cc:648] HybridClock initialized: now 1790434962773075 us; error 78 us; skew 500 ppm
I20260926 15:02:42.773658 10036 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:02:42.776278 10036 webserver.cc:460] Webserver started at http://127.8.138.188:38457/ using document root <none> and password file <none>
I20260926 15:02:42.777021 10036 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:42.777176 10036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:42.782727 10036 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.005s
I20260926 15:02:42.786123 10065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:42.787987 10036 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260926 15:02:42.788180 10036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data,/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/wal
uuid: "320cd09624624312bf1bcc0856c2874f"
format_stamp: "Formatted at 2026-09-26 15:02:35 on dist-test-slave-7qf4"
I20260926 15:02:42.789213 10036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:42.803666 10036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:42.804790 10036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:42.805204 10036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:42.825708 10036 rpc_server.cc:307] RPC server started. Bound to: 127.8.138.188:45473
I20260926 15:02:42.825750 10116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.138.188:45473 every 8 connection(s)
I20260926 15:02:42.827266 10036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data/info.pb
I20260926 15:02:42.828070 8746 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu as pid 10036
I20260926 15:02:42.835103 10117 sys_catalog.cc:263] Verifying existing consensus state
I20260926 15:02:42.839993 10117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: Bootstrap starting.
I20260926 15:02:42.852067 10117 log.cc:824] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: Log is configured to *not* fsync() on all Append() calls
I20260926 15:02:42.889258 10117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: Bootstrap replayed 1/1 log segments. Stats: ops{read=40 overwritten=0 applied=40 ignored=0} inserts{seen=27 ignored=0} mutations{seen=26 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 15:02:42.889837 10117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: Bootstrap complete.
I20260926 15:02:42.899044 10117 raft_consensus.cc:348] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 5 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:42.900179 10117 raft_consensus.cc:729] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 5 FOLLOWER]: Becoming Follower/Learner. State: Replica: 320cd09624624312bf1bcc0856c2874f, State: Initialized, Role: FOLLOWER
I20260926 15:02:42.900931 10117 consensus_queue.cc:260] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 40, Last appended: 5.40, Last appended by leader: 40, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:42.901777 10121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 5 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:42.902179 10121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:02:42.902918 10117 sys_catalog.cc:564] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:02:42.925360 10135 catalog_manager.cc:1259] Loaded cluster ID: 497ed160bc39477e99bf3beb637a4391
I20260926 15:02:42.925489 10135 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: loading cluster ID for follower catalog manager: success
I20260926 15:02:42.930600 10135 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: acquiring CA information for follower catalog manager: success
I20260926 15:02:42.933765 10135 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 15:02:42.964409 10036 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:02:42.970666 10036 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:02:43.098095 10140 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:02:43.098301 10140 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:43.099023 10140 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:02:43.101734 9791 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:02:43.107801 10092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6315fb5099e1420b95b8d83b3e19375c" candidate_term: 6 candidate_status { last_received { term: 5 index: 40 } } ignore_live_leader: false dest_uuid: "320cd09624624312bf1bcc0856c2874f" is_pre_election: true
I20260926 15:02:43.108603 10092 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6315fb5099e1420b95b8d83b3e19375c in term 5.
I20260926 15:02:43.109308 9790 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f, 6315fb5099e1420b95b8d83b3e19375c; no voters: be9958f2085242f7ab82ae933ceb1c42
I20260926 15:02:43.109650 10140 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260926 15:02:43.109783 10140 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:02:43.109902 10140 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 5 FOLLOWER]: Advancing to term 6
I20260926 15:02:43.113018 10140 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:43.113744 10140 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 6 election: Requested vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
I20260926 15:02:43.114800 10092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6315fb5099e1420b95b8d83b3e19375c" candidate_term: 6 candidate_status { last_received { term: 5 index: 40 } } ignore_live_leader: false dest_uuid: "320cd09624624312bf1bcc0856c2874f"
I20260926 15:02:43.115025 10092 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 5 FOLLOWER]: Advancing to term 6
W20260926 15:02:43.115336 9791 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:02:43.119524 10092 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6315fb5099e1420b95b8d83b3e19375c in term 6.
I20260926 15:02:43.120020 9790 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f, 6315fb5099e1420b95b8d83b3e19375c; no voters: be9958f2085242f7ab82ae933ceb1c42
I20260926 15:02:43.120390 10140 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 6 FOLLOWER]: Leader election won for term 6
I20260926 15:02:43.120911 10140 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 6 LEADER]: Becoming Leader. State: Replica: 6315fb5099e1420b95b8d83b3e19375c, State: Running, Role: LEADER
I20260926 15:02:43.121475 10140 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 40, Committed index: 40, Last appended: 5.40, Last appended by leader: 40, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:43.123390 10144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6315fb5099e1420b95b8d83b3e19375c. Latest consensus state: current_term: 6 leader_uuid: "6315fb5099e1420b95b8d83b3e19375c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:43.123960 10144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: This master's current role is: LEADER
I20260926 15:02:43.124845 10146 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:02:43.127461 10146 catalog_manager.cc:669] Loaded metadata for table default.table_43176f08d9e24974abdd53171465a15c [id=08b331ce5f9c44389fae209bac801f42]
I20260926 15:02:43.128013 10146 catalog_manager.cc:669] Loaded metadata for table default.table_854219551f9048c6925c711ce33d5256 [id=3295fc8a0a004843be045b8bf6a4f2cf]
I20260926 15:02:43.128427 10146 catalog_manager.cc:669] Loaded metadata for table default.table_96be20374f7e43e7a865549e39bd64f6 [id=51dea274f4bf4903baaddbbd8c257b27]
I20260926 15:02:43.128823 10146 catalog_manager.cc:669] Loaded metadata for table default.table_d2243274908043e58210a197141af813 [id=5357ddcf21634f99bb4f9abc5b6c4d80]
I20260926 15:02:43.129262 10146 catalog_manager.cc:669] Loaded metadata for table default.table_95f48725b6ce47fab5f96e18ccb69dbc [id=630deebc15b04878ae13844ff2808f52]
I20260926 15:02:43.129683 10146 catalog_manager.cc:669] Loaded metadata for table default.table_22fd4957e3d14964803428457f8490ac [id=7ba7b39b84394421b88a6ce7616c3c91]
I20260926 15:02:43.130110 10146 catalog_manager.cc:669] Loaded metadata for table default.table_41a3ce3c89684cbb9054c4251a0cc51c [id=7cc0c844d49c4ff08cef9e2c410991f8]
I20260926 15:02:43.130527 10146 catalog_manager.cc:669] Loaded metadata for table default.table_7b56039466d14dcab4dc63192de3b08e [id=b84ce6d15e4246dbbb299468138add55]
I20260926 15:02:43.130882 10146 catalog_manager.cc:669] Loaded metadata for table default.table_e3b713aac3c94e449bb5882310d55a40 [id=e9e378126d2649dab722c82f99d654b7]
I20260926 15:02:43.132762 10146 tablet_loader.cc:96] loaded metadata for tablet 0d837276d648440cbebe1f88c281cb11 (table default.table_96be20374f7e43e7a865549e39bd64f6 [id=51dea274f4bf4903baaddbbd8c257b27])
I20260926 15:02:43.133224 10146 tablet_loader.cc:96] loaded metadata for tablet 252e506a97564739bc26268653efa864 (table default.table_e3b713aac3c94e449bb5882310d55a40 [id=e9e378126d2649dab722c82f99d654b7])
I20260926 15:02:43.134125 10146 tablet_loader.cc:96] loaded metadata for tablet 3fae8a76002849de9bb26610a0c33406 (table default.table_95f48725b6ce47fab5f96e18ccb69dbc [id=630deebc15b04878ae13844ff2808f52])
I20260926 15:02:43.134507 10146 tablet_loader.cc:96] loaded metadata for tablet 41e641e28d1f48b2a7e0c8fb8964f678 (table default.table_d2243274908043e58210a197141af813 [id=5357ddcf21634f99bb4f9abc5b6c4d80])
I20260926 15:02:43.135224 10146 tablet_loader.cc:96] loaded metadata for tablet 51ff1e9e47864b24b7c7a869ff37e06a (table default.table_854219551f9048c6925c711ce33d5256 [id=3295fc8a0a004843be045b8bf6a4f2cf])
I20260926 15:02:43.135752 10146 tablet_loader.cc:96] loaded metadata for tablet 7fa51439f5324161991e8dbbd169851e (table default.table_41a3ce3c89684cbb9054c4251a0cc51c [id=7cc0c844d49c4ff08cef9e2c410991f8])
I20260926 15:02:43.136439 10146 tablet_loader.cc:96] loaded metadata for tablet c5d07d3a7a164d098e61f8f7f2ac4a67 (table default.table_43176f08d9e24974abdd53171465a15c [id=08b331ce5f9c44389fae209bac801f42])
I20260926 15:02:43.136773 10146 tablet_loader.cc:96] loaded metadata for tablet c8d090f2e0384924a72f288eff2281e7 (table default.table_22fd4957e3d14964803428457f8490ac [id=7ba7b39b84394421b88a6ce7616c3c91])
I20260926 15:02:43.137102 10146 tablet_loader.cc:96] loaded metadata for tablet d4c76c4e1678474c92af007a7b5f7596 (table default.table_7b56039466d14dcab4dc63192de3b08e [id=b84ce6d15e4246dbbb299468138add55])
I20260926 15:02:43.137413 10146 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:02:43.138531 10146 catalog_manager.cc:1259] Loaded cluster ID: 497ed160bc39477e99bf3beb637a4391
I20260926 15:02:43.138646 10146 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:02:43.140268 10146 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:02:43.141341 10146 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c: Loaded TSK: 0
I20260926 15:02:43.142395 10146 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:02:43.143541 10146 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260926 15:02:43.144981 10146 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260926 15:02:43.160157 9390 heartbeater.cc:347] Connected to a master server at 127.8.138.188:45473
I20260926 15:02:43.160430 9390 heartbeater.cc:503] Master 127.8.138.188:45473 was elected leader, sending a full tablet report...
I20260926 15:02:43.165007 10082 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" instance_seqno: 1790434956216737) as {username='slave'} at 127.8.138.129:58807; Asking this server to re-register.
I20260926 15:02:43.167913 9390 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:43.168296 9801 master_service.cc:946] ReplaceTablet: received request to replace tablet 7fa51439f5324161991e8dbbd169851e from {username='slave'} at 127.0.0.1:57478
I20260926 15:02:43.169348 9795 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:57478:
table { table_name: "default.Table_d2243274908043e58210a197141af813" } new_table_name: "default.Table_42b67a58948144f5a3b3abe972e975ba" modify_external_catalogs: true
I20260926 15:02:43.170009 9803 master_service.cc:946] ReplaceTablet: received request to replace tablet 51ff1e9e47864b24b7c7a869ff37e06a from {username='slave'} at 127.0.0.1:57478
I20260926 15:02:43.170790 10081 ts_manager.cc:194] Registered new tserver with Master: 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327)
I20260926 15:02:43.170478 9797 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57478:
name: "default.Table_a320fb1edf9a4372928664c4263b213e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:43.170897 9802 master_service.cc:946] ReplaceTablet: received request to replace tablet 51ff1e9e47864b24b7c7a869ff37e06a from {username='slave'} at 127.0.0.1:57478
I20260926 15:02:43.171226 9800 master_service.cc:946] ReplaceTablet: received request to replace tablet c8d090f2e0384924a72f288eff2281e7 from {username='slave'} at 127.0.0.1:57478
I20260926 15:02:43.171350 9799 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57478:
name: "default.Table_985d0971fb8a4d8eae0b2cb73ca99c6f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:43.172750 9804 master_service.cc:946] ReplaceTablet: received request to replace tablet d4c76c4e1678474c92af007a7b5f7596 from {username='slave'} at 127.0.0.1:57478
I20260926 15:02:43.172299 9796 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:57478:
table { table_name: "default.Table_41a3ce3c89684cbb9054c4251a0cc51c" } new_table_name: "default.Table_476468e6a23a414ebac439484ad9a68f" modify_external_catalogs: true
I20260926 15:02:43.178555 9798 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:57478:
table { table_name: "default.Table_43176f08d9e24974abdd53171465a15c" } new_table_name: "default.Table_1b2fcfff41b54f21b229a3e2064d7c26" modify_external_catalogs: true
F20260926 15:02:43.179319 9864 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_f356bc0dfc6a47f98430002dccbb873f
*** Check failure stack trace: ***
@ 0x7f93c11d29ad google::LogMessage::Fail() at ??:0
I20260926 15:02:43.183173 10092 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 6 FOLLOWER]: Refusing update from remote peer 6315fb5099e1420b95b8d83b3e19375c: Log matching property violated. Preceding OpId in replica: term: 5 index: 40. Preceding OpId from leader: term: 6 index: 42. (index mismatch)
@ 0x7f93c11d588a google::LogMessage::SendToLog() at ??:0
I20260926 15:02:43.185490 10144 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [LEADER]: Connected to new peer: Peer: permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 41, Last known committed idx: 40, Time since last communication: 0.000s
W20260926 15:02:43.186761 9791 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c -> Peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Couldn't send request to peer be9958f2085242f7ab82ae933ceb1c42. Status: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
@ 0x7f93c11d24de google::LogMessage::Flush() at ??:0
@ 0x7f93c11d26d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f93d6d70ba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260926 15:02:43.224581 9391 heartbeater.cc:503] Master 127.8.138.190:44281 was elected leader, sending a full tablet report...
I20260926 15:02:43.226699 9677 heartbeater.cc:347] Connected to a master server at 127.8.138.188:45473
I20260926 15:02:43.226894 9677 heartbeater.cc:503] Master 127.8.138.188:45473 was elected leader, sending a full tablet report...
@ 0x7f93d6d6f901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f93d6d7666c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f93d6d764e9 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f93c3dfceaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f93c70d66db start_thread at ??:0
@ 0x7f93c006c61f clone at ??:0
I20260926 15:02:43.276150 9532 heartbeater.cc:347] Connected to a master server at 127.8.138.188:45473
I20260926 15:02:43.276211 10121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6315fb5099e1420b95b8d83b3e19375c. Latest consensus state: current_term: 6 leader_uuid: "6315fb5099e1420b95b8d83b3e19375c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:43.276510 10121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:02:43.278226 10082 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" instance_seqno: 1790434957773527) as {username='slave'} at 127.8.138.131:39465; Asking this server to re-register.
I20260926 15:02:43.278492 10081 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" instance_seqno: 1790434956879292) as {username='slave'} at 127.8.138.130:40587; Asking this server to re-register.
I20260926 15:02:43.279291 9532 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:43.279393 9677 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:43.280964 10081 ts_manager.cc:194] Registered new tserver with Master: f052d5cbee65406593d8e44f74225bc2 (127.8.138.131:35445)
I20260926 15:02:43.282266 10082 ts_manager.cc:194] Registered new tserver with Master: ae330d635b1848aaaecee2c7ce96520a (127.8.138.130:44309)
W20260926 15:02:43.450609 10068 connection.cc:535] server connection from 127.0.0.1:48262 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 15:02:43.451296 9687 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.8.138.190:44281)
W20260926 15:02:43.451645 9687 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.8.138.190:44281)
W20260926 15:02:43.451934 9687 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.8.138.190:44281)
W20260926 15:02:43.453011 9687 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.8.138.190:44281)
I20260926 15:02:43.688117 10121 raft_consensus.cc:482] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6315fb5099e1420b95b8d83b3e19375c)
I20260926 15:02:43.688364 10121 raft_consensus.cc:504] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:43.690124 10121 leader_election.cc:288] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105)
W20260926 15:02:43.691313 10066 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
W20260926 15:02:43.692983 10069 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
W20260926 15:02:43.693208 10066 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281): Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
I20260926 15:02:43.693356 10066 leader_election.cc:302] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f; no voters: 6315fb5099e1420b95b8d83b3e19375c, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:02:43.693687 10121 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 15:02:44.376543 10188 raft_consensus.cc:482] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6315fb5099e1420b95b8d83b3e19375c)
I20260926 15:02:44.376785 10188 raft_consensus.cc:504] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:44.377486 10188 leader_election.cc:288] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105)
W20260926 15:02:44.380155 10069 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
W20260926 15:02:44.380460 10066 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281): Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
I20260926 15:02:44.380589 10066 leader_election.cc:302] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f; no voters: 6315fb5099e1420b95b8d83b3e19375c, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:02:44.380878 10188 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 15:02:45.027662 8746 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu pid 10036
I20260926 15:02:45.040220 10112 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:02:45.155874 8746 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu with pid 10036
I20260926 15:02:46.169186 8746 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu
/tmp/dist-test-taskLGH8i2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.138.188:45473
--webserver_interface=127.8.138.188
--webserver_port=38457
--builtin_ntp_servers=127.8.138.148:33835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473
--hive_metastore_uris=thrift://127.0.0.1:40047
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260926 15:02:46.516173 10211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:46.516505 10211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:46.516635 10211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:46.524367 10211 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:02:46.524487 10211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:46.524576 10211 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:02:46.524663 10211 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:02:46.534924 10211 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.138.148:33835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:40047
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.138.188:45473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.138.188
--webserver_port=38457
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:46.536473 10211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:46.538241 10211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:46.551911 10218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:46.551820 10219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:46.552134 10221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:46.552745 10211 server_base.cc:1034] running on GCE node
I20260926 15:02:46.553682 10211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:46.555258 10211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:46.556514 10211 hybrid_clock.cc:648] HybridClock initialized: now 1790434966556449 us; error 41 us; skew 500 ppm
I20260926 15:02:46.556905 10211 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:02:46.559644 10211 webserver.cc:460] Webserver started at http://127.8.138.188:38457/ using document root <none> and password file <none>
I20260926 15:02:46.560458 10211 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:46.560604 10211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:46.566576 10211 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 15:02:46.570088 10227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:46.571635 10211 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 15:02:46.571827 10211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data,/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/wal
uuid: "320cd09624624312bf1bcc0856c2874f"
format_stamp: "Formatted at 2026-09-26 15:02:35 on dist-test-slave-7qf4"
I20260926 15:02:46.572937 10211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:46.606860 10211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:46.607811 10211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:46.608240 10211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:46.627588 10211 rpc_server.cc:307] RPC server started. Bound to: 127.8.138.188:45473
I20260926 15:02:46.627610 10279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.138.188:45473 every 8 connection(s)
I20260926 15:02:46.629377 10211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data/info.pb
I20260926 15:02:46.636484 10280 sys_catalog.cc:263] Verifying existing consensus state
I20260926 15:02:46.639014 8746 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu as pid 10211
I20260926 15:02:46.640420 10280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: Bootstrap starting.
I20260926 15:02:46.654521 10280 log.cc:824] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: Log is configured to *not* fsync() on all Append() calls
I20260926 15:02:46.667553 9532 heartbeater.cc:347] Connected to a master server at 127.8.138.188:45473
I20260926 15:02:46.694309 10280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: Bootstrap replayed 1/1 log segments. Stats: ops{read=45 overwritten=0 applied=40 ignored=0} inserts{seen=27 ignored=0} mutations{seen=26 ignored=0} orphaned_commits=0. Pending: 5 replicates
I20260926 15:02:46.695040 10280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: Bootstrap complete.
I20260926 15:02:46.703481 10280 raft_consensus.cc:348] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 6 FOLLOWER]: Replica starting. Triggering 5 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:46.708264 10280 raft_consensus.cc:729] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: 320cd09624624312bf1bcc0856c2874f, State: Initialized, Role: FOLLOWER
I20260926 15:02:46.709055 10280 consensus_queue.cc:260] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 40, Last appended: 6.45, Last appended by leader: 45, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:46.709937 10285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:46.710421 10285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:02:46.711176 10280 sys_catalog.cc:564] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:02:46.725313 10299 catalog_manager.cc:1259] Loaded cluster ID: 497ed160bc39477e99bf3beb637a4391
I20260926 15:02:46.725481 10299 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: loading cluster ID for follower catalog manager: success
I20260926 15:02:46.731159 10299 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: acquiring CA information for follower catalog manager: success
I20260926 15:02:46.736443 10299 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 15:02:46.733196 10211 master_runner.cc:186] Error getting master registration for 127.8.138.190:44281: Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111),
I20260926 15:02:46.740995 10211 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:02:46.773784 9677 heartbeater.cc:347] Connected to a master server at 127.8.138.188:45473
I20260926 15:02:46.777208 10245 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" instance_seqno: 1790434957773527) as {username='slave'} at 127.8.138.131:35529; Asking this server to re-register.
I20260926 15:02:46.778008 9677 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:46.780316 10245 ts_manager.cc:194] Registered new tserver with Master: f052d5cbee65406593d8e44f74225bc2 (127.8.138.131:35445)
I20260926 15:02:46.970024 9390 heartbeater.cc:347] Connected to a master server at 127.8.138.188:45473
I20260926 15:02:46.972229 10245 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" instance_seqno: 1790434956216737) as {username='slave'} at 127.8.138.129:35059; Asking this server to re-register.
I20260926 15:02:46.972944 9390 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:46.974503 10245 ts_manager.cc:194] Registered new tserver with Master: 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327)
I20260926 15:02:47.103402 10245 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" instance_seqno: 1790434956879292) as {username='slave'} at 127.8.138.130:53791; Asking this server to re-register.
I20260926 15:02:47.104116 9532 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:47.105865 10245 ts_manager.cc:194] Registered new tserver with Master: ae330d635b1848aaaecee2c7ce96520a (127.8.138.130:44309)
I20260926 15:02:47.284725 10307 raft_consensus.cc:482] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:02:47.284996 10307 raft_consensus.cc:504] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:47.287034 10307 leader_election.cc:288] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105)
W20260926 15:02:47.288060 10228 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
W20260926 15:02:47.289547 10228 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281): Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
W20260926 15:02:47.289819 10231 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:02:47.290030 10231 leader_election.cc:302] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f; no voters: 6315fb5099e1420b95b8d83b3e19375c, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:02:47.290405 10307 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 15:02:48.004536 10317 raft_consensus.cc:482] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:02:48.004760 10317 raft_consensus.cc:504] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:48.005577 10317 leader_election.cc:288] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105)
W20260926 15:02:48.007922 10228 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281): Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
W20260926 15:02:48.008095 10231 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:02:48.008211 10231 leader_election.cc:302] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f; no voters: 6315fb5099e1420b95b8d83b3e19375c, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:02:48.008476 10317 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 15:02:48.886462 10332 raft_consensus.cc:482] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:02:48.886677 10332 raft_consensus.cc:504] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:48.887378 10332 leader_election.cc:288] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105)
W20260926 15:02:48.889667 10231 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
W20260926 15:02:48.889931 10228 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281): Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
I20260926 15:02:48.890128 10228 leader_election.cc:302] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f; no voters: 6315fb5099e1420b95b8d83b3e19375c, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:02:48.890519 10332 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 15:02:50.077616 8746 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu
/tmp/dist-test-taskLGH8i2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.138.189:37105
--webserver_interface=127.8.138.189
--webserver_port=40173
--builtin_ntp_servers=127.8.138.148:33835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473
--hive_metastore_uris=thrift://127.0.0.1:40047
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260926 15:02:50.422531 10345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:50.422839 10345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:50.422981 10345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:50.430872 10345 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:02:50.430984 10345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:50.431072 10345 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:02:50.431171 10345 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:02:50.441164 10345 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.138.148:33835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:40047
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.138.189:37105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.138.189
--webserver_port=40173
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:50.442602 10345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:50.444437 10345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:50.457643 10355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:50.457702 10357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:50.457726 10354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:50.459324 10345 server_base.cc:1034] running on GCE node
I20260926 15:02:50.460279 10345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:50.461792 10345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:50.463047 10345 hybrid_clock.cc:648] HybridClock initialized: now 1790434970462976 us; error 39 us; skew 500 ppm
I20260926 15:02:50.463443 10345 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:02:50.473426 10345 webserver.cc:460] Webserver started at http://127.8.138.189:40173/ using document root <none> and password file <none>
I20260926 15:02:50.474218 10345 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:50.474350 10345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:50.480427 10345 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260926 15:02:50.484133 10363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:50.485777 10345 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:02:50.486011 10345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/data,/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/wal
uuid: "be9958f2085242f7ab82ae933ceb1c42"
format_stamp: "Formatted at 2026-09-26 15:02:34 on dist-test-slave-7qf4"
I20260926 15:02:50.487114 10345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:50.515514 10345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:50.516680 10345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:50.516986 10345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:50.536324 10345 rpc_server.cc:307] RPC server started. Bound to: 127.8.138.189:37105
I20260926 15:02:50.536338 10415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.138.189:37105 every 8 connection(s)
I20260926 15:02:50.537835 10345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/data/info.pb
I20260926 15:02:50.538514 10416 raft_consensus.cc:482] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:02:50.538754 10416 raft_consensus.cc:504] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:50.539448 10416 leader_election.cc:288] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105)
I20260926 15:02:50.547417 8746 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu as pid 10345
I20260926 15:02:50.555065 10417 sys_catalog.cc:263] Verifying existing consensus state
I20260926 15:02:50.560465 10417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42: Bootstrap starting.
W20260926 15:02:50.563755 10228 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281): Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
I20260926 15:02:50.575759 10417 log.cc:824] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42: Log is configured to *not* fsync() on all Append() calls
I20260926 15:02:50.609050 10417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42: Bootstrap replayed 1/1 log segments. Stats: ops{read=34 overwritten=0 applied=34 ignored=0} inserts{seen=24 ignored=0} mutations{seen=20 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 15:02:50.609648 10417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42: Bootstrap complete.
I20260926 15:02:50.621112 10417 raft_consensus.cc:348] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:50.622545 10417 raft_consensus.cc:729] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9958f2085242f7ab82ae933ceb1c42, State: Initialized, Role: FOLLOWER
I20260926 15:02:50.623353 10417 consensus_queue.cc:260] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 34, Last appended: 4.34, Last appended by leader: 34, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:50.623968 9534 heartbeater.cc:347] Connected to a master server at 127.8.138.189:37105
I20260926 15:02:50.625034 10427 sys_catalog.cc:455] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:50.625604 10427 sys_catalog.cc:458] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:02:50.626113 10417 sys_catalog.cc:564] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:02:50.626981 10391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "320cd09624624312bf1bcc0856c2874f" candidate_term: 7 candidate_status { last_received { term: 6 index: 45 } } ignore_live_leader: false dest_uuid: "be9958f2085242f7ab82ae933ceb1c42" is_pre_election: true
I20260926 15:02:50.627712 10391 raft_consensus.cc:2454] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 320cd09624624312bf1bcc0856c2874f in term 4.
I20260926 15:02:50.628958 10231 leader_election.cc:302] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42; no voters: 6315fb5099e1420b95b8d83b3e19375c
I20260926 15:02:50.629436 10416 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260926 15:02:50.629552 10416 raft_consensus.cc:482] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:02:50.629644 10416 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 6 FOLLOWER]: Advancing to term 7
I20260926 15:02:50.634850 10416 raft_consensus.cc:504] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:50.636377 10416 leader_election.cc:288] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 7 election: Requested vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105)
I20260926 15:02:50.636646 10391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "320cd09624624312bf1bcc0856c2874f" candidate_term: 7 candidate_status { last_received { term: 6 index: 45 } } ignore_live_leader: false dest_uuid: "be9958f2085242f7ab82ae933ceb1c42"
I20260926 15:02:50.636948 10391 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 4 FOLLOWER]: Advancing to term 7
W20260926 15:02:50.640801 10228 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281): Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
I20260926 15:02:50.642540 10391 raft_consensus.cc:2454] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 320cd09624624312bf1bcc0856c2874f in term 7.
I20260926 15:02:50.643100 10231 leader_election.cc:302] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42; no voters: 6315fb5099e1420b95b8d83b3e19375c
I20260926 15:02:50.643512 10416 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 7 FOLLOWER]: Leader election won for term 7
I20260926 15:02:50.644131 10416 raft_consensus.cc:686] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 7 LEADER]: Becoming Leader. State: Replica: 320cd09624624312bf1bcc0856c2874f, State: Running, Role: LEADER
I20260926 15:02:50.644718 10416 consensus_queue.cc:237] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 40, Committed index: 40, Last appended: 6.45, Last appended by leader: 45, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:50.647606 10438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 320cd09624624312bf1bcc0856c2874f. Latest consensus state: current_term: 7 leader_uuid: "320cd09624624312bf1bcc0856c2874f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:50.648073 10438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: This master's current role is: LEADER
I20260926 15:02:50.663689 10445 catalog_manager.cc:1259] Loaded cluster ID: 497ed160bc39477e99bf3beb637a4391
I20260926 15:02:50.663800 10445 catalog_manager.cc:1552] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42: loading cluster ID for follower catalog manager: success
I20260926 15:02:50.654958 10345 master_runner.cc:186] Error getting master registration for 127.8.138.190:44281: Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111),
I20260926 15:02:50.664531 10345 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:02:50.666685 10445 catalog_manager.cc:1574] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42: acquiring CA information for follower catalog manager: success
I20260926 15:02:50.668628 10445 catalog_manager.cc:1602] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 15:02:50.788398 9679 heartbeater.cc:347] Connected to a master server at 127.8.138.189:37105
I20260926 15:02:50.792590 10381 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" instance_seqno: 1790434957773527) as {username='slave'} at 127.8.138.131:47041; Asking this server to re-register.
I20260926 15:02:50.793452 9679 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:50.795452 10381 ts_manager.cc:194] Registered new tserver with Master: f052d5cbee65406593d8e44f74225bc2 (127.8.138.131:35445)
I20260926 15:02:50.979903 9394 heartbeater.cc:347] Connected to a master server at 127.8.138.189:37105
I20260926 15:02:50.981902 10381 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" instance_seqno: 1790434956216737) as {username='slave'} at 127.8.138.129:58619; Asking this server to re-register.
I20260926 15:02:50.982592 9394 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:50.984259 10381 ts_manager.cc:194] Registered new tserver with Master: 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327)
I20260926 15:02:51.128716 9534 heartbeater.cc:347] Connected to a master server at 127.8.138.189:37105
I20260926 15:02:51.131013 10381 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" instance_seqno: 1790434956879292) as {username='slave'} at 127.8.138.130:35723; Asking this server to re-register.
I20260926 15:02:51.131688 9534 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:51.133180 10381 ts_manager.cc:194] Registered new tserver with Master: ae330d635b1848aaaecee2c7ce96520a (127.8.138.130:44309)
I20260926 15:02:51.135263 10391 raft_consensus.cc:1261] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 7 FOLLOWER]: Refusing update from remote peer 320cd09624624312bf1bcc0856c2874f: Log matching property violated. Preceding OpId in replica: term: 4 index: 34. Preceding OpId from leader: term: 7 index: 46. (index mismatch)
I20260926 15:02:51.136358 10438 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 46, Last known committed idx: 34, Time since last communication: 0.000s
I20260926 15:02:51.146873 10452 sys_catalog.cc:455] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 320cd09624624312bf1bcc0856c2874f. Latest consensus state: current_term: 7 leader_uuid: "320cd09624624312bf1bcc0856c2874f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:51.147271 10452 sys_catalog.cc:458] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:02:51.147001 10455 sys_catalog.cc:455] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "320cd09624624312bf1bcc0856c2874f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:51.147410 10455 sys_catalog.cc:458] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:02:51.150202 10452 sys_catalog.cc:455] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "320cd09624624312bf1bcc0856c2874f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:51.150177 10455 sys_catalog.cc:455] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "320cd09624624312bf1bcc0856c2874f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:51.150534 10452 sys_catalog.cc:458] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:02:51.150560 10455 sys_catalog.cc:458] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:02:51.159538 10458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 7 leader_uuid: "320cd09624624312bf1bcc0856c2874f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:51.159901 10458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: This master's current role is: LEADER
I20260926 15:02:51.159934 10438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "320cd09624624312bf1bcc0856c2874f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:51.160153 10438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: This master's current role is: LEADER
I20260926 15:02:51.161361 10453 mvcc.cc:205] Tried to move back new op lower bound from 7333621609201983488 to 7333621601781800960. Current Snapshot: MvccSnapshot[applied={T|T < 7333621609201983488}]
W20260926 15:02:51.162343 10228 consensus_peers.cc:487] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f -> Peer 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281): Couldn't send request to peer 6315fb5099e1420b95b8d83b3e19375c. Status: Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 15:02:51.169927 10444 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:02:51.172796 10444 catalog_manager.cc:669] Loaded metadata for table default.table_43176f08d9e24974abdd53171465a15c [id=08b331ce5f9c44389fae209bac801f42]
I20260926 15:02:51.173288 10444 catalog_manager.cc:669] Loaded metadata for table default.table_854219551f9048c6925c711ce33d5256 [id=3295fc8a0a004843be045b8bf6a4f2cf]
I20260926 15:02:51.173723 10444 catalog_manager.cc:669] Loaded metadata for table default.table_96be20374f7e43e7a865549e39bd64f6 [id=51dea274f4bf4903baaddbbd8c257b27]
I20260926 15:02:51.174154 10444 catalog_manager.cc:669] Loaded metadata for table default.table_d2243274908043e58210a197141af813 [id=5357ddcf21634f99bb4f9abc5b6c4d80]
I20260926 15:02:51.174571 10444 catalog_manager.cc:669] Loaded metadata for table default.table_95f48725b6ce47fab5f96e18ccb69dbc [id=630deebc15b04878ae13844ff2808f52]
I20260926 15:02:51.175071 10444 catalog_manager.cc:669] Loaded metadata for table default.table_22fd4957e3d14964803428457f8490ac [id=7ba7b39b84394421b88a6ce7616c3c91]
I20260926 15:02:51.175493 10444 catalog_manager.cc:669] Loaded metadata for table default.table_41a3ce3c89684cbb9054c4251a0cc51c [id=7cc0c844d49c4ff08cef9e2c410991f8]
I20260926 15:02:51.175904 10444 catalog_manager.cc:669] Loaded metadata for table default.table_7b56039466d14dcab4dc63192de3b08e [id=b84ce6d15e4246dbbb299468138add55]
I20260926 15:02:51.176241 10444 catalog_manager.cc:669] Loaded metadata for table default.table_e3b713aac3c94e449bb5882310d55a40 [id=e9e378126d2649dab722c82f99d654b7]
I20260926 15:02:51.178143 10444 tablet_loader.cc:96] loaded metadata for tablet 0d837276d648440cbebe1f88c281cb11 (table default.table_96be20374f7e43e7a865549e39bd64f6 [id=51dea274f4bf4903baaddbbd8c257b27])
I20260926 15:02:51.178427 10444 tablet_loader.cc:96] loaded metadata for tablet 12efb00aa1c649729d8a926b784e69c0 (table default.table_22fd4957e3d14964803428457f8490ac [id=7ba7b39b84394421b88a6ce7616c3c91])
I20260926 15:02:51.178752 10444 tablet_loader.cc:96] loaded metadata for tablet 252e506a97564739bc26268653efa864 (table default.table_e3b713aac3c94e449bb5882310d55a40 [id=e9e378126d2649dab722c82f99d654b7])
I20260926 15:02:51.179245 10444 tablet_loader.cc:96] loaded metadata for tablet 3fae8a76002849de9bb26610a0c33406 (table default.table_95f48725b6ce47fab5f96e18ccb69dbc [id=630deebc15b04878ae13844ff2808f52])
I20260926 15:02:51.179469 10444 tablet_loader.cc:96] loaded metadata for tablet 41e641e28d1f48b2a7e0c8fb8964f678 (table default.table_d2243274908043e58210a197141af813 [id=5357ddcf21634f99bb4f9abc5b6c4d80])
I20260926 15:02:51.180238 10444 tablet_loader.cc:96] loaded metadata for tablet 86fed5bdf8b7474dbff0c53d6d4d673b (table default.table_7b56039466d14dcab4dc63192de3b08e [id=b84ce6d15e4246dbbb299468138add55])
I20260926 15:02:51.180567 10444 tablet_loader.cc:96] loaded metadata for tablet a6a3a867e477463491cc7a24df6faa5d (table default.table_854219551f9048c6925c711ce33d5256 [id=3295fc8a0a004843be045b8bf6a4f2cf])
I20260926 15:02:51.180913 10444 tablet_loader.cc:96] loaded metadata for tablet c5d07d3a7a164d098e61f8f7f2ac4a67 (table default.table_43176f08d9e24974abdd53171465a15c [id=08b331ce5f9c44389fae209bac801f42])
I20260926 15:02:51.181357 10444 tablet_loader.cc:96] loaded metadata for tablet df8efd9136304dc09f153187a9e55cb1 (table default.table_41a3ce3c89684cbb9054c4251a0cc51c [id=7cc0c844d49c4ff08cef9e2c410991f8])
I20260926 15:02:51.181567 10444 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:02:51.182569 10444 catalog_manager.cc:1259] Loaded cluster ID: 497ed160bc39477e99bf3beb637a4391
I20260926 15:02:51.182680 10444 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:02:51.184015 10444 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:02:51.184938 10444 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: Loaded TSK: 0
I20260926 15:02:51.185763 10444 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:02:51.186800 10444 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260926 15:02:51.187819 10444 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260926 15:02:51.280349 10245 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48456:
table { table_name: "default.Table_e3b713aac3c94e449bb5882310d55a40" } new_table_name: "default.Table_fd56d79387ed41638b65164801eaa9ab" modify_external_catalogs: true
I20260926 15:02:51.280529 10244 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48456:
table { table_name: "default.Table_7b56039466d14dcab4dc63192de3b08e" } modify_external_catalogs: true
I20260926 15:02:51.280884 10244 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48456:
table { table_name: "default.Table_7b56039466d14dcab4dc63192de3b08e" } modify_external_catalogs: true
I20260926 15:02:51.283025 10236 master_service.cc:946] ReplaceTablet: received request to replace tablet c8d090f2e0384924a72f288eff2281e7 from {username='slave'} at 127.0.0.1:48456
I20260926 15:02:51.282763 10237 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48456:
name: "default.Table_985d0971fb8a4d8eae0b2cb73ca99c6f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:51.283098 10240 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48456:
table { table_name: "default.Table_96be20374f7e43e7a865549e39bd64f6" } modify_external_catalogs: true
I20260926 15:02:51.283437 10240 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48456:
table { table_name: "default.Table_96be20374f7e43e7a865549e39bd64f6" } modify_external_catalogs: true
I20260926 15:02:51.283502 10238 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48456:
name: "default.Table_6c170e0fd90e42f08500c2a89216a4d3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:51.283979 10239 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48456:
name: "default.Table_5bfdd1b3c4a247d58d08b75039b6124f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:51.284040 10242 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48456:
name: "default.Table_7257407810154f2aa325ed506a8a62d5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:51.284469 10241 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48456:
name: "default.Table_aa60eda53a3f4f338866d309cbde44a9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:51.284499 10243 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48456:
name: "default.Table_04fd049074c940dba0495d97eac1d272"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:51.284780 10236 master_service.cc:946] ReplaceTablet: received request to replace tablet 7fa51439f5324161991e8dbbd169851e from {username='slave'} at 127.0.0.1:48456
I20260926 15:02:51.286168 10236 master_service.cc:946] ReplaceTablet: received request to replace tablet 51ff1e9e47864b24b7c7a869ff37e06a from {username='slave'} at 127.0.0.1:48456
I20260926 15:02:51.286641 10236 master_service.cc:946] ReplaceTablet: received request to replace tablet d4c76c4e1678474c92af007a7b5f7596 from {username='slave'} at 127.0.0.1:48456
I20260926 15:02:51.287114 10236 master_service.cc:946] ReplaceTablet: received request to replace tablet 51ff1e9e47864b24b7c7a869ff37e06a from {username='slave'} at 127.0.0.1:48456
I20260926 15:02:51.287550 10236 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48456:
table { table_name: "default.Table_43176f08d9e24974abdd53171465a15c" } new_table_name: "default.Table_1b2fcfff41b54f21b229a3e2064d7c26" modify_external_catalogs: true
F20260926 15:02:51.288002 10298 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_f356bc0dfc6a47f98430002dccbb873f
*** Check failure stack trace: ***
@ 0x7fa8e22179ad google::LogMessage::Fail() at ??:0
@ 0x7fa8e221a88a google::LogMessage::SendToLog() at ??:0
@ 0x7fa8e22174de google::LogMessage::Flush() at ??:0
@ 0x7fa8e22176d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7fa8f7db5ba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7fa8f7db4901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7fa8f7dbb66c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7fa8f7dbb4e9 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fa8e4e41eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fa8e811b6db start_thread at ??:0
@ 0x7fa8e10b161f clone at ??:0
W20260926 15:02:51.513298 9689 connection.cc:535] client connection to 127.8.138.188:45473 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 15:02:51.513334 9280 connection.cc:535] client connection to 127.8.138.188:45473 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 15:02:51.514086 9689 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.8.138.188:45473)
W20260926 15:02:51.514822 9689 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.8.138.188:45473)
W20260926 15:02:51.515067 9689 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.8.138.188:45473)
W20260926 15:02:51.515612 9689 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.8.138.188:45473)
I20260926 15:02:51.649412 10455 raft_consensus.cc:482] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:02:51.649662 10455 raft_consensus.cc:504] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:51.651587 10455 leader_election.cc:288] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:02:51.652616 10365 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
W20260926 15:02:51.654516 10367 leader_election.cc:334] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
W20260926 15:02:51.654743 10365 leader_election.cc:334] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281): Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
I20260926 15:02:51.654966 10365 leader_election.cc:302] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9958f2085242f7ab82ae933ceb1c42; no voters: 320cd09624624312bf1bcc0856c2874f, 6315fb5099e1420b95b8d83b3e19375c
I20260926 15:02:51.655356 10455 raft_consensus.cc:2726] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260926 15:02:52.231278 10489 raft_consensus.cc:482] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:02:52.231495 10489 raft_consensus.cc:504] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:52.232381 10489 leader_election.cc:288] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:02:52.234705 10367 leader_election.cc:334] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
W20260926 15:02:52.234874 10365 leader_election.cc:334] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281): Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
I20260926 15:02:52.235009 10365 leader_election.cc:302] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9958f2085242f7ab82ae933ceb1c42; no voters: 320cd09624624312bf1bcc0856c2874f, 6315fb5099e1420b95b8d83b3e19375c
I20260926 15:02:52.235234 10489 raft_consensus.cc:2726] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260926 15:02:52.973726 10494 raft_consensus.cc:482] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:02:52.973918 10494 raft_consensus.cc:504] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:52.974606 10494 leader_election.cc:288] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:02:52.976961 10365 leader_election.cc:334] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281): Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
W20260926 15:02:52.977154 10367 leader_election.cc:334] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
I20260926 15:02:52.977311 10367 leader_election.cc:302] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9958f2085242f7ab82ae933ceb1c42; no voters: 320cd09624624312bf1bcc0856c2874f, 6315fb5099e1420b95b8d83b3e19375c
I20260926 15:02:52.977600 10494 raft_consensus.cc:2726] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260926 15:02:53.651177 8746 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu
/tmp/dist-test-taskLGH8i2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.138.188:45473
--webserver_interface=127.8.138.188
--webserver_port=38457
--builtin_ntp_servers=127.8.138.148:33835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473
--hive_metastore_uris=thrift://127.0.0.1:40047
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260926 15:02:53.828511 10501 raft_consensus.cc:482] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:02:53.828745 10501 raft_consensus.cc:504] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:53.829762 10501 leader_election.cc:288] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:02:53.832401 10365 leader_election.cc:334] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281): Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
W20260926 15:02:53.832665 10367 leader_election.cc:334] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
I20260926 15:02:53.832799 10367 leader_election.cc:302] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be9958f2085242f7ab82ae933ceb1c42; no voters: 320cd09624624312bf1bcc0856c2874f, 6315fb5099e1420b95b8d83b3e19375c
I20260926 15:02:53.833087 10501 raft_consensus.cc:2726] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260926 15:02:54.019630 10500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:54.020085 10500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:54.020347 10500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:54.029839 10500 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:02:54.030025 10500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:54.030114 10500 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:02:54.030220 10500 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:02:54.040694 10500 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.138.148:33835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:40047
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.138.188:45473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.138.188
--webserver_port=38457
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:54.042196 10500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:54.044016 10500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:54.057709 10510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:54.057648 10509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:54.057672 10512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:54.058652 10500 server_base.cc:1034] running on GCE node
I20260926 15:02:54.059664 10500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:54.061303 10500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:54.062572 10500 hybrid_clock.cc:648] HybridClock initialized: now 1790434974062482 us; error 70 us; skew 500 ppm
I20260926 15:02:54.062933 10500 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:02:54.065553 10500 webserver.cc:460] Webserver started at http://127.8.138.188:38457/ using document root <none> and password file <none>
I20260926 15:02:54.066286 10500 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:54.066448 10500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:54.072204 10500 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 15:02:54.075623 10518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:54.077353 10500 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:02:54.077577 10500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data,/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/wal
uuid: "320cd09624624312bf1bcc0856c2874f"
format_stamp: "Formatted at 2026-09-26 15:02:35 on dist-test-slave-7qf4"
I20260926 15:02:54.078728 10500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:54.095547 10500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:54.096575 10500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:54.096887 10500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:54.116554 10500 rpc_server.cc:307] RPC server started. Bound to: 127.8.138.188:45473
I20260926 15:02:54.116593 10569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.138.188:45473 every 8 connection(s)
I20260926 15:02:54.118206 10500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data/info.pb
I20260926 15:02:54.123441 8746 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu as pid 10500
I20260926 15:02:54.126641 10570 sys_catalog.cc:263] Verifying existing consensus state
I20260926 15:02:54.131414 10570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: Bootstrap starting.
I20260926 15:02:54.142441 9532 heartbeater.cc:347] Connected to a master server at 127.8.138.188:45473
I20260926 15:02:54.147295 10570 log.cc:824] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: Log is configured to *not* fsync() on all Append() calls
I20260926 15:02:54.188802 10570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: Bootstrap replayed 1/1 log segments. Stats: ops{read=46 overwritten=0 applied=46 ignored=0} inserts{seen=31 ignored=0} mutations{seen=30 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 15:02:54.189427 10570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: Bootstrap complete.
I20260926 15:02:54.199091 10570 raft_consensus.cc:348] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 7 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:54.200208 10570 raft_consensus.cc:729] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 7 FOLLOWER]: Becoming Follower/Learner. State: Replica: 320cd09624624312bf1bcc0856c2874f, State: Initialized, Role: FOLLOWER
I20260926 15:02:54.200887 10570 consensus_queue.cc:260] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 46, Last appended: 7.46, Last appended by leader: 46, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:54.201731 10575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 7 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:54.202188 10575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:02:54.202873 10570 sys_catalog.cc:564] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:02:54.222970 10589 catalog_manager.cc:1259] Loaded cluster ID: 497ed160bc39477e99bf3beb637a4391
I20260926 15:02:54.223207 10589 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: loading cluster ID for follower catalog manager: success
I20260926 15:02:54.216341 10500 master_runner.cc:186] Error getting master registration for 127.8.138.190:44281: Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111),
I20260926 15:02:54.228207 10500 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:02:54.232924 10589 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: acquiring CA information for follower catalog manager: success
I20260926 15:02:54.236095 10589 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 15:02:54.503151 10594 raft_consensus.cc:482] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:02:54.503409 10594 raft_consensus.cc:504] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:54.504384 10594 leader_election.cc:288] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
I20260926 15:02:54.533704 9677 heartbeater.cc:347] Connected to a master server at 127.8.138.188:45473
I20260926 15:02:54.537398 10535 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" instance_seqno: 1790434957773527) as {username='slave'} at 127.8.138.131:37235; Asking this server to re-register.
I20260926 15:02:54.545073 9390 heartbeater.cc:347] Connected to a master server at 127.8.138.188:45473
I20260926 15:02:54.544936 10545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "be9958f2085242f7ab82ae933ceb1c42" candidate_term: 8 candidate_status { last_received { term: 7 index: 46 } } ignore_live_leader: false dest_uuid: "320cd09624624312bf1bcc0856c2874f" is_pre_election: true
I20260926 15:02:54.545760 10545 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9958f2085242f7ab82ae933ceb1c42 in term 7.
I20260926 15:02:54.546536 9677 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:54.546844 10534 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" instance_seqno: 1790434956216737) as {username='slave'} at 127.8.138.129:59651; Asking this server to re-register.
I20260926 15:02:54.547446 9390 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:54.548220 10367 leader_election.cc:302] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42; no voters:
I20260926 15:02:54.548616 10594 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260926 15:02:54.548754 10594 raft_consensus.cc:482] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:02:54.548852 10594 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 7 FOLLOWER]: Advancing to term 8
I20260926 15:02:54.550880 10534 ts_manager.cc:194] Registered new tserver with Master: 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327)
I20260926 15:02:54.552016 10594 raft_consensus.cc:504] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:54.553659 10545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "be9958f2085242f7ab82ae933ceb1c42" candidate_term: 8 candidate_status { last_received { term: 7 index: 46 } } ignore_live_leader: false dest_uuid: "320cd09624624312bf1bcc0856c2874f"
I20260926 15:02:54.553957 10545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 7 FOLLOWER]: Advancing to term 8
I20260926 15:02:54.557258 10594 leader_election.cc:288] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 8 election: Requested vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:02:54.561687 10365 leader_election.cc:334] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281): Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
I20260926 15:02:54.562186 10545 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9958f2085242f7ab82ae933ceb1c42 in term 8.
I20260926 15:02:54.563719 10535 ts_manager.cc:194] Registered new tserver with Master: f052d5cbee65406593d8e44f74225bc2 (127.8.138.131:35445)
I20260926 15:02:54.565019 10367 leader_election.cc:302] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42; no voters:
I20260926 15:02:54.565402 10594 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 8 FOLLOWER]: Leader election won for term 8
I20260926 15:02:54.570222 10594 raft_consensus.cc:686] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 8 LEADER]: Becoming Leader. State: Replica: be9958f2085242f7ab82ae933ceb1c42, State: Running, Role: LEADER
W20260926 15:02:54.570294 10365 leader_election.cc:334] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281): Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
I20260926 15:02:54.571142 10594 consensus_queue.cc:237] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 46, Committed index: 46, Last appended: 7.46, Last appended by leader: 46, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:54.575340 10599 sys_catalog.cc:455] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be9958f2085242f7ab82ae933ceb1c42. Latest consensus state: current_term: 8 leader_uuid: "be9958f2085242f7ab82ae933ceb1c42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:54.575883 10599 sys_catalog.cc:458] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [sys.catalog]: This master's current role is: LEADER
I20260926 15:02:54.576704 10601 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:02:54.580199 10601 catalog_manager.cc:669] Loaded metadata for table default.table_43176f08d9e24974abdd53171465a15c [id=08b331ce5f9c44389fae209bac801f42]
I20260926 15:02:54.581434 10601 catalog_manager.cc:669] Loaded metadata for table default.table_854219551f9048c6925c711ce33d5256 [id=3295fc8a0a004843be045b8bf6a4f2cf]
I20260926 15:02:54.581892 10601 catalog_manager.cc:669] Loaded metadata for table default.table_96be20374f7e43e7a865549e39bd64f6 [id=51dea274f4bf4903baaddbbd8c257b27]
I20260926 15:02:54.582343 10601 catalog_manager.cc:669] Loaded metadata for table default.table_d2243274908043e58210a197141af813 [id=5357ddcf21634f99bb4f9abc5b6c4d80]
I20260926 15:02:54.582749 10601 catalog_manager.cc:669] Loaded metadata for table default.table_95f48725b6ce47fab5f96e18ccb69dbc [id=630deebc15b04878ae13844ff2808f52]
I20260926 15:02:54.583168 10601 catalog_manager.cc:669] Loaded metadata for table default.table_22fd4957e3d14964803428457f8490ac [id=7ba7b39b84394421b88a6ce7616c3c91]
I20260926 15:02:54.583580 10601 catalog_manager.cc:669] Loaded metadata for table default.table_41a3ce3c89684cbb9054c4251a0cc51c [id=7cc0c844d49c4ff08cef9e2c410991f8]
I20260926 15:02:54.584020 10601 catalog_manager.cc:669] Loaded metadata for table default.table_7b56039466d14dcab4dc63192de3b08e [id=b84ce6d15e4246dbbb299468138add55]
I20260926 15:02:54.584445 10601 catalog_manager.cc:669] Loaded metadata for table default.table_e3b713aac3c94e449bb5882310d55a40 [id=e9e378126d2649dab722c82f99d654b7]
I20260926 15:02:54.586824 10601 tablet_loader.cc:96] loaded metadata for tablet 0d837276d648440cbebe1f88c281cb11 (table default.table_96be20374f7e43e7a865549e39bd64f6 [id=51dea274f4bf4903baaddbbd8c257b27])
I20260926 15:02:54.587158 10601 tablet_loader.cc:96] loaded metadata for tablet 12efb00aa1c649729d8a926b784e69c0 (table default.table_22fd4957e3d14964803428457f8490ac [id=7ba7b39b84394421b88a6ce7616c3c91])
I20260926 15:02:54.587531 10601 tablet_loader.cc:96] loaded metadata for tablet 252e506a97564739bc26268653efa864 (table default.table_e3b713aac3c94e449bb5882310d55a40 [id=e9e378126d2649dab722c82f99d654b7])
I20260926 15:02:54.588258 10601 tablet_loader.cc:96] loaded metadata for tablet 3fae8a76002849de9bb26610a0c33406 (table default.table_95f48725b6ce47fab5f96e18ccb69dbc [id=630deebc15b04878ae13844ff2808f52])
I20260926 15:02:54.588613 10601 tablet_loader.cc:96] loaded metadata for tablet 41e641e28d1f48b2a7e0c8fb8964f678 (table default.table_d2243274908043e58210a197141af813 [id=5357ddcf21634f99bb4f9abc5b6c4d80])
I20260926 15:02:54.589761 10601 tablet_loader.cc:96] loaded metadata for tablet 86fed5bdf8b7474dbff0c53d6d4d673b (table default.table_7b56039466d14dcab4dc63192de3b08e [id=b84ce6d15e4246dbbb299468138add55])
I20260926 15:02:54.590260 10601 tablet_loader.cc:96] loaded metadata for tablet a6a3a867e477463491cc7a24df6faa5d (table default.table_854219551f9048c6925c711ce33d5256 [id=3295fc8a0a004843be045b8bf6a4f2cf])
I20260926 15:02:54.590895 10601 tablet_loader.cc:96] loaded metadata for tablet c5d07d3a7a164d098e61f8f7f2ac4a67 (table default.table_43176f08d9e24974abdd53171465a15c [id=08b331ce5f9c44389fae209bac801f42])
I20260926 15:02:54.591607 10601 tablet_loader.cc:96] loaded metadata for tablet df8efd9136304dc09f153187a9e55cb1 (table default.table_41a3ce3c89684cbb9054c4251a0cc51c [id=7cc0c844d49c4ff08cef9e2c410991f8])
I20260926 15:02:54.591941 10601 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:02:54.593442 10601 catalog_manager.cc:1259] Loaded cluster ID: 497ed160bc39477e99bf3beb637a4391
I20260926 15:02:54.593536 10601 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:02:54.595485 10601 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:02:54.596951 10601 catalog_manager.cc:5879] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42: Loaded TSK: 0
I20260926 15:02:54.598186 10601 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:02:54.599483 10601 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260926 15:02:54.600692 10601 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260926 15:02:54.627991 10380 master_service.cc:946] ReplaceTablet: received request to replace tablet 41e641e28d1f48b2a7e0c8fb8964f678 from {username='slave'} at 127.0.0.1:35210
I20260926 15:02:54.628877 10378 master_service.cc:946] ReplaceTablet: received request to replace tablet 7fa51439f5324161991e8dbbd169851e from {username='slave'} at 127.0.0.1:35210
I20260926 15:02:54.628937 10377 master_service.cc:946] ReplaceTablet: received request to replace tablet c5d07d3a7a164d098e61f8f7f2ac4a67 from {username='slave'} at 127.0.0.1:35210
I20260926 15:02:54.628034 10381 master_service.cc:946] ReplaceTablet: received request to replace tablet 0d837276d648440cbebe1f88c281cb11 from {username='slave'} at 127.0.0.1:35210
I20260926 15:02:54.630661 10374 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35210:
table { table_name: "default.Table_22fd4957e3d14964803428457f8490ac" } modify_external_catalogs: true
I20260926 15:02:54.631054 10374 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35210:
table { table_name: "default.Table_22fd4957e3d14964803428457f8490ac" } modify_external_catalogs: true
I20260926 15:02:54.631359 10379 master_service.cc:946] ReplaceTablet: received request to replace tablet d4c76c4e1678474c92af007a7b5f7596 from {username='slave'} at 127.0.0.1:35210
I20260926 15:02:54.631299 10373 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35210:
name: "default.Table_a20572133e32441ab0bfe9e3fceed04a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:54.632263 10379 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35210:
table { table_name: "default.Table_95f48725b6ce47fab5f96e18ccb69dbc" } new_table_name: "default.Table_85e820ec1f9f4f17be9385e54532213a" modify_external_catalogs: true
I20260926 15:02:54.640245 10375 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35210:
table { table_name: "default.Table_854219551f9048c6925c711ce33d5256" } modify_external_catalogs: true
I20260926 15:02:54.639710 10372 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35210:
name: "default.Table_e26e1e9fab934eab9d1b22663b28c712"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:54.640990 10376 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35210:
name: "default.Table_0e42d4f90b4e4529bb0a2ea0909cbadb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260926 15:02:54.642621 10442 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_f356bc0dfc6a47f98430002dccbb873f
*** Check failure stack trace: ***
@ 0x7fdd9e7279ad google::LogMessage::Fail() at ??:0
@ 0x7fdd9e72a88a google::LogMessage::SendToLog() at ??:0
@ 0x7fdd9e7274de google::LogMessage::Flush() at ??:0
I20260926 15:02:54.649488 10378 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35210:
table { table_name: "default.Table_d2243274908043e58210a197141af813" } new_table_name: "default.Table_42b67a58948144f5a3b3abe972e975ba" modify_external_catalogs: true
I20260926 15:02:54.649936 10535 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" instance_seqno: 1790434956879292) as {username='slave'} at 127.8.138.130:60887; Asking this server to re-register.
I20260926 15:02:54.650662 9532 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:54.651782 10375 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35210:
table { table_name: "default.Table_854219551f9048c6925c711ce33d5256" } modify_external_catalogs: true
I20260926 15:02:54.652736 10535 ts_manager.cc:194] Registered new tserver with Master: ae330d635b1848aaaecee2c7ce96520a (127.8.138.130:44309)
@ 0x7fdd9e7276d1 google::LogMessage::~LogMessage() at ??:0
I20260926 15:02:54.662410 10545 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 8 FOLLOWER]: Refusing update from remote peer be9958f2085242f7ab82ae933ceb1c42: Log matching property violated. Preceding OpId in replica: term: 7 index: 46. Preceding OpId from leader: term: 8 index: 48. (index mismatch)
W20260926 15:02:54.663115 10365 consensus_peers.cc:487] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 -> Peer 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281): Couldn't send request to peer 6315fb5099e1420b95b8d83b3e19375c. Status: Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 15:02:54.663379 10599 consensus_queue.cc:1035] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 47, Last known committed idx: 46, Time since last communication: 0.000s
@ 0x7fddb42c5ba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7fddb42c4901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7fddb42cb66c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7fddb42cb4e9 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
I20260926 15:02:54.688361 10575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: SysCatalogTable state changed. Reason: New leader be9958f2085242f7ab82ae933ceb1c42. Latest consensus state: current_term: 8 leader_uuid: "be9958f2085242f7ab82ae933ceb1c42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:54.688753 10575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: This master's current role is: FOLLOWER
@ 0x7fdda1351eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fdda462b6db start_thread at ??:0
@ 0x7fdd9d5c161f clone at ??:0
I20260926 15:02:54.697214 10575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 8 leader_uuid: "be9958f2085242f7ab82ae933ceb1c42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:54.697504 10575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: This master's current role is: FOLLOWER
W20260926 15:02:54.926744 9568 connection.cc:535] client connection to 127.8.138.189:37105 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 15:02:54.926821 9690 connection.cc:535] client connection to 127.8.138.189:37105 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 15:02:54.926779 10519 connection.cc:535] server connection from 127.0.0.1:41612 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 15:02:54.927955 9690 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.8.138.189:37105)
W20260926 15:02:54.928323 9690 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.8.138.189:37105)
W20260926 15:02:54.928565 9690 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.8.138.189:37105)
W20260926 15:02:54.929176 9690 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.8.138.189:37105)
I20260926 15:02:55.194635 10575 raft_consensus.cc:482] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader be9958f2085242f7ab82ae933ceb1c42)
I20260926 15:02:55.194929 10575 raft_consensus.cc:504] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:55.197165 10575 leader_election.cc:288] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105)
W20260926 15:02:55.198232 10519 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
W20260926 15:02:55.200019 10522 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
W20260926 15:02:55.200495 10519 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281): Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
I20260926 15:02:55.200685 10519 leader_election.cc:302] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f; no voters: 6315fb5099e1420b95b8d83b3e19375c, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:02:55.201128 10575 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260926 15:02:55.868280 10626 raft_consensus.cc:482] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader be9958f2085242f7ab82ae933ceb1c42)
I20260926 15:02:55.868500 10626 raft_consensus.cc:504] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:55.869390 10626 leader_election.cc:288] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105)
W20260926 15:02:55.871415 10519 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281): Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
W20260926 15:02:55.871588 10522 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:02:55.871728 10522 leader_election.cc:302] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f; no voters: 6315fb5099e1420b95b8d83b3e19375c, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:02:55.872042 10626 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260926 15:02:56.683370 10631 raft_consensus.cc:482] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader be9958f2085242f7ab82ae933ceb1c42)
I20260926 15:02:56.683653 10631 raft_consensus.cc:504] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:56.684376 10631 leader_election.cc:288] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105)
W20260926 15:02:56.686784 10519 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281): Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
W20260926 15:02:56.687023 10522 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:02:56.687181 10522 leader_election.cc:302] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f; no voters: 6315fb5099e1420b95b8d83b3e19375c, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:02:56.687418 10631 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260926 15:02:57.284565 8746 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu
/tmp/dist-test-taskLGH8i2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.138.189:37105
--webserver_interface=127.8.138.189
--webserver_port=40173
--builtin_ntp_servers=127.8.138.148:33835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473
--hive_metastore_uris=thrift://127.0.0.1:40047
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260926 15:02:57.289477 10637 raft_consensus.cc:482] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader be9958f2085242f7ab82ae933ceb1c42)
I20260926 15:02:57.289671 10637 raft_consensus.cc:504] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:57.290386 10637 leader_election.cc:288] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105)
W20260926 15:02:57.292428 10519 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281): Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
W20260926 15:02:57.292784 10522 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:02:57.292948 10522 leader_election.cc:302] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f; no voters: 6315fb5099e1420b95b8d83b3e19375c, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:02:57.293233 10637 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260926 15:02:57.632741 10636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:57.633045 10636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:57.633165 10636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:57.641453 10636 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:02:57.641569 10636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:57.641661 10636 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:02:57.641768 10636 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:02:57.651118 10636 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.138.148:33835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:40047
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.138.189:37105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.138.189
--webserver_port=40173
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:57.652685 10636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:57.654698 10636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:57.666978 10647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:57.666952 10646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:57.667450 10649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:57.668216 10636 server_base.cc:1034] running on GCE node
I20260926 15:02:57.669086 10636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:57.670617 10636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:57.671823 10636 hybrid_clock.cc:648] HybridClock initialized: now 1790434977671732 us; error 58 us; skew 500 ppm
I20260926 15:02:57.672240 10636 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:02:57.683288 10636 webserver.cc:460] Webserver started at http://127.8.138.189:40173/ using document root <none> and password file <none>
I20260926 15:02:57.684242 10636 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:57.684360 10636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:57.690392 10636 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260926 15:02:57.693593 10655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:57.695215 10636 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 15:02:57.695403 10636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/data,/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/wal
uuid: "be9958f2085242f7ab82ae933ceb1c42"
format_stamp: "Formatted at 2026-09-26 15:02:34 on dist-test-slave-7qf4"
I20260926 15:02:57.696471 10636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:57.720352 10636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:57.721345 10636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:57.721664 10636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:57.741132 10636 rpc_server.cc:307] RPC server started. Bound to: 127.8.138.189:37105
I20260926 15:02:57.741151 10706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.138.189:37105 every 8 connection(s)
I20260926 15:02:57.742750 10636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-1/data/info.pb
I20260926 15:02:57.745442 8746 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu as pid 10636
I20260926 15:02:57.750643 10707 sys_catalog.cc:263] Verifying existing consensus state
I20260926 15:02:57.754436 10707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42: Bootstrap starting.
I20260926 15:02:57.766429 10707 log.cc:824] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42: Log is configured to *not* fsync() on all Append() calls
I20260926 15:02:57.805176 10707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42: Bootstrap replayed 1/1 log segments. Stats: ops{read=50 overwritten=0 applied=47 ignored=0} inserts{seen=31 ignored=0} mutations{seen=30 ignored=0} orphaned_commits=0. Pending: 3 replicates
I20260926 15:02:57.805918 10707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42: Bootstrap complete.
I20260926 15:02:57.813337 10707 raft_consensus.cc:348] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 8 FOLLOWER]: Replica starting. Triggering 3 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:57.817004 10707 raft_consensus.cc:729] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 8 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9958f2085242f7ab82ae933ceb1c42, State: Initialized, Role: FOLLOWER
I20260926 15:02:57.817734 10707 consensus_queue.cc:260] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 47, Last appended: 8.50, Last appended by leader: 50, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:57.818576 10712 sys_catalog.cc:455] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 8 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:57.818892 10712 sys_catalog.cc:458] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:02:57.819667 10707 sys_catalog.cc:564] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:02:57.831403 10726 catalog_manager.cc:1259] Loaded cluster ID: 497ed160bc39477e99bf3beb637a4391
I20260926 15:02:57.831523 10726 catalog_manager.cc:1552] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42: loading cluster ID for follower catalog manager: success
I20260926 15:02:57.835988 10726 catalog_manager.cc:1574] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42: acquiring CA information for follower catalog manager: success
I20260926 15:02:57.839042 10726 catalog_manager.cc:1602] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 15:02:57.840667 10636 master_runner.cc:186] Error getting master registration for 127.8.138.190:44281: Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111),
I20260926 15:02:57.848533 10636 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:02:57.943892 9679 heartbeater.cc:347] Connected to a master server at 127.8.138.189:37105
I20260926 15:02:57.944244 9534 heartbeater.cc:347] Connected to a master server at 127.8.138.189:37105
I20260926 15:02:57.947415 10672 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" instance_seqno: 1790434956879292) as {username='slave'} at 127.8.138.130:41275; Asking this server to re-register.
I20260926 15:02:57.947443 10671 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" instance_seqno: 1790434957773527) as {username='slave'} at 127.8.138.131:35345; Asking this server to re-register.
I20260926 15:02:57.948196 9534 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:57.948266 9679 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:57.950016 10672 ts_manager.cc:194] Registered new tserver with Master: f052d5cbee65406593d8e44f74225bc2 (127.8.138.131:35445)
I20260926 15:02:57.950095 10671 ts_manager.cc:194] Registered new tserver with Master: ae330d635b1848aaaecee2c7ce96520a (127.8.138.130:44309)
I20260926 15:02:58.017764 9394 heartbeater.cc:347] Connected to a master server at 127.8.138.189:37105
I20260926 15:02:58.019405 10671 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" instance_seqno: 1790434956216737) as {username='slave'} at 127.8.138.129:59847; Asking this server to re-register.
I20260926 15:02:58.020258 9394 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:58.023032 10671 ts_manager.cc:194] Registered new tserver with Master: 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327)
I20260926 15:02:58.306295 10712 raft_consensus.cc:482] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:02:58.306568 10712 raft_consensus.cc:504] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:58.308439 10712 leader_election.cc:288] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:02:58.309211 10656 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
W20260926 15:02:58.311043 10656 leader_election.cc:334] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281): Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
I20260926 15:02:58.315537 10544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "be9958f2085242f7ab82ae933ceb1c42" candidate_term: 9 candidate_status { last_received { term: 8 index: 50 } } ignore_live_leader: false dest_uuid: "320cd09624624312bf1bcc0856c2874f" is_pre_election: true
I20260926 15:02:58.315892 10544 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9958f2085242f7ab82ae933ceb1c42 in term 8.
I20260926 15:02:58.316604 10658 leader_election.cc:302] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42; no voters: 6315fb5099e1420b95b8d83b3e19375c
I20260926 15:02:58.317137 10712 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260926 15:02:58.317294 10712 raft_consensus.cc:482] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:02:58.317477 10712 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 8 FOLLOWER]: Advancing to term 9
I20260926 15:02:58.321177 10712 raft_consensus.cc:504] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:58.321832 10712 leader_election.cc:288] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 9 election: Requested vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
I20260926 15:02:58.322495 10544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "be9958f2085242f7ab82ae933ceb1c42" candidate_term: 9 candidate_status { last_received { term: 8 index: 50 } } ignore_live_leader: false dest_uuid: "320cd09624624312bf1bcc0856c2874f"
I20260926 15:02:58.322741 10544 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 8 FOLLOWER]: Advancing to term 9
W20260926 15:02:58.323449 10656 leader_election.cc:334] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281): Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
I20260926 15:02:58.324985 10544 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9958f2085242f7ab82ae933ceb1c42 in term 9.
I20260926 15:02:58.325568 10658 leader_election.cc:302] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42; no voters: 6315fb5099e1420b95b8d83b3e19375c
I20260926 15:02:58.325932 10712 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 9 FOLLOWER]: Leader election won for term 9
I20260926 15:02:58.326346 10712 raft_consensus.cc:686] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [term 9 LEADER]: Becoming Leader. State: Replica: be9958f2085242f7ab82ae933ceb1c42, State: Running, Role: LEADER
I20260926 15:02:58.326830 10712 consensus_queue.cc:237] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 47, Committed index: 47, Last appended: 8.50, Last appended by leader: 50, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:58.329442 10735 sys_catalog.cc:455] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be9958f2085242f7ab82ae933ceb1c42. Latest consensus state: current_term: 9 leader_uuid: "be9958f2085242f7ab82ae933ceb1c42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:58.329790 10735 sys_catalog.cc:458] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [sys.catalog]: This master's current role is: LEADER
I20260926 15:02:58.860215 10544 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Refusing update from remote peer be9958f2085242f7ab82ae933ceb1c42: Log matching property violated. Preceding OpId in replica: term: 8 index: 50. Preceding OpId from leader: term: 9 index: 51. (index mismatch)
I20260926 15:02:58.861133 10740 consensus_queue.cc:1035] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 51, Last known committed idx: 48, Time since last communication: 0.000s
I20260926 15:02:58.863322 10741 sys_catalog.cc:455] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: SysCatalogTable state changed. Reason: New leader be9958f2085242f7ab82ae933ceb1c42. Latest consensus state: current_term: 9 leader_uuid: "be9958f2085242f7ab82ae933ceb1c42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:58.863695 10741 sys_catalog.cc:458] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:02:58.867650 10741 sys_catalog.cc:455] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 9 leader_uuid: "be9958f2085242f7ab82ae933ceb1c42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:58.867911 10741 sys_catalog.cc:458] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:02:58.868855 10743 sys_catalog.cc:455] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 9 leader_uuid: "be9958f2085242f7ab82ae933ceb1c42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:02:58.869153 10743 sys_catalog.cc:458] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 [sys.catalog]: This master's current role is: LEADER
W20260926 15:02:58.870352 10656 consensus_peers.cc:487] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42 -> Peer 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281): Couldn't send request to peer 6315fb5099e1420b95b8d83b3e19375c. Status: Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 15:02:58.975884 10737 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:02:58.978561 10737 catalog_manager.cc:669] Loaded metadata for table default.table_43176f08d9e24974abdd53171465a15c [id=08b331ce5f9c44389fae209bac801f42]
I20260926 15:02:58.979048 10737 catalog_manager.cc:669] Loaded metadata for table default.table_854219551f9048c6925c711ce33d5256 [id=3295fc8a0a004843be045b8bf6a4f2cf]
I20260926 15:02:58.979447 10737 catalog_manager.cc:669] Loaded metadata for table default.table_96be20374f7e43e7a865549e39bd64f6 [id=51dea274f4bf4903baaddbbd8c257b27]
I20260926 15:02:58.979808 10737 catalog_manager.cc:669] Loaded metadata for table default.table_d2243274908043e58210a197141af813 [id=5357ddcf21634f99bb4f9abc5b6c4d80]
I20260926 15:02:58.980218 10737 catalog_manager.cc:669] Loaded metadata for table default.table_95f48725b6ce47fab5f96e18ccb69dbc [id=630deebc15b04878ae13844ff2808f52]
I20260926 15:02:58.980585 10737 catalog_manager.cc:669] Loaded metadata for table default.table_22fd4957e3d14964803428457f8490ac [id=7ba7b39b84394421b88a6ce7616c3c91]
I20260926 15:02:58.980911 10737 catalog_manager.cc:669] Loaded metadata for table default.table_41a3ce3c89684cbb9054c4251a0cc51c [id=7cc0c844d49c4ff08cef9e2c410991f8]
I20260926 15:02:58.981235 10737 catalog_manager.cc:669] Loaded metadata for table default.table_7b56039466d14dcab4dc63192de3b08e [id=b84ce6d15e4246dbbb299468138add55]
I20260926 15:02:58.981550 10737 catalog_manager.cc:669] Loaded metadata for table default.table_e3b713aac3c94e449bb5882310d55a40 [id=e9e378126d2649dab722c82f99d654b7]
I20260926 15:02:58.983578 10737 tablet_loader.cc:96] loaded metadata for tablet 12efb00aa1c649729d8a926b784e69c0 (table default.table_22fd4957e3d14964803428457f8490ac [id=7ba7b39b84394421b88a6ce7616c3c91])
I20260926 15:02:58.984019 10737 tablet_loader.cc:96] loaded metadata for tablet 252e506a97564739bc26268653efa864 (table default.table_e3b713aac3c94e449bb5882310d55a40 [id=e9e378126d2649dab722c82f99d654b7])
I20260926 15:02:58.984755 10737 tablet_loader.cc:96] loaded metadata for tablet 3fae8a76002849de9bb26610a0c33406 (table default.table_95f48725b6ce47fab5f96e18ccb69dbc [id=630deebc15b04878ae13844ff2808f52])
I20260926 15:02:58.985152 10737 tablet_loader.cc:96] loaded metadata for tablet 48407739eb2542d59cf842cb5987ad8f (table default.table_43176f08d9e24974abdd53171465a15c [id=08b331ce5f9c44389fae209bac801f42])
I20260926 15:02:58.985801 10737 tablet_loader.cc:96] loaded metadata for tablet 6fa3445c418c4954b8550f556d373743 (table default.table_d2243274908043e58210a197141af813 [id=5357ddcf21634f99bb4f9abc5b6c4d80])
I20260926 15:02:58.986168 10737 tablet_loader.cc:96] loaded metadata for tablet 86fed5bdf8b7474dbff0c53d6d4d673b (table default.table_7b56039466d14dcab4dc63192de3b08e [id=b84ce6d15e4246dbbb299468138add55])
I20260926 15:02:58.986672 10737 tablet_loader.cc:96] loaded metadata for tablet a6a3a867e477463491cc7a24df6faa5d (table default.table_854219551f9048c6925c711ce33d5256 [id=3295fc8a0a004843be045b8bf6a4f2cf])
I20260926 15:02:58.987315 10737 tablet_loader.cc:96] loaded metadata for tablet cda28e268552442d8774ac5986e76801 (table default.table_96be20374f7e43e7a865549e39bd64f6 [id=51dea274f4bf4903baaddbbd8c257b27])
I20260926 15:02:58.987829 10737 tablet_loader.cc:96] loaded metadata for tablet df8efd9136304dc09f153187a9e55cb1 (table default.table_41a3ce3c89684cbb9054c4251a0cc51c [id=7cc0c844d49c4ff08cef9e2c410991f8])
I20260926 15:02:58.988131 10737 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:02:58.989171 10737 catalog_manager.cc:1259] Loaded cluster ID: 497ed160bc39477e99bf3beb637a4391
I20260926 15:02:58.989279 10737 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:02:58.990561 10737 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:02:58.991516 10737 catalog_manager.cc:5879] T 00000000000000000000000000000000 P be9958f2085242f7ab82ae933ceb1c42: Loaded TSK: 0
I20260926 15:02:58.992441 10737 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:02:58.993508 10737 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260926 15:02:58.994874 10737 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260926 15:02:59.050858 10671 master_service.cc:946] ReplaceTablet: received request to replace tablet 0d837276d648440cbebe1f88c281cb11 from {username='slave'} at 127.0.0.1:41106
I20260926 15:02:59.050889 10672 master_service.cc:946] ReplaceTablet: received request to replace tablet 7fa51439f5324161991e8dbbd169851e from {username='slave'} at 127.0.0.1:41106
I20260926 15:02:59.052469 10669 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41106:
table { table_name: "default.Table_43176f08d9e24974abdd53171465a15c" } new_table_name: "default.Table_1b2fcfff41b54f21b229a3e2064d7c26" modify_external_catalogs: true
I20260926 15:02:59.052095 10667 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41106:
name: "default.Table_6c170e0fd90e42f08500c2a89216a4d3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:59.052222 10668 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41106:
name: "default.Table_985d0971fb8a4d8eae0b2cb73ca99c6f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:59.055007 10672 master_service.cc:946] ReplaceTablet: received request to replace tablet c5d07d3a7a164d098e61f8f7f2ac4a67 from {username='slave'} at 127.0.0.1:41106
I20260926 15:02:59.054883 10664 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41106:
table { table_name: "default.Table_e3b713aac3c94e449bb5882310d55a40" } new_table_name: "default.Table_fd56d79387ed41638b65164801eaa9ab" modify_external_catalogs: true
I20260926 15:02:59.053501 10663 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41106:
name: "default.Table_04fd049074c940dba0495d97eac1d272"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:59.054977 10665 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41106:
table { table_name: "default.Table_7b56039466d14dcab4dc63192de3b08e" } modify_external_catalogs: true
I20260926 15:02:59.054044 10670 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41106:
name: "default.Table_a320fb1edf9a4372928664c4263b213e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:59.054854 10671 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41106:
name: "default.Table_0e42d4f90b4e4529bb0a2ea0909cbadb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:59.056983 10672 master_service.cc:946] ReplaceTablet: received request to replace tablet 0d837276d648440cbebe1f88c281cb11 from {username='slave'} at 127.0.0.1:41106
I20260926 15:02:59.056891 10665 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41106:
table { table_name: "default.Table_7b56039466d14dcab4dc63192de3b08e" } modify_external_catalogs: true
I20260926 15:02:59.057471 10672 master_service.cc:946] ReplaceTablet: received request to replace tablet 41e641e28d1f48b2a7e0c8fb8964f678 from {username='slave'} at 127.0.0.1:41106
I20260926 15:02:59.055686 10666 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41106:
name: "default.Table_5bfdd1b3c4a247d58d08b75039b6124f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:59.058023 10672 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41106:
table { table_name: "default.Table_854219551f9048c6925c711ce33d5256" } modify_external_catalogs: true
I20260926 15:02:59.058239 10672 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41106:
table { table_name: "default.Table_854219551f9048c6925c711ce33d5256" } modify_external_catalogs: true
F20260926 15:02:59.060482 10725 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_f356bc0dfc6a47f98430002dccbb873f
*** Check failure stack trace: ***
@ 0x7f97b91c49ad google::LogMessage::Fail() at ??:0
@ 0x7f97b91c788a google::LogMessage::SendToLog() at ??:0
@ 0x7f97b91c44de google::LogMessage::Flush() at ??:0
@ 0x7f97b91c46d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f97ced62ba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f97ced61901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f97ced6866c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f97ced684e9 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f97bbdeeeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f97bf0c86db start_thread at ??:0
@ 0x7f97b805e61f clone at ??:0
I20260926 15:02:59.230412 10500 master_runner.cc:186] Error getting master registration for 127.8.138.190:44281: Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111),
I20260926 15:02:59.230669 10500 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 15:02:59.267791 9690 connection.cc:535] client connection to 127.8.138.189:37105 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 15:02:59.268599 9690 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.8.138.189:37105)
W20260926 15:02:59.269335 9690 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.8.138.189:37105)
W20260926 15:02:59.269647 9690 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.8.138.189:37105)
W20260926 15:02:59.269862 9690 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.8.138.189:37105)
I20260926 15:02:59.366129 10741 raft_consensus.cc:482] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader be9958f2085242f7ab82ae933ceb1c42)
I20260926 15:02:59.366372 10741 raft_consensus.cc:504] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:02:59.367252 10741 leader_election.cc:288] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105)
W20260926 15:02:59.369649 10522 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
W20260926 15:02:59.369848 10519 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281): Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
I20260926 15:02:59.370043 10519 leader_election.cc:302] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f; no voters: 6315fb5099e1420b95b8d83b3e19375c, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:02:59.370325 10741 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260926 15:02:59.528936 9394 heartbeater.cc:503] Master 127.8.138.189:37105 was elected leader, sending a full tablet report...
I20260926 15:02:59.963783 8746 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu pid 10500
I20260926 15:02:59.970806 10565 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:00.084903 10776 raft_consensus.cc:482] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader be9958f2085242f7ab82ae933ceb1c42)
I20260926 15:03:00.085153 8746 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu with pid 10500
W20260926 15:03:00.105716 9567 connection.cc:535] client connection to 127.8.138.188:45473 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 15:03:00.830343 8746 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu
/tmp/dist-test-taskLGH8i2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.138.188:45473
--webserver_interface=127.8.138.188
--webserver_port=38457
--builtin_ntp_servers=127.8.138.148:33835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473
--hive_metastore_uris=thrift://127.0.0.1:40047
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260926 15:03:01.175455 10779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:01.175769 10779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:01.175946 10779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:01.183559 10779 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:03:01.183708 10779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:01.183807 10779 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:03:01.183938 10779 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:03:01.193699 10779 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.138.148:33835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:40047
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.138.188:45473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.138.188
--webserver_port=38457
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:01.195176 10779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:01.197036 10779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:01.210136 10784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:01.210124 10785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:01.210088 10787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:01.211052 10779 server_base.cc:1034] running on GCE node
I20260926 15:03:01.211903 10779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:01.213357 10779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:01.214618 10779 hybrid_clock.cc:648] HybridClock initialized: now 1790434981214554 us; error 30 us; skew 500 ppm
I20260926 15:03:01.215035 10779 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:03:01.217720 10779 webserver.cc:460] Webserver started at http://127.8.138.188:38457/ using document root <none> and password file <none>
I20260926 15:03:01.218462 10779 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:01.218618 10779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:01.224670 10779 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 15:03:01.228278 10793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:01.230624 10779 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 15:03:01.230846 10779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data,/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/wal
uuid: "320cd09624624312bf1bcc0856c2874f"
format_stamp: "Formatted at 2026-09-26 15:02:35 on dist-test-slave-7qf4"
I20260926 15:03:01.232183 10779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:01.261938 10779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:01.262913 10779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:01.263248 10779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:01.284744 10779 rpc_server.cc:307] RPC server started. Bound to: 127.8.138.188:45473
I20260926 15:03:01.284761 10844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.138.188:45473 every 8 connection(s)
I20260926 15:03:01.286615 10779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data/info.pb
I20260926 15:03:01.290174 8746 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu as pid 10779
I20260926 15:03:01.294675 10845 sys_catalog.cc:263] Verifying existing consensus state
I20260926 15:03:01.298566 10845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: Bootstrap starting.
I20260926 15:03:01.312319 10845 log.cc:824] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: Log is configured to *not* fsync() on all Append() calls
I20260926 15:03:01.354239 10845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: Bootstrap replayed 1/1 log segments. Stats: ops{read=51 overwritten=0 applied=51 ignored=0} inserts{seen=34 ignored=0} mutations{seen=33 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 15:03:01.354830 10845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: Bootstrap complete.
I20260926 15:03:01.362531 10845 raft_consensus.cc:348] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:03:01.363464 10845 raft_consensus.cc:729] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Becoming Follower/Learner. State: Replica: 320cd09624624312bf1bcc0856c2874f, State: Initialized, Role: FOLLOWER
I20260926 15:03:01.364193 10845 consensus_queue.cc:260] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 51, Last appended: 9.51, Last appended by leader: 51, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:03:01.365013 10849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 9 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:03:01.365414 10849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:03:01.366127 10845 sys_catalog.cc:564] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:03:01.378388 10863 catalog_manager.cc:1259] Loaded cluster ID: 497ed160bc39477e99bf3beb637a4391
I20260926 15:03:01.378595 10863 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: loading cluster ID for follower catalog manager: success
I20260926 15:03:01.382920 10863 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: acquiring CA information for follower catalog manager: success
I20260926 15:03:01.386071 10863 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 15:03:01.387563 10779 master_runner.cc:186] Error getting master registration for 127.8.138.190:44281: Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111),
I20260926 15:03:01.397567 10779 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:03:01.596230 9390 heartbeater.cc:347] Connected to a master server at 127.8.138.188:45473
I20260926 15:03:01.600576 10810 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" instance_seqno: 1790434956216737) as {username='slave'} at 127.8.138.129:50913; Asking this server to re-register.
I20260926 15:03:01.601490 9390 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:01.603837 10810 ts_manager.cc:194] Registered new tserver with Master: 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327)
I20260926 15:03:01.616225 9677 heartbeater.cc:347] Connected to a master server at 127.8.138.188:45473
I20260926 15:03:01.617817 10810 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" instance_seqno: 1790434957773527) as {username='slave'} at 127.8.138.131:38405; Asking this server to re-register.
I20260926 15:03:01.618467 9677 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:01.619827 10810 ts_manager.cc:194] Registered new tserver with Master: f052d5cbee65406593d8e44f74225bc2 (127.8.138.131:35445)
I20260926 15:03:01.691052 9532 heartbeater.cc:347] Connected to a master server at 127.8.138.188:45473
I20260926 15:03:01.692528 10810 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" instance_seqno: 1790434956879292) as {username='slave'} at 127.8.138.130:38065; Asking this server to re-register.
I20260926 15:03:01.693132 9532 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:01.694622 10810 ts_manager.cc:194] Registered new tserver with Master: ae330d635b1848aaaecee2c7ce96520a (127.8.138.130:44309)
I20260926 15:03:01.789362 10849 raft_consensus.cc:482] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:03:01.789695 10849 raft_consensus.cc:504] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:03:01.791565 10849 leader_election.cc:288] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105)
W20260926 15:03:01.792506 10794 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
W20260926 15:03:01.793908 10797 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
W20260926 15:03:01.794289 10794 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281): Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
I20260926 15:03:01.794473 10794 leader_election.cc:302] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f; no voters: 6315fb5099e1420b95b8d83b3e19375c, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:03:01.794849 10849 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260926 15:03:02.504194 10870 raft_consensus.cc:482] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:03:02.504402 10870 raft_consensus.cc:504] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:03:02.505347 10870 leader_election.cc:288] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105)
W20260926 15:03:02.507513 10794 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281): Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
W20260926 15:03:02.507730 10797 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:03:02.507911 10797 leader_election.cc:302] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f; no voters: 6315fb5099e1420b95b8d83b3e19375c, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:03:02.508148 10870 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260926 15:03:03.031772 10874 raft_consensus.cc:482] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:03:03.032040 10874 raft_consensus.cc:504] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:03:03.032703 10874 leader_election.cc:288] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105)
W20260926 15:03:03.034931 10794 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281): Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
W20260926 15:03:03.035131 10797 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:03:03.035249 10797 leader_election.cc:302] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f; no voters: 6315fb5099e1420b95b8d83b3e19375c, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:03:03.035456 10874 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260926 15:03:03.582201 10879 raft_consensus.cc:482] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:03:03.582480 10879 raft_consensus.cc:504] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:03:03.583175 10879 leader_election.cc:288] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105)
W20260926 15:03:03.585424 10797 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
W20260926 15:03:03.585628 10794 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281): Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
I20260926 15:03:03.585794 10794 leader_election.cc:302] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f; no voters: 6315fb5099e1420b95b8d83b3e19375c, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:03:03.586063 10879 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260926 15:03:03.867419 8746 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu pid 10779
I20260926 15:03:03.874951 10840 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:03.992873 8746 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu with pid 10779
I20260926 15:03:04.637143 8746 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu
/tmp/dist-test-taskLGH8i2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.138.188:45473
--webserver_interface=127.8.138.188
--webserver_port=38457
--builtin_ntp_servers=127.8.138.148:33835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473
--hive_metastore_uris=thrift://127.0.0.1:40047
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260926 15:03:04.981803 10887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:04.982101 10887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:04.982272 10887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:04.989895 10887 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:03:04.990020 10887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:04.990095 10887 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:03:04.990172 10887 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:03:04.999826 10887 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.138.148:33835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:40047
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.138.188:45473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.138.188
--webserver_port=38457
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:05.001336 10887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:05.003172 10887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:05.016784 10892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:05.016714 10893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:05.016749 10895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:05.017807 10887 server_base.cc:1034] running on GCE node
I20260926 15:03:05.018738 10887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:05.020299 10887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:05.021554 10887 hybrid_clock.cc:648] HybridClock initialized: now 1790434985021488 us; error 45 us; skew 500 ppm
I20260926 15:03:05.021900 10887 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:03:05.025166 10887 webserver.cc:460] Webserver started at http://127.8.138.188:38457/ using document root <none> and password file <none>
I20260926 15:03:05.025913 10887 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:05.026043 10887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:05.031625 10887 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.000s
I20260926 15:03:05.034742 10901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:05.036410 10887 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:03:05.036597 10887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data,/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/wal
uuid: "320cd09624624312bf1bcc0856c2874f"
format_stamp: "Formatted at 2026-09-26 15:02:35 on dist-test-slave-7qf4"
I20260926 15:03:05.037596 10887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:05.053819 10887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:05.054718 10887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:05.055002 10887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:05.073915 10887 rpc_server.cc:307] RPC server started. Bound to: 127.8.138.188:45473
I20260926 15:03:05.073949 10952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.138.188:45473 every 8 connection(s)
I20260926 15:03:05.075397 10887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-2/data/info.pb
I20260926 15:03:05.075801 8746 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu as pid 10887
I20260926 15:03:05.083287 10953 sys_catalog.cc:263] Verifying existing consensus state
I20260926 15:03:05.088253 10953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: Bootstrap starting.
I20260926 15:03:05.101052 10953 log.cc:824] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: Log is configured to *not* fsync() on all Append() calls
I20260926 15:03:05.126044 9390 heartbeater.cc:347] Connected to a master server at 127.8.138.188:45473
I20260926 15:03:05.141194 9677 heartbeater.cc:347] Connected to a master server at 127.8.138.188:45473
I20260926 15:03:05.145484 10953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: Bootstrap replayed 1/1 log segments. Stats: ops{read=51 overwritten=0 applied=51 ignored=0} inserts{seen=34 ignored=0} mutations{seen=33 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 15:03:05.146061 10953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: Bootstrap complete.
I20260926 15:03:05.154007 10953 raft_consensus.cc:348] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:03:05.155227 10953 raft_consensus.cc:729] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Becoming Follower/Learner. State: Replica: 320cd09624624312bf1bcc0856c2874f, State: Initialized, Role: FOLLOWER
I20260926 15:03:05.155891 10953 consensus_queue.cc:260] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 51, Last appended: 9.51, Last appended by leader: 51, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:03:05.156708 10957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 9 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:03:05.157119 10957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:03:05.158052 10953 sys_catalog.cc:564] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:03:05.178192 10971 catalog_manager.cc:1259] Loaded cluster ID: 497ed160bc39477e99bf3beb637a4391
I20260926 15:03:05.178320 10971 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: loading cluster ID for follower catalog manager: success
I20260926 15:03:05.171499 10887 master_runner.cc:186] Error getting master registration for 127.8.138.190:44281: Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111),
I20260926 15:03:05.181658 10887 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:03:05.183302 10971 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: acquiring CA information for follower catalog manager: success
I20260926 15:03:05.186534 10971 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 15:03:05.215818 9532 heartbeater.cc:347] Connected to a master server at 127.8.138.188:45473
I20260926 15:03:05.218597 10918 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" instance_seqno: 1790434956879292) as {username='slave'} at 127.8.138.130:44435; Asking this server to re-register.
I20260926 15:03:05.219309 9532 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:05.221254 10918 ts_manager.cc:194] Registered new tserver with Master: ae330d635b1848aaaecee2c7ce96520a (127.8.138.130:44309)
I20260926 15:03:05.630674 10918 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" instance_seqno: 1790434956216737) as {username='slave'} at 127.8.138.129:46081; Asking this server to re-register.
I20260926 15:03:05.631551 9390 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:05.633168 10918 ts_manager.cc:194] Registered new tserver with Master: 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327)
I20260926 15:03:05.644630 10918 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" instance_seqno: 1790434957773527) as {username='slave'} at 127.8.138.131:39051; Asking this server to re-register.
I20260926 15:03:05.645319 9677 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:05.646739 10918 ts_manager.cc:194] Registered new tserver with Master: f052d5cbee65406593d8e44f74225bc2 (127.8.138.131:35445)
I20260926 15:03:05.700714 10975 raft_consensus.cc:482] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:03:05.701045 10975 raft_consensus.cc:504] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:03:05.703049 10975 leader_election.cc:288] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105)
W20260926 15:03:05.704066 10902 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
W20260926 15:03:05.709596 10905 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
W20260926 15:03:05.709872 10902 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281): Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
I20260926 15:03:05.710140 10905 leader_election.cc:302] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f; no voters: 6315fb5099e1420b95b8d83b3e19375c, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:03:05.710551 10975 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260926 15:03:06.212049 10979 raft_consensus.cc:482] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:03:06.212267 10979 raft_consensus.cc:504] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:03:06.213085 10979 leader_election.cc:288] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105)
W20260926 15:03:06.215185 10905 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
W20260926 15:03:06.215422 10902 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281): Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
I20260926 15:03:06.215520 10902 leader_election.cc:302] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f; no voters: 6315fb5099e1420b95b8d83b3e19375c, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:03:06.215830 10979 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260926 15:03:06.980448 10983 raft_consensus.cc:482] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:03:06.980619 10983 raft_consensus.cc:504] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:03:06.981225 10983 leader_election.cc:288] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105)
W20260926 15:03:06.983377 10902 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281): Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
W20260926 15:03:06.983574 10905 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:03:06.983721 10905 leader_election.cc:302] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f; no voters: 6315fb5099e1420b95b8d83b3e19375c, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:03:06.984012 10983 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260926 15:03:07.524694 10987 raft_consensus.cc:482] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:03:07.524923 10987 raft_consensus.cc:504] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:03:07.525552 10987 leader_election.cc:288] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105)
W20260926 15:03:07.527742 10905 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
W20260926 15:03:07.527961 10902 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281): Network error: Client connection negotiation failed: client connection to 127.8.138.190:44281: connect: Connection refused (error 111)
I20260926 15:03:07.528113 10902 leader_election.cc:302] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f; no voters: 6315fb5099e1420b95b8d83b3e19375c, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:03:07.528448 10987 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260926 15:03:08.061949 8746 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu
/tmp/dist-test-taskLGH8i2/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.138.190:44281
--webserver_interface=127.8.138.190
--webserver_port=37041
--builtin_ntp_servers=127.8.138.148:33835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473
--hive_metastore_uris=thrift://127.0.0.1:40047
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260926 15:03:08.404044 10991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:08.404461 10991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:08.404610 10991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:08.412410 10991 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:03:08.412518 10991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:08.412633 10991 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:03:08.412729 10991 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:03:08.422675 10991 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.138.148:33835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:40047
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.138.190:44281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.138.190
--webserver_port=37041
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:08.424289 10991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:08.426213 10991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:08.438710 10997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:08.438670 10998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:08.438670 11000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:08.439649 10991 server_base.cc:1034] running on GCE node
I20260926 15:03:08.440498 10991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:08.441992 10991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:08.443245 10991 hybrid_clock.cc:648] HybridClock initialized: now 1790434988443166 us; error 58 us; skew 500 ppm
I20260926 15:03:08.443593 10991 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:03:08.446040 10991 webserver.cc:460] Webserver started at http://127.8.138.190:37041/ using document root <none> and password file <none>
I20260926 15:03:08.446730 10991 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:08.446889 10991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:08.452494 10991 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260926 15:03:08.455467 11006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:08.456847 10991 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:03:08.457052 10991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/data,/tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/wal
uuid: "6315fb5099e1420b95b8d83b3e19375c"
format_stamp: "Formatted at 2026-09-26 15:02:34 on dist-test-slave-7qf4"
I20260926 15:03:08.458150 10991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:08.485004 10991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:08.485905 10991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:08.486219 10991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:08.505383 10991 rpc_server.cc:307] RPC server started. Bound to: 127.8.138.190:44281
I20260926 15:03:08.505409 11057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.138.190:44281 every 8 connection(s)
I20260926 15:03:08.506850 10991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLGH8i2/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1790434939573899-8746-0/minicluster-data/master-0/data/info.pb
I20260926 15:03:08.511365 8746 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLGH8i2/build/asan/bin/kudu as pid 10991
I20260926 15:03:08.514113 11058 sys_catalog.cc:263] Verifying existing consensus state
I20260926 15:03:08.518325 11058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c: Bootstrap starting.
I20260926 15:03:08.530683 9391 heartbeater.cc:347] Connected to a master server at 127.8.138.190:44281
I20260926 15:03:08.530936 9391 heartbeater.cc:503] Master 127.8.138.190:44281 was elected leader, sending a full tablet report...
I20260926 15:03:08.533463 11058 log.cc:824] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c: Log is configured to *not* fsync() on all Append() calls
I20260926 15:03:08.566325 11058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c: Bootstrap replayed 1/1 log segments. Stats: ops{read=46 overwritten=1 applied=40 ignored=0} inserts{seen=27 ignored=0} mutations{seen=26 ignored=0} orphaned_commits=0. Pending: 5 replicates
I20260926 15:03:08.567066 11058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c: Bootstrap complete.
I20260926 15:03:08.574658 11058 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 6 FOLLOWER]: Replica starting. Triggering 5 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:03:08.578632 11058 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6315fb5099e1420b95b8d83b3e19375c, State: Initialized, Role: FOLLOWER
I20260926 15:03:08.579337 11058 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 40, Last appended: 6.45, Last appended by leader: 45, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:03:08.580240 11064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:03:08.580708 11064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:03:08.581647 11058 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:03:08.596812 11078 catalog_manager.cc:1259] Loaded cluster ID: 497ed160bc39477e99bf3beb637a4391
I20260926 15:03:08.596936 11078 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c: loading cluster ID for follower catalog manager: success
I20260926 15:03:08.601167 11078 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c: acquiring CA information for follower catalog manager: success
I20260926 15:03:08.604079 11078 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 15:03:08.607442 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:03:08.616974 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:03:08.670403 9533 heartbeater.cc:347] Connected to a master server at 127.8.138.190:44281
I20260926 15:03:08.673852 11023 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "ae330d635b1848aaaecee2c7ce96520a" instance_seqno: 1790434956879292) as {username='slave'} at 127.8.138.130:57171; Asking this server to re-register.
I20260926 15:03:08.674587 9533 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:08.676518 11023 ts_manager.cc:194] Registered new tserver with Master: ae330d635b1848aaaecee2c7ce96520a (127.8.138.130:44309)
I20260926 15:03:08.838177 9678 heartbeater.cc:347] Connected to a master server at 127.8.138.190:44281
I20260926 15:03:08.840405 11023 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "f052d5cbee65406593d8e44f74225bc2" instance_seqno: 1790434957773527) as {username='slave'} at 127.8.138.131:39639; Asking this server to re-register.
I20260926 15:03:08.841161 9678 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:08.842978 11023 ts_manager.cc:194] Registered new tserver with Master: f052d5cbee65406593d8e44f74225bc2 (127.8.138.131:35445)
I20260926 15:03:08.983824 11082 raft_consensus.cc:482] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:03:08.984143 11082 raft_consensus.cc:504] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:03:08.985013 11082 leader_election.cc:288] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105)
W20260926 15:03:08.987560 10905 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:03:08.992683 11033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "320cd09624624312bf1bcc0856c2874f" candidate_term: 10 candidate_status { last_received { term: 9 index: 51 } } ignore_live_leader: false dest_uuid: "6315fb5099e1420b95b8d83b3e19375c" is_pre_election: true
I20260926 15:03:08.993224 11033 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 320cd09624624312bf1bcc0856c2874f in term 6.
I20260926 15:03:08.993897 10902 leader_election.cc:302] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f, 6315fb5099e1420b95b8d83b3e19375c; no voters: be9958f2085242f7ab82ae933ceb1c42
I20260926 15:03:08.994203 11082 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260926 15:03:08.994451 11082 raft_consensus.cc:482] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:03:08.994552 11082 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 9 FOLLOWER]: Advancing to term 10
I20260926 15:03:08.998878 11082 raft_consensus.cc:504] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:03:08.999648 11082 leader_election.cc:288] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 election: Requested vote from peers 6315fb5099e1420b95b8d83b3e19375c (127.8.138.190:44281), be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105)
I20260926 15:03:09.000275 11033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "320cd09624624312bf1bcc0856c2874f" candidate_term: 10 candidate_status { last_received { term: 9 index: 51 } } ignore_live_leader: false dest_uuid: "6315fb5099e1420b95b8d83b3e19375c"
I20260926 15:03:09.000641 11033 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 6 FOLLOWER]: Advancing to term 10
W20260926 15:03:09.001423 10905 leader_election.cc:334] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:03:09.004884 11033 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 320cd09624624312bf1bcc0856c2874f in term 10.
I20260926 15:03:09.005435 10902 leader_election.cc:302] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 320cd09624624312bf1bcc0856c2874f, 6315fb5099e1420b95b8d83b3e19375c; no voters: be9958f2085242f7ab82ae933ceb1c42
I20260926 15:03:09.005780 11082 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 10 FOLLOWER]: Leader election won for term 10
I20260926 15:03:09.006361 11082 raft_consensus.cc:686] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [term 10 LEADER]: Becoming Leader. State: Replica: 320cd09624624312bf1bcc0856c2874f, State: Running, Role: LEADER
I20260926 15:03:09.006870 11082 consensus_queue.cc:237] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 51, Committed index: 51, Last appended: 9.51, Last appended by leader: 51, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:03:09.009783 11086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 320cd09624624312bf1bcc0856c2874f. Latest consensus state: current_term: 10 leader_uuid: "320cd09624624312bf1bcc0856c2874f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:03:09.010244 11086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:09.010941 11088 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:03:09.013808 11088 catalog_manager.cc:669] Loaded metadata for table default.table_43176f08d9e24974abdd53171465a15c [id=08b331ce5f9c44389fae209bac801f42]
I20260926 15:03:09.014381 11088 catalog_manager.cc:669] Loaded metadata for table default.table_854219551f9048c6925c711ce33d5256 [id=3295fc8a0a004843be045b8bf6a4f2cf]
I20260926 15:03:09.014858 11088 catalog_manager.cc:669] Loaded metadata for table default.table_96be20374f7e43e7a865549e39bd64f6 [id=51dea274f4bf4903baaddbbd8c257b27]
I20260926 15:03:09.015357 11088 catalog_manager.cc:669] Loaded metadata for table default.table_d2243274908043e58210a197141af813 [id=5357ddcf21634f99bb4f9abc5b6c4d80]
I20260926 15:03:09.015710 11088 catalog_manager.cc:669] Loaded metadata for table default.table_95f48725b6ce47fab5f96e18ccb69dbc [id=630deebc15b04878ae13844ff2808f52]
I20260926 15:03:09.016179 11088 catalog_manager.cc:669] Loaded metadata for table default.table_22fd4957e3d14964803428457f8490ac [id=7ba7b39b84394421b88a6ce7616c3c91]
I20260926 15:03:09.016601 11088 catalog_manager.cc:669] Loaded metadata for table default.table_41a3ce3c89684cbb9054c4251a0cc51c [id=7cc0c844d49c4ff08cef9e2c410991f8]
I20260926 15:03:09.017033 11088 catalog_manager.cc:669] Loaded metadata for table default.table_7b56039466d14dcab4dc63192de3b08e [id=b84ce6d15e4246dbbb299468138add55]
I20260926 15:03:09.017383 11088 catalog_manager.cc:669] Loaded metadata for table default.table_e3b713aac3c94e449bb5882310d55a40 [id=e9e378126d2649dab722c82f99d654b7]
I20260926 15:03:09.019522 11088 tablet_loader.cc:96] loaded metadata for tablet 12efb00aa1c649729d8a926b784e69c0 (table default.table_22fd4957e3d14964803428457f8490ac [id=7ba7b39b84394421b88a6ce7616c3c91])
I20260926 15:03:09.019979 11088 tablet_loader.cc:96] loaded metadata for tablet 252e506a97564739bc26268653efa864 (table default.table_e3b713aac3c94e449bb5882310d55a40 [id=e9e378126d2649dab722c82f99d654b7])
I20260926 15:03:09.020629 11088 tablet_loader.cc:96] loaded metadata for tablet 3fae8a76002849de9bb26610a0c33406 (table default.table_95f48725b6ce47fab5f96e18ccb69dbc [id=630deebc15b04878ae13844ff2808f52])
I20260926 15:03:09.021052 11088 tablet_loader.cc:96] loaded metadata for tablet 48407739eb2542d59cf842cb5987ad8f (table default.table_43176f08d9e24974abdd53171465a15c [id=08b331ce5f9c44389fae209bac801f42])
I20260926 15:03:09.021780 11088 tablet_loader.cc:96] loaded metadata for tablet 6fa3445c418c4954b8550f556d373743 (table default.table_d2243274908043e58210a197141af813 [id=5357ddcf21634f99bb4f9abc5b6c4d80])
I20260926 15:03:09.022181 11088 tablet_loader.cc:96] loaded metadata for tablet 86fed5bdf8b7474dbff0c53d6d4d673b (table default.table_7b56039466d14dcab4dc63192de3b08e [id=b84ce6d15e4246dbbb299468138add55])
I20260926 15:03:09.022589 11088 tablet_loader.cc:96] loaded metadata for tablet a6a3a867e477463491cc7a24df6faa5d (table default.table_854219551f9048c6925c711ce33d5256 [id=3295fc8a0a004843be045b8bf6a4f2cf])
I20260926 15:03:09.023281 11088 tablet_loader.cc:96] loaded metadata for tablet cda28e268552442d8774ac5986e76801 (table default.table_96be20374f7e43e7a865549e39bd64f6 [id=51dea274f4bf4903baaddbbd8c257b27])
I20260926 15:03:09.023725 11088 tablet_loader.cc:96] loaded metadata for tablet df8efd9136304dc09f153187a9e55cb1 (table default.table_41a3ce3c89684cbb9054c4251a0cc51c [id=7cc0c844d49c4ff08cef9e2c410991f8])
I20260926 15:03:09.024072 11088 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:03:09.025342 11088 catalog_manager.cc:1259] Loaded cluster ID: 497ed160bc39477e99bf3beb637a4391
I20260926 15:03:09.025417 11088 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:03:09.026872 11088 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:03:09.027951 11088 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f: Loaded TSK: 0
I20260926 15:03:09.028837 11088 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:03:09.029943 11088 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260926 15:03:09.030990 11088 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260926 15:03:09.034970 9391 heartbeater.cc:503] Master 127.8.138.190:44281 was elected leader, sending a full tablet report...
I20260926 15:03:09.037089 11023 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "66ca148489d64deabd1dcbf7870c6d57" instance_seqno: 1790434956216737) as {username='slave'} at 127.8.138.129:45355; Asking this server to re-register.
I20260926 15:03:09.038030 9391 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:09.039600 11023 ts_manager.cc:194] Registered new tserver with Master: 66ca148489d64deabd1dcbf7870c6d57 (127.8.138.129:40327)
I20260926 15:03:09.056455 10918 master_service.cc:946] ReplaceTablet: received request to replace tablet 7fa51439f5324161991e8dbbd169851e from {username='slave'} at 127.0.0.1:57326
I20260926 15:03:09.056583 10917 master_service.cc:946] ReplaceTablet: received request to replace tablet c8d090f2e0384924a72f288eff2281e7 from {username='slave'} at 127.0.0.1:57326
I20260926 15:03:09.057854 10916 master_service.cc:946] ReplaceTablet: received request to replace tablet 7fa51439f5324161991e8dbbd169851e from {username='slave'} at 127.0.0.1:57326
I20260926 15:03:09.058279 10915 master_service.cc:946] ReplaceTablet: received request to replace tablet 41e641e28d1f48b2a7e0c8fb8964f678 from {username='slave'} at 127.0.0.1:57326
I20260926 15:03:09.058053 10913 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57326:
name: "default.Table_7257407810154f2aa325ed506a8a62d5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:03:09.058516 10909 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:57326:
table { table_name: "default.Table_95f48725b6ce47fab5f96e18ccb69dbc" } new_table_name: "default.Table_85e820ec1f9f4f17be9385e54532213a" modify_external_catalogs: true
I20260926 15:03:09.059322 10910 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:57326:
table { table_name: "default.Table_41a3ce3c89684cbb9054c4251a0cc51c" } new_table_name: "default.Table_476468e6a23a414ebac439484ad9a68f" modify_external_catalogs: true
I20260926 15:03:09.059190 10915 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57326:
name: "default.Table_e26e1e9fab934eab9d1b22663b28c712"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:03:09.060067 10911 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57326:
table { table_name: "default.Table_96be20374f7e43e7a865549e39bd64f6" } modify_external_catalogs: true
I20260926 15:03:09.060354 10918 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57326:
name: "default.Table_a20572133e32441ab0bfe9e3fceed04a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:03:09.060421 10911 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57326:
table { table_name: "default.Table_96be20374f7e43e7a865549e39bd64f6" } modify_external_catalogs: true
I20260926 15:03:09.060738 10914 master_service.cc:946] ReplaceTablet: received request to replace tablet 252e506a97564739bc26268653efa864 from {username='slave'} at 127.0.0.1:57326
I20260926 15:03:09.061180 10916 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57326:
name: "default.Table_6c170e0fd90e42f08500c2a89216a4d3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:03:09.062495 10912 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57326:
name: "default.Table_aa60eda53a3f4f338866d309cbde44a9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:03:09.064230 10917 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:57326:
table { table_name: "default.Table_d2243274908043e58210a197141af813" } new_table_name: "default.Table_42b67a58948144f5a3b3abe972e975ba" modify_external_catalogs: true
F20260926 15:03:09.066200 10970 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 1): 1 CREATE_TABLE default.table_f356bc0dfc6a47f98430002dccbb873f
*** Check failure stack trace: ***
@ 0x7fa91c2e09ad google::LogMessage::Fail() at ??:0
@ 0x7fa91c2e388a google::LogMessage::SendToLog() at ??:0
I20260926 15:03:09.072152 11033 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Refusing update from remote peer 320cd09624624312bf1bcc0856c2874f: Log matching property violated. Preceding OpId in replica: term: 6 index: 45. Preceding OpId from leader: term: 10 index: 53. (index mismatch)
W20260926 15:03:09.072420 10905 consensus_peers.cc:487] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f -> Peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Couldn't send request to peer be9958f2085242f7ab82ae933ceb1c42. Status: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 15:03:09.073215 11082 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 320cd09624624312bf1bcc0856c2874f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 52, Last known committed idx: 40, Time since last communication: 0.000s
@ 0x7fa91c2e04de google::LogMessage::Flush() at ??:0
@ 0x7fa91c2e06d1 google::LogMessage::~LogMessage() at ??:0
@ 0x7fa931e7eba8 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7fa931e7d901 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7fa931e8466c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7fa931e844e9 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fa91ef0aeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fa9221e46db start_thread at ??:0
@ 0x7fa91b17a61f clone at ??:0
I20260926 15:03:09.123304 11104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 10 leader_uuid: "320cd09624624312bf1bcc0856c2874f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:03:09.123731 11104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:03:09.124766 11064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 320cd09624624312bf1bcc0856c2874f. Latest consensus state: current_term: 10 leader_uuid: "320cd09624624312bf1bcc0856c2874f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:03:09.124737 11094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 10 leader_uuid: "320cd09624624312bf1bcc0856c2874f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:03:09.125114 11064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:03:09.125195 11094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:03:09.125479 11102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 10 leader_uuid: "320cd09624624312bf1bcc0856c2874f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:03:09.125851 11102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:03:09.127488 11100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 10 leader_uuid: "320cd09624624312bf1bcc0856c2874f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } } }
I20260926 15:03:09.127918 11100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [sys.catalog]: This master's current role is: FOLLOWER
W20260926 15:03:09.296521 11008 connection.cc:535] server connection from 127.0.0.1:59446 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 15:03:09.296908 9567 connection.cc:535] client connection to 127.8.138.188:45473 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 15:03:09.296989 9422 connection.cc:535] client connection to 127.8.138.188:45473 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 15:03:09.297372 9689 connection.cc:535] client connection to 127.8.138.188:45473 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 15:03:09.297865 9280 connection.cc:535] client connection to 127.8.138.188:45473 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 15:03:09.298154 9689 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.8.138.188:45473)
W20260926 15:03:09.298867 9689 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.8.138.188:45473)
W20260926 15:03:09.299228 9689 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.8.138.188:45473)
W20260926 15:03:09.299566 9689 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.8.138.188:45473)
I20260926 15:03:09.576848 11100 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:03:09.577030 11100 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:03:09.578847 11100 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:03:09.580300 11010 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
W20260926 15:03:09.581761 11009 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
W20260926 15:03:09.581984 11010 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:03:09.582137 11010 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:03:09.582616 11100 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 15:03:10.315555 11117 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:03:10.315740 11117 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:03:10.316639 11117 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:03:10.318701 11010 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
W20260926 15:03:10.318890 11009 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
I20260926 15:03:10.319087 11009 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:03:10.319342 11117 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 15:03:10.926168 11122 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:03:10.926527 11122 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:03:10.927351 11122 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:03:10.929569 11010 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
W20260926 15:03:10.930121 11009 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
I20260926 15:03:10.930326 11009 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:03:10.930714 11122 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 15:03:11.048384 8746 master-stress-test.cc:578] Killing test thread 1/25
I20260926 15:03:11.600193 11126 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:03:11.600412 11126 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:03:11.601074 11126 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:03:11.603585 11009 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
W20260926 15:03:11.603817 11010 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:03:11.604023 11010 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:03:11.604310 11126 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 15:03:13.620139 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:03:13.620345 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:03:13.807142 11132 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:03:13.807354 11132 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:03:13.808183 11132 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:03:13.810503 11010 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
W20260926 15:03:13.810942 11009 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
I20260926 15:03:13.811046 11009 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:03:13.811331 11132 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 15:03:15.319031 11136 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:03:15.319216 11136 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:03:15.319833 11136 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:03:15.321054 11009 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20260926 15:03:15.322424 11010 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
W20260926 15:03:15.322616 11009 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
I20260926 15:03:15.322753 11009 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:03:15.322984 11136 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 15:03:17.410311 11140 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:03:17.410563 11140 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:03:17.411319 11140 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:03:17.413895 11009 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
W20260926 15:03:17.414191 11010 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:03:17.414345 11010 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:03:17.414619 11140 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 15:03:18.623924 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:03:18.624125 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:03:23.627704 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:03:23.627916 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:03:24.233659 11151 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:03:24.233942 11151 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:03:24.234781 11151 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:03:24.236672 11010 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260926 15:03:24.238593 11009 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
W20260926 15:03:24.238840 11010 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:03:24.238981 11010 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:03:24.239392 11151 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 15:03:28.631150 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:03:28.631426 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:03:31.311206 11162 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:03:31.311416 11162 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:03:31.312328 11162 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:03:31.313668 11009 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 15:03:31.315085 11010 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
W20260926 15:03:31.315366 11009 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
I20260926 15:03:31.315470 11009 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:03:31.315755 11162 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 15:03:33.634835 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:03:33.635062 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 15:03:38.378031 9390 heartbeater.cc:644] Failed to heartbeat to 127.8.138.188:45473 (58 consecutive failures): Network error: Failed to ping master at 127.8.138.188:45473: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
W20260926 15:03:38.378159 9677 heartbeater.cc:644] Failed to heartbeat to 127.8.138.188:45473 (58 consecutive failures): Network error: Failed to ping master at 127.8.138.188:45473: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
W20260926 15:03:38.378186 9532 heartbeater.cc:644] Failed to heartbeat to 127.8.138.188:45473 (58 consecutive failures): Network error: Failed to ping master at 127.8.138.188:45473: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
I20260926 15:03:38.638481 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:03:38.638724 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:03:43.641785 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:03:43.641996 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:03:47.850471 11192 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:03:47.850714 11192 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:03:47.851709 11192 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:03:47.852921 11010 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 15:03:47.854337 11009 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
W20260926 15:03:47.854488 11010 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:03:47.854588 11010 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:03:47.854841 11192 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 15:03:48.645450 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:03:48.645669 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:03:53.649029 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:03:53.649190 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:03:58.652612 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:03:58.652784 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:04:03.656351 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:04:03.656519 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:04:07.564785 11217 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:04:07.565100 11217 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:04:07.566015 11217 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:04:07.567270 11009 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 15:04:07.568734 11010 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
W20260926 15:04:07.569135 11009 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
I20260926 15:04:07.569235 11009 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:04:07.569588 11217 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 15:04:08.659649 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:04:08.659816 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:04:13.662405 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:04:13.662580 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:04:18.555222 11229 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:04:18.555506 11229 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:04:18.556224 11229 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:04:18.557418 11010 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 15:04:18.558719 11009 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
W20260926 15:04:18.558949 11010 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:04:18.559118 11010 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:04:18.559563 11229 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 15:04:18.666059 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:04:18.666275 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:04:23.669843 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:04:23.670019 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:04:28.673694 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:04:28.673870 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:04:32.770685 11251 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:04:32.770938 11251 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:04:32.771867 11251 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:04:32.773161 11009 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 15:04:32.774623 11010 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
W20260926 15:04:32.774823 11009 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
I20260926 15:04:32.774947 11009 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:04:32.775233 11251 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 15:04:33.679783 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:04:33.680032 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 15:04:38.542797 9390 heartbeater.cc:644] Failed to heartbeat to 127.8.138.188:45473 (178 consecutive failures): Network error: Failed to ping master at 127.8.138.188:45473: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
W20260926 15:04:38.542927 9677 heartbeater.cc:644] Failed to heartbeat to 127.8.138.188:45473 (178 consecutive failures): Network error: Failed to ping master at 127.8.138.188:45473: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
W20260926 15:04:38.543298 9532 heartbeater.cc:644] Failed to heartbeat to 127.8.138.188:45473 (178 consecutive failures): Network error: Failed to ping master at 127.8.138.188:45473: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
I20260926 15:04:38.683686 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:04:38.683995 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:04:43.687330 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:04:43.687561 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:04:48.690737 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:04:48.690907 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:04:49.481005 11276 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:04:49.481208 11276 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:04:49.482174 11276 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:04:49.483704 11009 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 15:04:49.485324 11009 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
W20260926 15:04:49.485704 11010 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:04:49.485885 11010 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:04:49.486173 11276 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 15:04:52.217609 11283 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:04:52.217803 11283 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:04:52.218477 11283 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:04:52.220861 11009 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
W20260926 15:04:52.221163 11010 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:04:52.221745 11010 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:04:52.222043 11283 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 15:04:53.694149 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:04:53.694355 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:04:58.697376 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:04:58.697525 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:05:03.700977 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:05:03.701152 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:05:05.858666 11304 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:05:05.858901 11304 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:05:05.859790 11304 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:05:05.860890 11010 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260926 15:05:05.862315 11010 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
W20260926 15:05:05.862491 11009 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
I20260926 15:05:05.862668 11009 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:05:05.862968 11304 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 15:05:08.682125 11012 maintenance_manager.cc:419] P 6315fb5099e1420b95b8d83b3e19375c: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033362
I20260926 15:05:08.717347 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:05:08.717631 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:05:08.736981 11011 maintenance_manager.cc:643] P 6315fb5099e1420b95b8d83b3e19375c: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.054s user 0.023s sys 0.005s Metrics: {"bytes_written":14158,"cfile_init":1,"dirs.queue_time_us":406,"dirs.run_cpu_time_us":222,"dirs.run_wall_time_us":1008,"drs_written":1,"lbm_read_time_us":68,"lbm_reads_lt_1ms":4,"lbm_write_time_us":793,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":34,"thread_start_us":566,"threads_started":2,"wal-append.queue_time_us":430}
I20260926 15:05:08.738339 11012 maintenance_manager.cc:419] P 6315fb5099e1420b95b8d83b3e19375c: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1660 bytes on disk
I20260926 15:05:08.739293 11011 maintenance_manager.cc:643] P 6315fb5099e1420b95b8d83b3e19375c: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":108,"lbm_reads_lt_1ms":4}
I20260926 15:05:13.721110 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:05:13.721307 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:05:18.724563 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:05:18.724772 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:05:23.728003 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:05:23.728211 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:05:23.982577 11357 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:05:23.982906 11357 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:05:23.985536 11357 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:05:23.986073 11009 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 15:05:23.987951 11010 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
W20260926 15:05:23.988165 11009 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
I20260926 15:05:23.988274 11009 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:05:23.988543 11357 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 15:05:28.731730 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:05:28.732076 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:05:33.735320 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:05:33.735545 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 15:05:38.710009 9532 heartbeater.cc:644] Failed to heartbeat to 127.8.138.188:45473 (298 consecutive failures): Network error: Failed to ping master at 127.8.138.188:45473: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
W20260926 15:05:38.710105 9677 heartbeater.cc:644] Failed to heartbeat to 127.8.138.188:45473 (298 consecutive failures): Network error: Failed to ping master at 127.8.138.188:45473: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
W20260926 15:05:38.710160 9390 heartbeater.cc:644] Failed to heartbeat to 127.8.138.188:45473 (298 consecutive failures): Network error: Failed to ping master at 127.8.138.188:45473: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
I20260926 15:05:38.739275 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:05:38.739464 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:05:41.778419 11440 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:05:41.778635 11440 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:05:41.779750 11440 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:05:41.781214 11010 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 15:05:41.782820 11009 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
W20260926 15:05:41.783102 11010 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:05:41.783299 11010 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:05:41.783635 11440 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 15:05:43.742632 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:05:43.742806 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:05:44.418903 11458 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:05:44.419122 11458 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:05:44.419816 11458 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:05:44.422202 11009 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
W20260926 15:05:44.422518 11010 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:05:44.422712 11010 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:05:44.423009 11458 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 15:05:48.746484 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:05:48.746748 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:05:53.750012 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:05:53.750173 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:05:54.622869 11516 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:05:54.623085 11516 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:05:54.624042 11516 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:05:54.625216 11010 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260926 15:05:54.627030 11010 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
W20260926 15:05:54.627213 11009 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
I20260926 15:05:54.627354 11009 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:05:54.627615 11516 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 15:05:58.753659 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:05:58.753911 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:06:03.756942 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:06:03.757125 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:06:08.760283 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:06:08.760462 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:06:13.552415 11603 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:06:13.552709 11603 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:06:13.553786 11603 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:06:13.555595 11009 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 15:06:13.557336 11009 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
W20260926 15:06:13.557613 11010 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:06:13.557787 11010 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:06:13.558056 11603 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 15:06:13.763443 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:06:13.763612 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:06:16.592322 11620 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:06:16.592625 11620 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:06:16.593816 11620 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:06:16.595633 11010 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
W20260926 15:06:16.595839 11009 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
I20260926 15:06:16.595985 11009 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:06:16.596220 11620 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 15:06:18.767153 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:06:18.767319 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:06:23.770466 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:06:23.770645 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:06:26.972968 11666 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:06:26.973169 11666 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:06:26.973789 11666 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:06:26.976677 11009 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260926 15:06:26.978044 11009 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
W20260926 15:06:26.978315 11010 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:06:26.978401 11010 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:06:26.978615 11666 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 15:06:28.378796 11677 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:06:28.379020 11677 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:06:28.379716 11677 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:06:28.382360 11009 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
W20260926 15:06:28.382598 11010 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:06:28.382766 11010 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:06:28.383184 11677 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 15:06:28.773824 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:06:28.773990 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:06:33.777683 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:06:33.777865 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:06:38.781167 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:06:38.781360 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 15:06:38.876502 9677 heartbeater.cc:644] Failed to heartbeat to 127.8.138.188:45473 (418 consecutive failures): Network error: Failed to ping master at 127.8.138.188:45473: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
W20260926 15:06:38.876864 9532 heartbeater.cc:644] Failed to heartbeat to 127.8.138.188:45473 (418 consecutive failures): Network error: Failed to ping master at 127.8.138.188:45473: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
W20260926 15:06:38.877043 9390 heartbeater.cc:644] Failed to heartbeat to 127.8.138.188:45473 (418 consecutive failures): Network error: Failed to ping master at 127.8.138.188:45473: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
I20260926 15:06:43.784377 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:06:43.784549 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:06:44.871510 11748 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:06:44.871739 11748 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:06:44.872622 11748 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:06:44.873925 11009 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260926 15:06:44.875370 11009 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
W20260926 15:06:44.875607 11010 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:06:44.875732 11010 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:06:44.876035 11748 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 15:06:48.787432 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:06:48.787676 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:06:53.790930 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:06:53.791100 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:06:58.794725 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:06:58.794934 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:07:03.038878 11827 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:07:03.039171 11827 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:07:03.040145 11827 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:07:03.041501 11010 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 15:07:03.043072 11010 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
W20260926 15:07:03.043269 11009 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
I20260926 15:07:03.043421 11009 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:07:03.043706 11827 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 15:07:03.798534 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:07:03.798702 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:07:08.801994 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:07:08.802325 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:07:10.702847 11862 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:07:10.703063 11862 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:07:10.704247 11862 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:07:10.705287 11010 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 15:07:10.706820 11010 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
W20260926 15:07:10.707017 11009 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
I20260926 15:07:10.707177 11009 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:07:10.707492 11862 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 15:07:13.805590 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:07:13.805826 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:07:18.809192 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:07:18.809473 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:07:21.570519 11906 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:07:21.570725 11906 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:07:21.571426 11906 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:07:21.572634 11010 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 15:07:21.574024 11009 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
W20260926 15:07:21.574326 11010 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:07:21.574461 11010 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:07:21.574707 11906 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 15:07:23.813167 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:07:23.813413 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:07:27.432669 11935 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:07:27.432893 11935 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:07:27.433516 11935 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:07:27.436681 11010 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 15:07:27.438230 11010 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
W20260926 15:07:27.438474 11009 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
I20260926 15:07:27.438580 11009 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:07:27.438941 11935 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 15:07:28.816226 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:07:28.816406 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:07:33.819486 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:07:33.819648 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:07:35.157938 11972 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:07:35.158133 11972 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:07:35.159039 11972 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:07:35.160283 11009 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 15:07:35.161538 11010 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
W20260926 15:07:35.161713 11009 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
I20260926 15:07:35.161851 11009 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:07:35.162168 11972 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 15:07:37.800400 11986 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:07:37.800621 11986 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:07:37.801327 11986 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:07:37.804164 11010 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
W20260926 15:07:37.804327 11009 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
I20260926 15:07:37.804487 11009 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:07:37.804761 11986 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 15:07:38.822853 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:07:38.823056 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 15:07:39.040688 9532 heartbeater.cc:644] Failed to heartbeat to 127.8.138.188:45473 (538 consecutive failures): Network error: Failed to ping master at 127.8.138.188:45473: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
W20260926 15:07:39.040833 9677 heartbeater.cc:644] Failed to heartbeat to 127.8.138.188:45473 (538 consecutive failures): Network error: Failed to ping master at 127.8.138.188:45473: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
W20260926 15:07:39.040920 9390 heartbeater.cc:644] Failed to heartbeat to 127.8.138.188:45473 (538 consecutive failures): Network error: Failed to ping master at 127.8.138.188:45473: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
I20260926 15:07:43.826602 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:07:43.826807 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:07:48.830377 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:07:48.830556 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:07:53.618429 12048 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:07:53.618705 12048 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:07:53.619647 12048 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:07:53.620867 11010 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260926 15:07:53.622148 11009 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
W20260926 15:07:53.622354 11010 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:07:53.622467 11010 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:07:53.622733 12048 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 15:07:53.833717 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:07:53.833936 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:07:54.767613 11012 maintenance_manager.cc:419] P 6315fb5099e1420b95b8d83b3e19375c: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1660 bytes on disk
I20260926 15:07:54.774155 11011 maintenance_manager.cc:643] P 6315fb5099e1420b95b8d83b3e19375c: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.006s sys 0.000s Metrics: {"dirs.queue_time_us":367,"dirs.run_cpu_time_us":87,"dirs.run_wall_time_us":89,"thread_start_us":600,"threads_started":2,"wal-append.queue_time_us":520}
I20260926 15:07:56.533941 12062 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:07:56.534129 12062 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:07:56.534786 12062 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:07:56.538057 11009 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
W20260926 15:07:56.538371 11010 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
I20260926 15:07:56.538492 11010 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:07:56.538797 12062 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 15:07:58.837052 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:07:58.837250 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:08:03.840555 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:08:03.840776 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:08:07.270114 12110 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 320cd09624624312bf1bcc0856c2874f)
I20260926 15:08:07.270340 12110 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6315fb5099e1420b95b8d83b3e19375c" member_type: VOTER last_known_addr { host: "127.8.138.190" port: 44281 } } peers { permanent_uuid: "be9958f2085242f7ab82ae933ceb1c42" member_type: VOTER last_known_addr { host: "127.8.138.189" port: 37105 } } peers { permanent_uuid: "320cd09624624312bf1bcc0856c2874f" member_type: VOTER last_known_addr { host: "127.8.138.188" port: 45473 } }
I20260926 15:08:07.271175 12110 leader_election.cc:288] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105), 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473)
W20260926 15:08:07.272440 11010 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260926 15:08:07.283236 11010 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer be9958f2085242f7ab82ae933ceb1c42 (127.8.138.189:37105): Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111)
W20260926 15:08:07.283569 11009 leader_election.cc:334] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 320cd09624624312bf1bcc0856c2874f (127.8.138.188:45473): Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
I20260926 15:08:07.283756 11009 leader_election.cc:302] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6315fb5099e1420b95b8d83b3e19375c; no voters: 320cd09624624312bf1bcc0856c2874f, be9958f2085242f7ab82ae933ceb1c42
I20260926 15:08:07.284080 12110 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6315fb5099e1420b95b8d83b3e19375c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260926 15:08:08.843833 10991 master_runner.cc:186] Error getting master registration for 127.8.138.189:37105: Network error: Client connection negotiation failed: client connection to 127.8.138.189:37105: connect: Connection refused (error 111),
I20260926 15:08:08.844074 10991 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 15:08:09.270031 9690 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
W20260926 15:08:09.270423 9690 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
W20260926 15:08:09.270582 9690 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
W20260926 15:08:09.270746 9690 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
F20260926 15:08:09.281590 9721 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
F20260926 15:08:09.281670 9716 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.281931 9700 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5bfdd1b3c4a247d58d08b75039b6124f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.282524 9702 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a320fb1edf9a4372928664c4263b213e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
*** Check failure stack trace: ***
*** Check failure stack trace: ***
F20260926 15:08:09.281670 9716 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.281931 9700 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5bfdd1b3c4a247d58d08b75039b6124f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.282524 9702 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a320fb1edf9a4372928664c4263b213e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
*** Aborted at 1790435289 (unix time) try "date -d @1790435289" if you are using GNU date ***
*** Check failure stack trace: ***
F20260926 15:08:09.281670 9716 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.281931 9700 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5bfdd1b3c4a247d58d08b75039b6124f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.282524 9702 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a320fb1edf9a4372928664c4263b213e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260926 15:08:09.281670 9716 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.281931 9700 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5bfdd1b3c4a247d58d08b75039b6124f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.282524 9702 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a320fb1edf9a4372928664c4263b213e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.283835 9713 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260926 15:08:09.281670 9716 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.281931 9700 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5bfdd1b3c4a247d58d08b75039b6124f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.282524 9702 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a320fb1edf9a4372928664c4263b213e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.283835 9713 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284251 9703 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e26e1e9fab934eab9d1b22663b28c712 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260926 15:08:09.281670 9716 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.281931 9700 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5bfdd1b3c4a247d58d08b75039b6124f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.282524 9702 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a320fb1edf9a4372928664c4263b213e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.283835 9713 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284251 9703 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e26e1e9fab934eab9d1b22663b28c712 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284402 9698 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_6c170e0fd90e42f08500c2a89216a4d3 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260926 15:08:09.281670 9716 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.281931 9700 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5bfdd1b3c4a247d58d08b75039b6124f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.282524 9702 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a320fb1edf9a4372928664c4263b213e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.283835 9713 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284251 9703 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e26e1e9fab934eab9d1b22663b28c712 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284402 9698 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_6c170e0fd90e42f08500c2a89216a4d3 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284729 9705 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260926 15:08:09.284739 9704 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a20572133e32441ab0bfe9e3fceed04a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260926 15:08:09.281670 9716 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.281931 9700 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5bfdd1b3c4a247d58d08b75039b6124f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.282524 9702 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a320fb1edf9a4372928664c4263b213e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.283835 9713 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284251 9703 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e26e1e9fab934eab9d1b22663b28c712 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284402 9698 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_6c170e0fd90e42f08500c2a89216a4d3 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284729 9705 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260926 15:08:09.284739 9704 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a20572133e32441ab0bfe9e3fceed04a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260926 15:08:09.281670 9716 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.281931 9700 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5bfdd1b3c4a247d58d08b75039b6124f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.282524 9702 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a320fb1edf9a4372928664c4263b213e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.283835 9713 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284251 9703 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e26e1e9fab934eab9d1b22663b28c712 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284402 9698 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_6c170e0fd90e42f08500c2a89216a4d3 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284729 9705 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260926 15:08:09.284739 9704 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a20572133e32441ab0bfe9e3fceed04a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.285004 9695 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_7257407810154f2aa325ed506a8a62d5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260926 15:08:09.281670 9716 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.281931 9700 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5bfdd1b3c4a247d58d08b75039b6124f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.282524 9702 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a320fb1edf9a4372928664c4263b213e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.283835 9713 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284251 9703 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e26e1e9fab934eab9d1b22663b28c712 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284402 9698 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_6c170e0fd90e42f08500c2a89216a4d3 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284729 9705 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260926 15:08:09.284739 9704 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a20572133e32441ab0bfe9e3fceed04a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.285004 9695 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_7257407810154f2aa325ed506a8a62d5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.285059 9709 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260926 15:08:09.281670 9716 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.281931 9700 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5bfdd1b3c4a247d58d08b75039b6124f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.282524 9702 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a320fb1edf9a4372928664c4263b213e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.283835 9713 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284251 9703 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e26e1e9fab934eab9d1b22663b28c712 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284402 9698 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_6c170e0fd90e42f08500c2a89216a4d3 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284729 9705 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260926 15:08:09.284739 9704 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a20572133e32441ab0bfe9e3fceed04a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.285004 9695 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_7257407810154f2aa325ed506a8a62d5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.285059 9709 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.286116 9710 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260926 15:08:09.281670 9716 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.281931 9700 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5bfdd1b3c4a247d58d08b75039b6124f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.282524 9702 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a320fb1edf9a4372928664c4263b213e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.283835 9713 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284251 9703 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e26e1e9fab934eab9d1b22663b28c712 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284402 9698 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_6c170e0fd90e42f08500c2a89216a4d3 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284729 9705 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260926 15:08:09.284739 9704 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a20572133e32441ab0bfe9e3fceed04a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.285004 9695 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_7257407810154f2aa325ed506a8a62d5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.285059 9709 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.286116 9710 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.286530 9696 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F2026Timed out: Error creating table default.Table_aa60eda53a3f4f338866d309cbde44a9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111) Check failed: _s.ok() Bad status:
*** Check failure stack trace: ***
F20260926 15:08:09.281670 9716 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.281931 9700 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5bfdd1b3c4a247d58d08b75039b6124f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.282524 9702 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a320fb1edf9a4372928664c4263b213e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.283835 9713 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284251 9703 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e26e1e9fab934eab9d1b22663b28c712 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284402 9698 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_6c170e0fd90e42f08500c2a89216a4d3 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284729 9705 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260926 15:08:09.284739 9704 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a20572133e32441ab0bfe9e3fceed04a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.285004 9695 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_7257407810154f2aa325ed506a8a62d5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.285059 9709 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.286116 9710 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.286530 9696 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F2026Timed out: Error creating table default.Table_aa60eda53a3f4f338866d309cbde44a9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111) Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_985d0971fb8a4d8eae0b2cb73ca99c6f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260926 15:08:09.281670 9716 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.281931 9700 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5bfdd1b3c4a247d58d08b75039b6124f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.282524 9702 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a320fb1edf9a4372928664c4263b213e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.283835 9713 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284251 9703 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e26e1e9fab934eab9d1b22663b28c712 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284402 9698 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_6c170e0fd90e42f08500c2a89216a4d3 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284729 9705 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260926 15:08:09.284739 9704 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a20572133e32441ab0bfe9e3fceed04a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.285004 9695 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_7257407810154f2aa325ed506a8a62d5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.285059 9709 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.286116 9710 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.286530 9696 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F2026Timed out: Error creating table default.Table_aa60eda53a3f4f338866d309cbde44a9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111) Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_985d0971fb8a4d8eae0b2cb73ca99c6f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.287648 9706 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.287738 9712 master-stress-test.cc:352] Check failed: _s.ok()
*** Check failure stack trace: ***
F20260926 15:08:09.281670 9716 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.281931 9700 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5bfdd1b3c4a247d58d08b75039b6124f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.282524 9702 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a320fb1edf9a4372928664c4263b213e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.283835 9713 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284251 9703 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e26e1e9fab934eab9d1b22663b28c712 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284402 9698 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_6c170e0fd90e42f08500c2a89216a4d3 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284729 9705 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260926 15:08:09.284739 9704 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a20572133e32441ab0bfe9e3fceed04a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.285004 9695 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_7257407810154f2aa325ed506a8a62d5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.285059 9709 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.286116 9710 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.286530 9696 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F2026Timed out: Error creating table default.Table_aa60eda53a3f4f338866d309cbde44a9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111) Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_985d0971fb8a4d8eae0b2cb73ca99c6f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.287648 9706 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.287738 9712 master-stress-test.cc:352] Check failed: _s.ok()
d status: F20260926 15:08:09.287901 9718 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260926 15:08:09.281670 9716 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.281931 9700 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5bfdd1b3c4a247d58d08b75039b6124f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.282524 9702 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a320fb1edf9a4372928664c4263b213e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.283835 9713 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284251 9703 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e26e1e9fab934eab9d1b22663b28c712 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284402 9698 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_6c170e0fd90e42f08500c2a89216a4d3 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284729 9705 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260926 15:08:09.284739 9704 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a20572133e32441ab0bfe9e3fceed04a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.285004 9695 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_7257407810154f2aa325ed506a8a62d5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.285059 9709 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.286116 9710 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.286530 9696 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F2026Timed out: Error creating table default.Table_aa60eda53a3f4f338866d309cbde44a9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111) Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_985d0971fb8a4d8eae0b2cb73ca99c6f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.287648 9706 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.287738 9712 master-stress-test.cc:352] Check failed: _s.ok()
d status: F20260926 15:08:09.287901 9718 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260926 15:08:09.281670 9716 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.281931 9700 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5bfdd1b3c4a247d58d08b75039b6124f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.282524 9702 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a320fb1edf9a4372928664c4263b213e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.283835 9713 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284251 9703 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e26e1e9fab934eab9d1b22663b28c712 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284402 9698 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_6c170e0fd90e42f08500c2a89216a4d3 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284729 9705 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260926 15:08:09.284739 9704 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a20572133e32441ab0bfe9e3fceed04a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.285004 9695 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_7257407810154f2aa325ed506a8a62d5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.285059 9709 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.286116 9710 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.286530 9696 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F2026Timed out: Error creating table default.Table_aa60eda53a3f4f338866d309cbde44a9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111) Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_985d0971fb8a4d8eae0b2cb73ca99c6f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.287648 9706 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.287738 9712 master-stress-test.cc:352] Check failed: _s.ok()
d status: F20260926 15:08:09.287901 9718 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.288152 9708 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260926 15:08:09.281670 9716 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.281931 9700 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5bfdd1b3c4a247d58d08b75039b6124f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.282524 9702 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a320fb1edf9a4372928664c4263b213e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.283835 9713 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284251 9703 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e26e1e9fab934eab9d1b22663b28c712 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284402 9698 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_6c170e0fd90e42f08500c2a89216a4d3 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284729 9705 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260926 15:08:09.284739 9704 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a20572133e32441ab0bfe9e3fceed04a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.285004 9695 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_7257407810154f2aa325ed506a8a62d5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.285059 9709 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.286116 9710 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.286530 9696 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F2026Timed out: Error creating table default.Table_aa60eda53a3f4f338866d309cbde44a9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111) Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_985d0971fb8a4d8eae0b2cb73ca99c6f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.287648 9706 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.287738 9712 master-stress-test.cc:352] Check failed: _s.ok()
d status: F20260926 15:08:09.287901 9718 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.288152 9708 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.289251 9699 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0e42d4f90b4e4529bb0a2ea0909cbadb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260926 15:08:09.281670 9716 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.281931 9700 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5bfdd1b3c4a247d58d08b75039b6124f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.282524 9702 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a320fb1edf9a4372928664c4263b213e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.283835 9713 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284251 9703 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e26e1e9fab934eab9d1b22663b28c712 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284402 9698 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_6c170e0fd90e42f08500c2a89216a4d3 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284729 9705 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260926 15:08:09.284739 9704 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a20572133e32441ab0bfe9e3fceed04a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.285004 9695 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_7257407810154f2aa325ed506a8a62d5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.285059 9709 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.286116 9710 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.286530 9696 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F2026Timed out: Error creating table default.Table_aa60eda53a3f4f338866d309cbde44a9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111) Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_985d0971fb8a4d8eae0b2cb73ca99c6f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.287648 9706 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.287738 9712 master-stress-test.cc:352] Check failed: _s.ok()
d status: F20260926 15:08:09.287901 9718 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.288152 9708 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.289251 9699 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0e42d4f90b4e4529bb0a2ea0909cbadb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.289397 9711 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260926 15:08:09.281670 9716 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.281931 9700 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5bfdd1b3c4a247d58d08b75039b6124f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.282524 9702 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a320fb1edf9a4372928664c4263b213e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.283835 9713 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284251 9703 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e26e1e9fab934eab9d1b22663b28c712 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284402 9698 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_6c170e0fd90e42f08500c2a89216a4d3 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284729 9705 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260926 15:08:09.284739 9704 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a20572133e32441ab0bfe9e3fceed04a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.285004 9695 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_7257407810154f2aa325ed506a8a62d5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.285059 9709 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.286116 9710 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.286530 9696 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F2026Timed out: Error creating table default.Table_aa60eda53a3f4f338866d309cbde44a9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111) Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_985d0971fb8a4d8eae0b2cb73ca99c6f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.287648 9706 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.287738 9712 master-stress-test.cc:352] Check failed: _s.ok()
d status: F20260926 15:08:09.287901 9718 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.288152 9708 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.289251 9699 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0e42d4f90b4e4529bb0a2ea0909cbadb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.289397 9711 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.290508 9715 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
*** Check failure stack trace: ***
PC: @ 0x0 (unknown)
F20260926 15:08:09.281670 9716 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.281931 9700 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5bfdd1b3c4a247d58d08b75039b6124f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.282524 9702 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a320fb1edf9a4372928664c4263b213e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.283835 9713 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284251 9703 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e26e1e9fab934eab9d1b22663b28c712 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284402 9698 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_6c170e0fd90e42f08500c2a89216a4d3 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284729 9705 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260926 15:08:09.284739 9704 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a20572133e32441ab0bfe9e3fceed04a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.285004 9695 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_7257407810154f2aa325ed506a8a62d5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.285059 9709 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.286116 9710 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.286530 9696 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F2026Timed out: Error creating table default.Table_aa60eda53a3f4f338866d309cbde44a9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111) Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_985d0971fb8a4d8eae0b2cb73ca99c6f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.287648 9706 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.287738 9712 master-stress-test.cc:352] Check failed: _s.ok()
d status: F20260926 15:08:09.287901 9718 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.288152 9708 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.289251 9699 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0e42d4f90b4e4529bb0a2ea0909cbadb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.289397 9711 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.290508 9715 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.290650 9707 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
*** Check failure stack trace: ***
*** SIGABRT (@0x3e80000222a) received by PID 8746 (TID 0x7fa0bb850700) from PID 8746; stack trace: ***
F20260926 15:08:09.281670 9716 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.281931 9700 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5bfdd1b3c4a247d58d08b75039b6124f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.282524 9702 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a320fb1edf9a4372928664c4263b213e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.283835 9713 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284251 9703 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e26e1e9fab934eab9d1b22663b28c712 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284402 9698 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_6c170e0fd90e42f08500c2a89216a4d3 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284729 9705 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260926 15:08:09.284739 9704 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a20572133e32441ab0bfe9e3fceed04a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.285004 9695 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_7257407810154f2aa325ed506a8a62d5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.285059 9709 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.286116 9710 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.286530 9696 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F2026Timed out: Error creating table default.Table_aa60eda53a3f4f338866d309cbde44a9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111) Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_985d0971fb8a4d8eae0b2cb73ca99c6f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.287648 9706 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.287738 9712 master-stress-test.cc:352] Check failed: _s.ok()
d status: F20260926 15:08:09.287901 9718 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.288152 9708 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.289251 9699 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0e42d4f90b4e4529bb0a2ea0909cbadb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.289397 9711 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.290508 9715 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.290650 9707 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.290988 9701 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_04fd049074c940dba0495d97eac1d272 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260926 15:08:09.281670 9716 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.281931 9700 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5bfdd1b3c4a247d58d08b75039b6124f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.282524 9702 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a320fb1edf9a4372928664c4263b213e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.283835 9713 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284251 9703 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e26e1e9fab934eab9d1b22663b28c712 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284402 9698 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_6c170e0fd90e42f08500c2a89216a4d3 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.284729 9705 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F20260926 15:08:09.284739 9704 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_a20572133e32441ab0bfe9e3fceed04a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.285004 9695 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_7257407810154f2aa325ed506a8a62d5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.285059 9709 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.286116 9710 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.286530 9696 master-stress-test.cc:298] Check failed: _s.ok() Bad status: F2026Timed out: Error creating table default.Table_aa60eda53a3f4f338866d309cbde44a9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111) Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_985d0971fb8a4d8eae0b2cb73ca99c6f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.287648 9706 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.287738 9712 master-stress-test.cc:352] Check failed: _s.ok()
d status: F20260926 15:08:09.287901 9718 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.288152 9708 master-stress-test.cc:329] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.289251 9699 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0e42d4f90b4e4529bb0a2ea0909cbadb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.289397 9711 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.290508 9715 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.290650 9707 master-stress-test.cc:392] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.290988 9701 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_04fd049074c940dba0495d97eac1d272 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)F20260926 15:08:09.290664 9717 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.8.138.190:44281,127.8.138.189:37105,127.8.138.188:45473, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.8.138.188:45473: connect: Connection refused (error 111)
*** Check failure stack trace: ***
@ 0x7fa0d5ef7a46 google::(anonymous namespace)::FailureSignalHandler() at ??:0
@ 0x7fa0da039980 (unknown) at ??:0
@ 0x7fa0d3225e87 gsignal at ??:0
@ 0x7fa0d32277f1 abort at ??:0
@ 0x7fa0d5ee99ad google::LogMessage::Fail() at ??:0
@ 0x7fa0d5eec88a google::LogMessage::SendToLog() at ??:0
@ 0x7fa0d5ee94de google::LogMessage::Flush() at ??:0
@ 0x7fa0d5eeaa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x405f4e kudu::MasterStressTest::ReplaceTabletThread() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/master-stress-test.cc:392
@ 0x444b44 std::__invoke_impl<>() at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/bits/invoke.h:73
@ 0x4449fd std::__invoke<>() at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/bits/invoke.h:95
@ 0x4449ba _ZNSt6thread8_InvokerISt5tupleIJMN4kudu16MasterStressTestEFvvEPNS2_26MasterStressTest_Test_TestEEEE9_M_invokeIJLm0ELm1EEEEDTclsr3stdE8__invokespcl10_S_declvalIXT_EEEEESt12_Index_tupleIJXspT_EEE at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/thread:244
@ 0x44496e std::thread::_Invoker<>::operator()() at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/thread:253
@ 0x4446a9 std::thread::_State_impl<>::_M_run() at /opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8/../../../../include/c++/8/thread:196
@ 0x7fa0de975a4f execute_native_thread_routine at ??:0
@ 0x7fa0da02e6db start_thread at ??:0
@ 0x7fa0d330861f clone at ??:0