Note: This is test shard 3 of 3.
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from CatalogManagerConfigurations/MasterStressTest
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/2
2026-05-03T19:53:08Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-05-03T19:53:08Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-task12vCql/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-task12vCql/thirdparty/src/hadoop-3.4.1/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/security.properties
2026-05-03 19:53:15: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-task12vCql/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-task12vCql/thirdparty/src/hadoop-3.4.1/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260503 19:53:26.437266 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-task12vCql/build/asan/bin/kudu
/tmp/dist-test-task12vCql/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.62:33821
--webserver_interface=127.0.105.62
--webserver_port=0
--builtin_ntp_servers=127.0.105.20:36687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203
--hive_metastore_uris=thrift://127.0.0.1:38395
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260503 19:53:26.783710 709 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260503 19:53:26.784116 709 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260503 19:53:26.784247 709 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260503 19:53:26.793195 709 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260503 19:53:26.793341 709 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260503 19:53:26.793454 709 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260503 19:53:26.793540 709 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260503 19:53:26.804366 709 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.105.20:36687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:38395
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.62:33821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.105.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 0ba7f54e15c9f673bf3e350f30b8d5f1a4037b79
build type FASTDEBUG
built by None at 03 May 2026 19:43:25 UTC on bdcb31816ec0
build id 11730
ASAN enabled
I20260503 19:53:26.806172 709 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260503 19:53:26.807989 709 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260503 19:53:26.817385 715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260503 19:53:26.817551 714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260503 19:53:26.818037 717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260503 19:53:26.819214 709 server_base.cc:1061] running on GCE node
I20260503 19:53:26.820210 709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260503 19:53:26.821877 709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260503 19:53:26.823114 709 hybrid_clock.cc:648] HybridClock initialized: now 1777838006823036 us; error 81 us; skew 500 ppm
I20260503 19:53:26.823526 709 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260503 19:53:26.825898 709 webserver.cc:492] Webserver started at http://127.0.105.62:42361/ using document root <none> and password file <none>
I20260503 19:53:26.826660 709 fs_manager.cc:362] Metadata directory not provided
I20260503 19:53:26.826756 709 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260503 19:53:26.827142 709 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260503 19:53:26.830164 709 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-0/data/instance:
uuid: "f3a96e537cec46df958468878509b9f9"
format_stamp: "Formatted at 2026-05-03 19:53:26 on dist-test-slave-gl77"
I20260503 19:53:26.830969 709 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-0/wal/instance:
uuid: "f3a96e537cec46df958468878509b9f9"
format_stamp: "Formatted at 2026-05-03 19:53:26 on dist-test-slave-gl77"
I20260503 19:53:26.836643 709 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260503 19:53:26.840408 723 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260503 19:53:26.841964 709 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260503 19:53:26.842185 709 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-0/data,/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-0/wal
uuid: "f3a96e537cec46df958468878509b9f9"
format_stamp: "Formatted at 2026-05-03 19:53:26 on dist-test-slave-gl77"
I20260503 19:53:26.842419 709 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260503 19:53:26.862682 709 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260503 19:53:26.864185 709 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260503 19:53:26.864526 709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260503 19:53:26.888151 709 rpc_server.cc:307] RPC server started. Bound to: 127.0.105.62:33821
I20260503 19:53:26.888164 774 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.105.62:33821 every 8 connection(s)
I20260503 19:53:26.889860 709 server_base.cc:1193] Dumped server information to /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-0/data/info.pb
I20260503 19:53:26.894194 775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:26.896329 420 external_mini_cluster.cc:1428] Started /tmp/dist-test-task12vCql/build/asan/bin/kudu as pid 709
I20260503 19:53:26.896616 420 external_mini_cluster.cc:1442] Reading /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-0/wal/instance
I20260503 19:53:26.899298 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-task12vCql/build/asan/bin/kudu
/tmp/dist-test-task12vCql/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.61:35345
--webserver_interface=127.0.105.61
--webserver_port=0
--builtin_ntp_servers=127.0.105.20:36687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203
--hive_metastore_uris=thrift://127.0.0.1:38395
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260503 19:53:26.900686 775 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } has no permanent_uuid. Determining permanent_uuid...
I20260503 19:53:26.923388 775 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } has no permanent_uuid. Determining permanent_uuid...
W20260503 19:53:26.924532 724 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.61:35345: connect: Connection refused (error 111)
W20260503 19:53:26.926255 775 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.61:35345: Network error: Client connection negotiation failed: client connection to 127.0.105.61:35345: connect: Connection refused (error 111)
I20260503 19:53:26.975678 775 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } attempt: 1
W20260503 19:53:26.978123 775 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.61:35345: Network error: Client connection negotiation failed: client connection to 127.0.105.61:35345: connect: Connection refused (error 111)
I20260503 19:53:27.046427 775 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } attempt: 2
W20260503 19:53:27.048601 775 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.61:35345: Network error: Client connection negotiation failed: client connection to 127.0.105.61:35345: connect: Connection refused (error 111)
I20260503 19:53:27.139910 775 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } attempt: 3
W20260503 19:53:27.142320 775 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.61:35345: Network error: Client connection negotiation failed: client connection to 127.0.105.61:35345: connect: Connection refused (error 111)
W20260503 19:53:27.258486 776 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260503 19:53:27.258900 776 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260503 19:53:27.259027 776 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260503 19:53:27.267684 776 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260503 19:53:27.267803 776 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260503 19:53:27.267865 776 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260503 19:53:27.267956 776 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260503 19:53:27.278934 776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.105.20:36687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:38395
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.61:35345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.105.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 0ba7f54e15c9f673bf3e350f30b8d5f1a4037b79
build type FASTDEBUG
built by None at 03 May 2026 19:43:25 UTC on bdcb31816ec0
build id 11730
ASAN enabled
I20260503 19:53:27.280611 776 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260503 19:53:27.282439 776 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20260503 19:53:27.285605 775 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } attempt: 4
W20260503 19:53:27.287678 775 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.61:35345: Network error: Client connection negotiation failed: client connection to 127.0.105.61:35345: connect: Connection refused (error 111)
W20260503 19:53:27.291702 784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260503 19:53:27.292176 785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260503 19:53:27.293159 787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260503 19:53:27.293712 776 server_base.cc:1061] running on GCE node
I20260503 19:53:27.294538 776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260503 19:53:27.296042 776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260503 19:53:27.297277 776 hybrid_clock.cc:648] HybridClock initialized: now 1777838007297250 us; error 48 us; skew 500 ppm
I20260503 19:53:27.297664 776 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260503 19:53:27.300016 776 webserver.cc:492] Webserver started at http://127.0.105.61:39263/ using document root <none> and password file <none>
I20260503 19:53:27.300798 776 fs_manager.cc:362] Metadata directory not provided
I20260503 19:53:27.300946 776 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260503 19:53:27.301388 776 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260503 19:53:27.303840 776 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data/instance:
uuid: "2ba485a545f44fdd8217c3f86c79d0ca"
format_stamp: "Formatted at 2026-05-03 19:53:27 on dist-test-slave-gl77"
I20260503 19:53:27.304651 776 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/wal/instance:
uuid: "2ba485a545f44fdd8217c3f86c79d0ca"
format_stamp: "Formatted at 2026-05-03 19:53:27 on dist-test-slave-gl77"
I20260503 19:53:27.310791 776 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260503 19:53:27.314539 793 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260503 19:53:27.316073 776 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260503 19:53:27.316284 776 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data,/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/wal
uuid: "2ba485a545f44fdd8217c3f86c79d0ca"
format_stamp: "Formatted at 2026-05-03 19:53:27 on dist-test-slave-gl77"
I20260503 19:53:27.316506 776 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260503 19:53:27.345867 776 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260503 19:53:27.346763 776 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260503 19:53:27.347055 776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260503 19:53:27.370195 776 rpc_server.cc:307] RPC server started. Bound to: 127.0.105.61:35345
I20260503 19:53:27.370234 844 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.105.61:35345 every 8 connection(s)
I20260503 19:53:27.371826 776 server_base.cc:1193] Dumped server information to /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data/info.pb
I20260503 19:53:27.376315 845 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:27.380076 420 external_mini_cluster.cc:1428] Started /tmp/dist-test-task12vCql/build/asan/bin/kudu as pid 776
I20260503 19:53:27.380317 420 external_mini_cluster.cc:1442] Reading /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/wal/instance
I20260503 19:53:27.381446 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-task12vCql/build/asan/bin/kudu
/tmp/dist-test-task12vCql/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.60:42203
--webserver_interface=127.0.105.60
--webserver_port=0
--builtin_ntp_servers=127.0.105.20:36687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203
--hive_metastore_uris=thrift://127.0.0.1:38395
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260503 19:53:27.382273 845 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } has no permanent_uuid. Determining permanent_uuid...
I20260503 19:53:27.397219 845 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } has no permanent_uuid. Determining permanent_uuid...
I20260503 19:53:27.406972 845 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } has no permanent_uuid. Determining permanent_uuid...
W20260503 19:53:27.408109 794 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
W20260503 19:53:27.409771 845 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.60:42203: Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:27.459133 845 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } attempt: 1
W20260503 19:53:27.461182 845 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.60:42203: Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:27.529515 845 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } attempt: 2
W20260503 19:53:27.531646 845 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.60:42203: Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:27.586999 775 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } attempt: 5
I20260503 19:53:27.594727 775 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } has no permanent_uuid. Determining permanent_uuid...
W20260503 19:53:27.596752 775 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.60:42203: Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:27.622921 845 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } attempt: 3
W20260503 19:53:27.625401 845 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.60:42203: Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:27.648005 775 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } attempt: 1
W20260503 19:53:27.650440 775 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.60:42203: Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:27.718698 775 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } attempt: 2
W20260503 19:53:27.720963 775 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.60:42203: Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
W20260503 19:53:27.737344 846 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260503 19:53:27.737686 846 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260503 19:53:27.737785 846 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260503 19:53:27.746901 846 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260503 19:53:27.747007 846 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260503 19:53:27.747117 846 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260503 19:53:27.747200 846 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260503 19:53:27.757817 846 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.105.20:36687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:38395
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.60:42203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.105.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 0ba7f54e15c9f673bf3e350f30b8d5f1a4037b79
build type FASTDEBUG
built by None at 03 May 2026 19:43:25 UTC on bdcb31816ec0
build id 11730
ASAN enabled
I20260503 19:53:27.759409 846 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260503 19:53:27.761116 846 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20260503 19:53:27.768638 845 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } attempt: 4
W20260503 19:53:27.770025 854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260503 19:53:27.770061 855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260503 19:53:27.770848 857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260503 19:53:27.771382 845 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.60:42203: Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:27.771589 846 server_base.cc:1061] running on GCE node
I20260503 19:53:27.772507 846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260503 19:53:27.774025 846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260503 19:53:27.775226 846 hybrid_clock.cc:648] HybridClock initialized: now 1777838007775188 us; error 33 us; skew 500 ppm
I20260503 19:53:27.775555 846 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260503 19:53:27.778215 846 webserver.cc:492] Webserver started at http://127.0.105.60:36851/ using document root <none> and password file <none>
I20260503 19:53:27.778928 846 fs_manager.cc:362] Metadata directory not provided
I20260503 19:53:27.779021 846 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260503 19:53:27.779309 846 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260503 19:53:27.781752 846 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/data/instance:
uuid: "b0cad11e600c48a3af9a3c56115696d5"
format_stamp: "Formatted at 2026-05-03 19:53:27 on dist-test-slave-gl77"
I20260503 19:53:27.782536 846 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/wal/instance:
uuid: "b0cad11e600c48a3af9a3c56115696d5"
format_stamp: "Formatted at 2026-05-03 19:53:27 on dist-test-slave-gl77"
I20260503 19:53:27.787808 846 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260503 19:53:27.791288 863 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260503 19:53:27.792619 846 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260503 19:53:27.792809 846 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/data,/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/wal
uuid: "b0cad11e600c48a3af9a3c56115696d5"
format_stamp: "Formatted at 2026-05-03 19:53:27 on dist-test-slave-gl77"
I20260503 19:53:27.793020 846 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260503 19:53:27.809311 846 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260503 19:53:27.810267 846 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260503 19:53:27.810556 846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260503 19:53:27.827244 775 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } attempt: 3
W20260503 19:53:27.829173 775 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.60:42203: Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:27.833833 846 rpc_server.cc:307] RPC server started. Bound to: 127.0.105.60:42203
I20260503 19:53:27.833850 914 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.105.60:42203 every 8 connection(s)
I20260503 19:53:27.835422 846 server_base.cc:1193] Dumped server information to /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/data/info.pb
I20260503 19:53:27.839828 915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:27.840919 420 external_mini_cluster.cc:1428] Started /tmp/dist-test-task12vCql/build/asan/bin/kudu as pid 846
I20260503 19:53:27.841126 420 external_mini_cluster.cc:1442] Reading /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/wal/instance
I20260503 19:53:27.846298 915 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } has no permanent_uuid. Determining permanent_uuid...
I20260503 19:53:27.872097 915 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } has no permanent_uuid. Determining permanent_uuid...
I20260503 19:53:27.880095 915 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } has no permanent_uuid. Determining permanent_uuid...
I20260503 19:53:27.892547 915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5: Bootstrap starting.
I20260503 19:53:27.896330 915 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:27.897846 915 log.cc:826] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5: Log is configured to *not* fsync() on all Append() calls
I20260503 19:53:27.901202 915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5: No bootstrap required, opened a new log
I20260503 19:53:27.907838 915 raft_consensus.cc:359] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:27.908149 915 raft_consensus.cc:385] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:27.908303 915 raft_consensus.cc:740] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0cad11e600c48a3af9a3c56115696d5, State: Initialized, Role: FOLLOWER
I20260503 19:53:27.909014 915 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:27.910101 923 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:27.910408 923 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260503 19:53:27.911239 915 sys_catalog.cc:565] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: configured and running, proceeding with master startup.
W20260503 19:53:27.923385 937 catalog_manager.cc:1568] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260503 19:53:27.923524 937 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260503 19:53:27.923257 846 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: OK, instance_id { permanent_uuid: "f3a96e537cec46df958468878509b9f9" instance_seqno: 1777838006881544 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260503 19:53:27.929893 846 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:53:28.006470 775 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } attempt: 4
I20260503 19:53:28.020778 775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9: Bootstrap starting.
I20260503 19:53:28.025759 775 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:28.028244 775 log.cc:826] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9: Log is configured to *not* fsync() on all Append() calls
I20260503 19:53:28.031453 775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9: No bootstrap required, opened a new log
I20260503 19:53:28.038698 775 raft_consensus.cc:359] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:28.039043 775 raft_consensus.cc:385] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:28.039220 775 raft_consensus.cc:740] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3a96e537cec46df958468878509b9f9, State: Initialized, Role: FOLLOWER
I20260503 19:53:28.039917 775 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:28.040942 941 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:28.041311 941 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260503 19:53:28.042222 775 sys_catalog.cc:565] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [sys.catalog]: configured and running, proceeding with master startup.
W20260503 19:53:28.062940 869 tablet.cc:2404] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260503 19:53:28.069466 956 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260503 19:53:28.070602 845 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } attempt: 5
I20260503 19:53:28.077003 709 master_runner.cc:186] Error getting master registration for 127.0.105.61:35345: OK, instance_id { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" instance_seqno: 1777838007363736 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260503 19:53:28.086144 709 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:53:28.087520 845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: Bootstrap starting.
I20260503 19:53:28.091753 845 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:28.093623 845 log.cc:826] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: Log is configured to *not* fsync() on all Append() calls
I20260503 19:53:28.096722 845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: No bootstrap required, opened a new log
I20260503 19:53:28.104357 845 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:28.104773 845 raft_consensus.cc:385] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:28.105262 845 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba485a545f44fdd8217c3f86c79d0ca, State: Initialized, Role: FOLLOWER
I20260503 19:53:28.106048 845 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:28.107635 958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:28.107965 958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: This master's current role is: FOLLOWER
I20260503 19:53:28.108990 845 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: configured and running, proceeding with master startup.
W20260503 19:53:28.117748 729 tablet.cc:2404] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260503 19:53:28.126169 973 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260503 19:53:28.152133 776 master_runner.cc:263] No leader master found from master 2ba485a545f44fdd8217c3f86c79d0ca
I20260503 19:53:28.152277 776 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:53:28.196672 923 raft_consensus.cc:493] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:53:28.197036 923 raft_consensus.cc:515] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:28.198987 923 leader_election.cc:290] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345)
I20260503 19:53:28.199832 750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b0cad11e600c48a3af9a3c56115696d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3a96e537cec46df958468878509b9f9" is_pre_election: true
I20260503 19:53:28.199851 820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b0cad11e600c48a3af9a3c56115696d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" is_pre_election: true
I20260503 19:53:28.200335 750 raft_consensus.cc:2468] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0cad11e600c48a3af9a3c56115696d5 in term 0.
I20260503 19:53:28.200352 820 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0cad11e600c48a3af9a3c56115696d5 in term 0.
I20260503 19:53:28.200953 864 leader_election.cc:304] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca, b0cad11e600c48a3af9a3c56115696d5; no voters:
I20260503 19:53:28.201381 923 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260503 19:53:28.201550 923 raft_consensus.cc:493] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260503 19:53:28.201655 923 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 0 FOLLOWER]: Advancing to term 1
I20260503 19:53:28.204851 923 raft_consensus.cc:515] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:28.205619 923 leader_election.cc:290] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 1 election: Requested vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345)
I20260503 19:53:28.206159 750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b0cad11e600c48a3af9a3c56115696d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3a96e537cec46df958468878509b9f9"
I20260503 19:53:28.206239 820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b0cad11e600c48a3af9a3c56115696d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ba485a545f44fdd8217c3f86c79d0ca"
I20260503 19:53:28.206410 820 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 0 FOLLOWER]: Advancing to term 1
I20260503 19:53:28.206385 750 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 0 FOLLOWER]: Advancing to term 1
I20260503 19:53:28.210263 820 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0cad11e600c48a3af9a3c56115696d5 in term 1.
I20260503 19:53:28.210533 750 raft_consensus.cc:2468] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0cad11e600c48a3af9a3c56115696d5 in term 1.
I20260503 19:53:28.211503 864 leader_election.cc:304] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9; no voters:
I20260503 19:53:28.211839 923 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260503 19:53:28.212369 923 raft_consensus.cc:697] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 1 LEADER]: Becoming Leader. State: Replica: b0cad11e600c48a3af9a3c56115696d5, State: Running, Role: LEADER
I20260503 19:53:28.212925 923 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:28.216626 974 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0cad11e600c48a3af9a3c56115696d5. Latest consensus state: current_term: 1 leader_uuid: "b0cad11e600c48a3af9a3c56115696d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:28.216960 974 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: This master's current role is: LEADER
I20260503 19:53:28.217885 976 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260503 19:53:28.220007 976 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260503 19:53:28.228416 820 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 1 FOLLOWER]: Refusing update from remote peer b0cad11e600c48a3af9a3c56115696d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260503 19:53:28.228499 750 raft_consensus.cc:1275] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 1 FOLLOWER]: Refusing update from remote peer b0cad11e600c48a3af9a3c56115696d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260503 19:53:28.229475 923 consensus_queue.cc:1048] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260503 19:53:28.229810 974 consensus_queue.cc:1048] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260503 19:53:28.239125 941 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0cad11e600c48a3af9a3c56115696d5. Latest consensus state: current_term: 1 leader_uuid: "b0cad11e600c48a3af9a3c56115696d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:28.239403 941 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260503 19:53:28.243899 978 mvcc.cc:204] Tried to move back new op lower bound from 7282024481693679616 to 7282024481645219840. Current Snapshot: MvccSnapshot[applied={T|T < 7282024481693679616}]
I20260503 19:53:28.249011 977 mvcc.cc:204] Tried to move back new op lower bound from 7282024481693679616 to 7282024481645219840. Current Snapshot: MvccSnapshot[applied={T|T < 7282024481693679616}]
I20260503 19:53:28.250346 941 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b0cad11e600c48a3af9a3c56115696d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:28.250564 941 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260503 19:53:28.251650 923 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b0cad11e600c48a3af9a3c56115696d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:28.251701 958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0cad11e600c48a3af9a3c56115696d5. Latest consensus state: current_term: 1 leader_uuid: "b0cad11e600c48a3af9a3c56115696d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:28.251904 923 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: This master's current role is: LEADER
I20260503 19:53:28.251930 958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: This master's current role is: FOLLOWER
I20260503 19:53:28.255129 923 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b0cad11e600c48a3af9a3c56115696d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:28.255391 923 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: This master's current role is: LEADER
I20260503 19:53:28.255968 976 catalog_manager.cc:1357] Generated new cluster ID: 636a1bc8fd6746489a9e892cd3f8ca43
I20260503 19:53:28.256065 976 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260503 19:53:28.259339 958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b0cad11e600c48a3af9a3c56115696d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:28.259542 958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: This master's current role is: FOLLOWER
I20260503 19:53:28.274564 976 catalog_manager.cc:1380] Generated new certificate authority record
I20260503 19:53:28.277274 976 catalog_manager.cc:1514] Loading token signing keys...
I20260503 19:53:28.294778 976 catalog_manager.cc:6044] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5: Generated new TSK 0
I20260503 19:53:28.295974 976 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260503 19:53:28.297361 976 catalog_manager.cc:1537] Loading latest processed Hive Metastore notification log event ID...
I20260503 19:53:28.298617 976 catalog_manager.cc:5893] Last processed Hive Metastore notification event ID: -1
I20260503 19:53:28.308126 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-task12vCql/build/asan/bin/kudu
/tmp/dist-test-task12vCql/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.105.1:0
--local_ip_for_outbound_sockets=127.0.105.1
--webserver_interface=127.0.105.1
--webserver_port=0
--tserver_master_addrs=127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203
--builtin_ntp_servers=127.0.105.20:36687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
W20260503 19:53:28.667491 987 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260503 19:53:28.667867 987 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260503 19:53:28.668030 987 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260503 19:53:28.677151 987 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260503 19:53:28.677414 987 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.105.1
I20260503 19:53:28.688020 987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.105.20:36687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.105.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.105.1
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.105.1
--log_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 0ba7f54e15c9f673bf3e350f30b8d5f1a4037b79
build type FASTDEBUG
built by None at 03 May 2026 19:43:25 UTC on bdcb31816ec0
build id 11730
ASAN enabled
I20260503 19:53:28.689702 987 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260503 19:53:28.691514 987 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260503 19:53:28.701696 992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260503 19:53:28.701714 993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260503 19:53:28.702767 995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260503 19:53:28.703676 987 server_base.cc:1061] running on GCE node
I20260503 19:53:28.704348 987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260503 19:53:28.705317 987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260503 19:53:28.706544 987 hybrid_clock.cc:648] HybridClock initialized: now 1777838008706494 us; error 45 us; skew 500 ppm
I20260503 19:53:28.706897 987 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260503 19:53:28.709204 987 webserver.cc:492] Webserver started at http://127.0.105.1:45185/ using document root <none> and password file <none>
I20260503 19:53:28.709961 987 fs_manager.cc:362] Metadata directory not provided
I20260503 19:53:28.710105 987 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260503 19:53:28.710456 987 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260503 19:53:28.712963 987 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-0/data/instance:
uuid: "b8594a7337c54dc58d701259120d361b"
format_stamp: "Formatted at 2026-05-03 19:53:28 on dist-test-slave-gl77"
I20260503 19:53:28.713804 987 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-0/wal/instance:
uuid: "b8594a7337c54dc58d701259120d361b"
format_stamp: "Formatted at 2026-05-03 19:53:28 on dist-test-slave-gl77"
I20260503 19:53:28.719501 987 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260503 19:53:28.723115 1001 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260503 19:53:28.724733 987 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260503 19:53:28.724934 987 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-0/data,/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-0/wal
uuid: "b8594a7337c54dc58d701259120d361b"
format_stamp: "Formatted at 2026-05-03 19:53:28 on dist-test-slave-gl77"
I20260503 19:53:28.725265 987 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260503 19:53:28.782006 987 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260503 19:53:28.782961 987 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260503 19:53:28.783274 987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260503 19:53:28.784235 987 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260503 19:53:28.785980 987 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260503 19:53:28.786114 987 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260503 19:53:28.786250 987 ts_tablet_manager.cc:616] Registered 0 tablets
I20260503 19:53:28.786339 987 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260503 19:53:28.795303 1024 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b0cad11e600c48a3af9a3c56115696d5)
I20260503 19:53:28.795902 1024 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:28.797201 1024 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
I20260503 19:53:28.797757 750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3a96e537cec46df958468878509b9f9" is_pre_election: true
I20260503 19:53:28.797892 890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b0cad11e600c48a3af9a3c56115696d5" is_pre_election: true
I20260503 19:53:28.798774 794 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:53:28.799520 1024 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260503 19:53:28.829775 987 rpc_server.cc:307] RPC server started. Bound to: 127.0.105.1:40289
I20260503 19:53:28.829888 1114 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.105.1:40289 every 8 connection(s)
I20260503 19:53:28.831327 987 server_base.cc:1193] Dumped server information to /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-0/data/info.pb
I20260503 19:53:28.839450 420 external_mini_cluster.cc:1428] Started /tmp/dist-test-task12vCql/build/asan/bin/kudu as pid 987
I20260503 19:53:28.839772 420 external_mini_cluster.cc:1442] Reading /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-0/wal/instance
I20260503 19:53:28.845875 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-task12vCql/build/asan/bin/kudu
/tmp/dist-test-task12vCql/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.105.2:0
--local_ip_for_outbound_sockets=127.0.105.2
--webserver_interface=127.0.105.2
--webserver_port=0
--tserver_master_addrs=127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203
--builtin_ntp_servers=127.0.105.20:36687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260503 19:53:28.871057 1116 heartbeater.cc:344] Connected to a master server at 127.0.105.62:33821
I20260503 19:53:28.871536 1116 heartbeater.cc:461] Registering TS with master...
I20260503 19:53:28.873524 1116 heartbeater.cc:507] Master 127.0.105.62:33821 requested a full tablet report, sending...
I20260503 19:53:28.875839 1115 heartbeater.cc:344] Connected to a master server at 127.0.105.60:42203
I20260503 19:53:28.876046 1115 heartbeater.cc:461] Registering TS with master...
I20260503 19:53:28.876348 740 ts_manager.cc:194] Registered new tserver with Master: b8594a7337c54dc58d701259120d361b (127.0.105.1:40289)
I20260503 19:53:28.876559 1115 heartbeater.cc:507] Master 127.0.105.60:42203 requested a full tablet report, sending...
I20260503 19:53:28.877708 1117 heartbeater.cc:344] Connected to a master server at 127.0.105.61:35345
I20260503 19:53:28.877877 1117 heartbeater.cc:461] Registering TS with master...
I20260503 19:53:28.878244 880 ts_manager.cc:194] Registered new tserver with Master: b8594a7337c54dc58d701259120d361b (127.0.105.1:40289)
I20260503 19:53:28.878401 1117 heartbeater.cc:507] Master 127.0.105.61:35345 requested a full tablet report, sending...
I20260503 19:53:28.879870 880 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.105.1:33297
I20260503 19:53:28.880072 810 ts_manager.cc:194] Registered new tserver with Master: b8594a7337c54dc58d701259120d361b (127.0.105.1:40289)
I20260503 19:53:29.075686 956 catalog_manager.cc:1269] Loaded cluster ID: 636a1bc8fd6746489a9e892cd3f8ca43
I20260503 19:53:29.075845 956 catalog_manager.cc:1562] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9: loading cluster ID for follower catalog manager: success
I20260503 19:53:29.080442 956 catalog_manager.cc:1584] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9: acquiring CA information for follower catalog manager: success
I20260503 19:53:29.083063 956 catalog_manager.cc:1612] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260503 19:53:29.132354 973 catalog_manager.cc:1269] Loaded cluster ID: 636a1bc8fd6746489a9e892cd3f8ca43
I20260503 19:53:29.132571 973 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: loading cluster ID for follower catalog manager: success
I20260503 19:53:29.137111 973 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: acquiring CA information for follower catalog manager: success
I20260503 19:53:29.139664 973 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260503 19:53:29.287679 1121 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260503 19:53:29.288107 1121 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260503 19:53:29.288239 1121 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260503 19:53:29.302982 1121 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260503 19:53:29.303268 1121 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.105.2
I20260503 19:53:29.321460 1121 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.105.20:36687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.105.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.105.2
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.105.2
--log_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 0ba7f54e15c9f673bf3e350f30b8d5f1a4037b79
build type FASTDEBUG
built by None at 03 May 2026 19:43:25 UTC on bdcb31816ec0
build id 11730
ASAN enabled
I20260503 19:53:29.323563 1121 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260503 19:53:29.325977 1121 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260503 19:53:29.338366 1132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260503 19:53:29.339655 1133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260503 19:53:29.347296 1135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260503 19:53:29.347779 1121 server_base.cc:1061] running on GCE node
I20260503 19:53:29.348824 1121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260503 19:53:29.350571 1121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260503 19:53:29.351922 1121 hybrid_clock.cc:648] HybridClock initialized: now 1777838009351849 us; error 62 us; skew 500 ppm
I20260503 19:53:29.352352 1121 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260503 19:53:29.355441 1121 webserver.cc:492] Webserver started at http://127.0.105.2:44813/ using document root <none> and password file <none>
I20260503 19:53:29.356585 1121 fs_manager.cc:362] Metadata directory not provided
I20260503 19:53:29.356729 1121 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260503 19:53:29.357219 1121 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260503 19:53:29.361285 1121 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-1/data/instance:
uuid: "fc902de5b2274103ab5e76f967503afb"
format_stamp: "Formatted at 2026-05-03 19:53:29 on dist-test-slave-gl77"
I20260503 19:53:29.362368 1121 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-1/wal/instance:
uuid: "fc902de5b2274103ab5e76f967503afb"
format_stamp: "Formatted at 2026-05-03 19:53:29 on dist-test-slave-gl77"
I20260503 19:53:29.370990 1121 fs_manager.cc:696] Time spent creating directory manager: real 0.008s user 0.007s sys 0.001s
I20260503 19:53:29.382541 1115 heartbeater.cc:499] Master 127.0.105.60:42203 was elected leader, sending a full tablet report...
I20260503 19:53:29.391427 1141 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260503 19:53:29.393733 1121 fs_manager.cc:730] Time spent opening block manager: real 0.020s user 0.005s sys 0.012s
I20260503 19:53:29.394013 1121 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-1/data,/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-1/wal
uuid: "fc902de5b2274103ab5e76f967503afb"
format_stamp: "Formatted at 2026-05-03 19:53:29 on dist-test-slave-gl77"
I20260503 19:53:29.394361 1121 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260503 19:53:29.401775 1142 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b0cad11e600c48a3af9a3c56115696d5)
I20260503 19:53:29.402074 1142 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:29.403426 750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3a96e537cec46df958468878509b9f9" is_pre_election: true
I20260503 19:53:29.404876 890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b0cad11e600c48a3af9a3c56115696d5" is_pre_election: true
I20260503 19:53:29.405627 794 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:53:29.411299 1142 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
I20260503 19:53:29.411798 1142 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260503 19:53:29.426163 1121 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260503 19:53:29.427125 1121 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260503 19:53:29.427412 1121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260503 19:53:29.428481 1121 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260503 19:53:29.430526 1121 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260503 19:53:29.430642 1121 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260503 19:53:29.430814 1121 ts_tablet_manager.cc:616] Registered 0 tablets
I20260503 19:53:29.430903 1121 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260503 19:53:29.474481 1121 rpc_server.cc:307] RPC server started. Bound to: 127.0.105.2:46339
I20260503 19:53:29.474505 1255 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.105.2:46339 every 8 connection(s)
I20260503 19:53:29.475958 1121 server_base.cc:1193] Dumped server information to /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-1/data/info.pb
I20260503 19:53:29.478231 420 external_mini_cluster.cc:1428] Started /tmp/dist-test-task12vCql/build/asan/bin/kudu as pid 1121
I20260503 19:53:29.478446 420 external_mini_cluster.cc:1442] Reading /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-1/wal/instance
I20260503 19:53:29.491222 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-task12vCql/build/asan/bin/kudu
/tmp/dist-test-task12vCql/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.105.3:0
--local_ip_for_outbound_sockets=127.0.105.3
--webserver_interface=127.0.105.3
--webserver_port=0
--tserver_master_addrs=127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203
--builtin_ntp_servers=127.0.105.20:36687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260503 19:53:29.528350 1256 heartbeater.cc:344] Connected to a master server at 127.0.105.60:42203
I20260503 19:53:29.528780 1256 heartbeater.cc:461] Registering TS with master...
I20260503 19:53:29.530386 1256 heartbeater.cc:507] Master 127.0.105.60:42203 requested a full tablet report, sending...
I20260503 19:53:29.532373 880 ts_manager.cc:194] Registered new tserver with Master: fc902de5b2274103ab5e76f967503afb (127.0.105.2:46339)
I20260503 19:53:29.533578 880 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.105.2:50777
I20260503 19:53:29.541869 1257 heartbeater.cc:344] Connected to a master server at 127.0.105.62:33821
I20260503 19:53:29.542071 1257 heartbeater.cc:461] Registering TS with master...
I20260503 19:53:29.542553 1257 heartbeater.cc:507] Master 127.0.105.62:33821 requested a full tablet report, sending...
I20260503 19:53:29.544317 740 ts_manager.cc:194] Registered new tserver with Master: fc902de5b2274103ab5e76f967503afb (127.0.105.2:46339)
I20260503 19:53:29.561137 1261 heartbeater.cc:344] Connected to a master server at 127.0.105.61:35345
I20260503 19:53:29.561335 1261 heartbeater.cc:461] Registering TS with master...
I20260503 19:53:29.561682 1261 heartbeater.cc:507] Master 127.0.105.61:35345 requested a full tablet report, sending...
I20260503 19:53:29.562891 810 ts_manager.cc:194] Registered new tserver with Master: fc902de5b2274103ab5e76f967503afb (127.0.105.2:46339)
I20260503 19:53:29.574200 1266 raft_consensus.cc:493] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b0cad11e600c48a3af9a3c56115696d5)
I20260503 19:53:29.574497 1266 raft_consensus.cc:515] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:29.576161 1266 leader_election.cc:290] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
I20260503 19:53:29.577168 820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f3a96e537cec46df958468878509b9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" is_pre_election: true
I20260503 19:53:29.577523 890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f3a96e537cec46df958468878509b9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b0cad11e600c48a3af9a3c56115696d5" is_pre_election: true
I20260503 19:53:29.578125 724 leader_election.cc:304] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3a96e537cec46df958468878509b9f9; no voters: 2ba485a545f44fdd8217c3f86c79d0ca, b0cad11e600c48a3af9a3c56115696d5
I20260503 19:53:29.578837 1266 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260503 19:53:29.947813 1260 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260503 19:53:29.948812 1260 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260503 19:53:29.949000 1260 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260503 19:53:29.960632 1260 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260503 19:53:29.960870 1260 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.105.3
I20260503 19:53:29.971858 1260 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.105.20:36687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.105.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.105.3
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.105.3
--log_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 0ba7f54e15c9f673bf3e350f30b8d5f1a4037b79
build type FASTDEBUG
built by None at 03 May 2026 19:43:25 UTC on bdcb31816ec0
build id 11730
ASAN enabled
I20260503 19:53:29.973661 1260 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260503 19:53:29.975483 1260 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260503 19:53:29.985400 1271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260503 19:53:29.985451 1272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260503 19:53:29.986130 1274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260503 19:53:29.986939 1260 server_base.cc:1061] running on GCE node
I20260503 19:53:29.987537 1260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260503 19:53:29.988529 1260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260503 19:53:29.989730 1260 hybrid_clock.cc:648] HybridClock initialized: now 1777838009989683 us; error 38 us; skew 500 ppm
I20260503 19:53:29.990101 1260 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260503 19:53:29.993150 1260 webserver.cc:492] Webserver started at http://127.0.105.3:38945/ using document root <none> and password file <none>
I20260503 19:53:29.993996 1260 fs_manager.cc:362] Metadata directory not provided
I20260503 19:53:29.994113 1260 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260503 19:53:29.994470 1260 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260503 19:53:29.997022 1260 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-2/data/instance:
uuid: "b68b1efd1cc44e15b8c69e730229dbdd"
format_stamp: "Formatted at 2026-05-03 19:53:29 on dist-test-slave-gl77"
I20260503 19:53:29.997959 1260 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-2/wal/instance:
uuid: "b68b1efd1cc44e15b8c69e730229dbdd"
format_stamp: "Formatted at 2026-05-03 19:53:29 on dist-test-slave-gl77"
I20260503 19:53:30.003854 1260 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260503 19:53:30.007634 1280 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260503 19:53:30.009218 1260 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260503 19:53:30.009433 1260 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-2/data,/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-2/wal
uuid: "b68b1efd1cc44e15b8c69e730229dbdd"
format_stamp: "Formatted at 2026-05-03 19:53:29 on dist-test-slave-gl77"
I20260503 19:53:30.009666 1260 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260503 19:53:30.014451 1281 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b0cad11e600c48a3af9a3c56115696d5)
I20260503 19:53:30.014667 1281 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:30.015502 1281 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
I20260503 19:53:30.016208 750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3a96e537cec46df958468878509b9f9" is_pre_election: true
I20260503 19:53:30.016227 890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b0cad11e600c48a3af9a3c56115696d5" is_pre_election: true
I20260503 19:53:30.017140 794 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:53:30.017554 1281 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260503 19:53:30.030995 1260 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260503 19:53:30.031986 1260 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260503 19:53:30.032294 1260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260503 19:53:30.033327 1260 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260503 19:53:30.035100 1260 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260503 19:53:30.035189 1260 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260503 19:53:30.035341 1260 ts_tablet_manager.cc:616] Registered 0 tablets
I20260503 19:53:30.035434 1260 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260503 19:53:30.037671 1256 heartbeater.cc:499] Master 127.0.105.60:42203 was elected leader, sending a full tablet report...
I20260503 19:53:30.079663 1260 rpc_server.cc:307] RPC server started. Bound to: 127.0.105.3:41575
I20260503 19:53:30.079720 1393 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.105.3:41575 every 8 connection(s)
I20260503 19:53:30.081219 1260 server_base.cc:1193] Dumped server information to /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-2/data/info.pb
I20260503 19:53:30.088060 420 external_mini_cluster.cc:1428] Started /tmp/dist-test-task12vCql/build/asan/bin/kudu as pid 1260
I20260503 19:53:30.088347 420 external_mini_cluster.cc:1442] Reading /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-2/wal/instance
I20260503 19:53:30.102095 1394 heartbeater.cc:344] Connected to a master server at 127.0.105.60:42203
I20260503 19:53:30.102259 1396 heartbeater.cc:344] Connected to a master server at 127.0.105.61:35345
I20260503 19:53:30.102563 1396 heartbeater.cc:461] Registering TS with master...
I20260503 19:53:30.102571 1394 heartbeater.cc:461] Registering TS with master...
I20260503 19:53:30.103683 1394 heartbeater.cc:507] Master 127.0.105.60:42203 requested a full tablet report, sending...
I20260503 19:53:30.103705 1396 heartbeater.cc:507] Master 127.0.105.61:35345 requested a full tablet report, sending...
I20260503 19:53:30.104068 1395 heartbeater.cc:344] Connected to a master server at 127.0.105.62:33821
I20260503 19:53:30.104249 1395 heartbeater.cc:461] Registering TS with master...
I20260503 19:53:30.104770 1395 heartbeater.cc:507] Master 127.0.105.62:33821 requested a full tablet report, sending...
I20260503 19:53:30.105679 880 ts_manager.cc:194] Registered new tserver with Master: b68b1efd1cc44e15b8c69e730229dbdd (127.0.105.3:41575)
I20260503 19:53:30.105696 810 ts_manager.cc:194] Registered new tserver with Master: b68b1efd1cc44e15b8c69e730229dbdd (127.0.105.3:41575)
I20260503 19:53:30.105934 740 ts_manager.cc:194] Registered new tserver with Master: b68b1efd1cc44e15b8c69e730229dbdd (127.0.105.3:41575)
I20260503 19:53:30.106884 880 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.105.3:56205
I20260503 19:53:30.117756 420 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20260503 19:53:30.152484 879 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45448:
name: "default.Table_f423a73d5c224651bb5eb676ee98c5e3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:30.152791 877 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45448:
name: "default.Table_dafc7a171f954df18eee7c2911f793f1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:30.153512 880 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45448:
name: "default.Table_15b3325c8812438786ade0fa5c759e21"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:30.153604 876 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45448:
name: "default.Table_cb2fc46da15e4d3cbc25e42aac22a26c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:30.154114 875 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45448:
name: "default.Table_1148c991089047a88d36862e99224cac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:30.152503 878 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45448:
name: "default.Table_db1c67410d564682adca12ef0e942ca2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:30.155867 874 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45448:
name: "default.Table_11964e32037841689422d2a453fff70f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:30.156041 873 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45448:
name: "default.Table_f91787a2979849d78ba6ba4daf0b4e70"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:30.156112 872 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45448:
name: "default.Table_0fc3df0b61324d7eac247468c05ce87f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:30.156839 871 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45448:
name: "default.Table_9c30ca5293344be3964460b578f9d6df"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260503 19:53:30.162637 877 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_dafc7a171f954df18eee7c2911f793f1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260503 19:53:30.165433 876 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_cb2fc46da15e4d3cbc25e42aac22a26c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260503 19:53:30.166725 880 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_15b3325c8812438786ade0fa5c759e21 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260503 19:53:30.183722 420 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-task12vCql/build/asan/bin/kudu pid 846
I20260503 19:53:30.185690 910 generic_service.cc:196] Checking for leaks (request via RPC)
I20260503 19:53:30.356478 420 external_mini_cluster.cc:1658] Killing /tmp/dist-test-task12vCql/build/asan/bin/kudu with pid 846
W20260503 19:53:30.377591 1404 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.105.60:42203)
W20260503 19:53:30.388135 1115 heartbeater.cc:646] Failed to heartbeat to 127.0.105.60:42203 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:30.526227 1446 raft_consensus.cc:493] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b0cad11e600c48a3af9a3c56115696d5)
I20260503 19:53:30.526456 1446 raft_consensus.cc:515] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:30.527137 1446 leader_election.cc:290] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
I20260503 19:53:30.527653 820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f3a96e537cec46df958468878509b9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" is_pre_election: true
W20260503 19:53:30.529722 724 leader_election.cc:336] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:30.529841 724 leader_election.cc:304] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3a96e537cec46df958468878509b9f9; no voters: 2ba485a545f44fdd8217c3f86c79d0ca, b0cad11e600c48a3af9a3c56115696d5
I20260503 19:53:30.530146 1446 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260503 19:53:30.541953 1256 heartbeater.cc:646] Failed to heartbeat to 127.0.105.60:42203 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:30.609048 1394 heartbeater.cc:499] Master 127.0.105.60:42203 was elected leader, sending a full tablet report...
W20260503 19:53:30.610532 1394 heartbeater.cc:646] Failed to heartbeat to 127.0.105.60:42203 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:30.630796 1451 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b0cad11e600c48a3af9a3c56115696d5)
I20260503 19:53:30.631000 1451 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:30.631764 1451 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
I20260503 19:53:30.632339 750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3a96e537cec46df958468878509b9f9" is_pre_election: true
I20260503 19:53:30.632644 750 raft_consensus.cc:2468] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ba485a545f44fdd8217c3f86c79d0ca in term 1.
I20260503 19:53:30.633566 794 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca, f3a96e537cec46df958468878509b9f9; no voters:
I20260503 19:53:30.633853 1451 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260503 19:53:30.633988 1451 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 1 FOLLOWER]: Starting leader election (detected failure of leader b0cad11e600c48a3af9a3c56115696d5)
I20260503 19:53:30.634109 1451 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 1 FOLLOWER]: Advancing to term 2
W20260503 19:53:30.634649 794 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:30.684057 1451 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:30.685261 1451 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 2 election: Requested vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
I20260503 19:53:30.687049 750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3a96e537cec46df958468878509b9f9"
I20260503 19:53:30.687521 750 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 1 FOLLOWER]: Advancing to term 2
W20260503 19:53:30.689301 794 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:30.784935 750 raft_consensus.cc:2468] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba485a545f44fdd8217c3f86c79d0ca in term 2.
I20260503 19:53:30.786454 794 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca, f3a96e537cec46df958468878509b9f9; no voters: b0cad11e600c48a3af9a3c56115696d5
I20260503 19:53:30.786903 1451 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 2 FOLLOWER]: Leader election won for term 2
I20260503 19:53:30.787350 1451 raft_consensus.cc:697] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 2 LEADER]: Becoming Leader. State: Replica: 2ba485a545f44fdd8217c3f86c79d0ca, State: Running, Role: LEADER
I20260503 19:53:30.787827 1451 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:30.789958 1454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ba485a545f44fdd8217c3f86c79d0ca. Latest consensus state: current_term: 2 leader_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:30.790359 1454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: This master's current role is: LEADER
I20260503 19:53:30.791101 1456 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260503 19:53:30.793898 1456 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260503 19:53:30.795004 1456 catalog_manager.cc:1269] Loaded cluster ID: 636a1bc8fd6746489a9e892cd3f8ca43
I20260503 19:53:30.795099 1456 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260503 19:53:30.796519 1456 catalog_manager.cc:1514] Loading token signing keys...
I20260503 19:53:30.797552 1456 catalog_manager.cc:6055] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: Loaded TSK: 0
I20260503 19:53:30.798758 1456 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260503 19:53:30.799819 1456 catalog_manager.cc:1537] Loading latest processed Hive Metastore notification log event ID...
I20260503 19:53:30.800853 1456 catalog_manager.cc:5893] Last processed Hive Metastore notification event ID: -1
I20260503 19:53:30.817785 810 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54616:
name: "default.Table_0fc3df0b61324d7eac247468c05ce87f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:30.818328 806 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54616:
name: "default.Table_f423a73d5c224651bb5eb676ee98c5e3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:30.818380 804 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54616:
name: "default.Table_db1c67410d564682adca12ef0e942ca2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:30.818719 803 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54616:
name: "default.Table_dafc7a171f954df18eee7c2911f793f1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:30.817646 809 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54616:
name: "default.Table_9c30ca5293344be3964460b578f9d6df"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:30.818663 808 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54616:
name: "default.Table_f91787a2979849d78ba6ba4daf0b4e70"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:30.818156 805 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54616:
name: "default.Table_15b3325c8812438786ade0fa5c759e21"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:30.818960 802 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54616:
name: "default.Table_cb2fc46da15e4d3cbc25e42aac22a26c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:30.818886 801 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54616:
name: "default.Table_1148c991089047a88d36862e99224cac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:30.817952 807 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54616:
name: "default.Table_11964e32037841689422d2a453fff70f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260503 19:53:30.830638 806 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_f423a73d5c224651bb5eb676ee98c5e3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260503 19:53:30.832047 809 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_9c30ca5293344be3964460b578f9d6df in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260503 19:53:30.833323 803 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_dafc7a171f954df18eee7c2911f793f1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260503 19:53:30.834367 810 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_0fc3df0b61324d7eac247468c05ce87f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260503 19:53:30.834812 802 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_cb2fc46da15e4d3cbc25e42aac22a26c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260503 19:53:30.834903 808 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_f91787a2979849d78ba6ba4daf0b4e70 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260503 19:53:30.835583 804 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_db1c67410d564682adca12ef0e942ca2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260503 19:53:30.837574 801 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_1148c991089047a88d36862e99224cac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260503 19:53:30.838495 807 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_11964e32037841689422d2a453fff70f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260503 19:53:30.838567 805 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_15b3325c8812438786ade0fa5c759e21 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260503 19:53:31.043608 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-task12vCql/build/asan/bin/kudu
/tmp/dist-test-task12vCql/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.60:42203
--webserver_interface=127.0.105.60
--webserver_port=36851
--builtin_ntp_servers=127.0.105.20:36687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203
--hive_metastore_uris=thrift://127.0.0.1:38395
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260503 19:53:31.370745 750 raft_consensus.cc:1275] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 2 FOLLOWER]: Refusing update from remote peer 2ba485a545f44fdd8217c3f86c79d0ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260503 19:53:31.372128 1462 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260503 19:53:31.406023 1464 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ba485a545f44fdd8217c3f86c79d0ca. Latest consensus state: current_term: 2 leader_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:31.406301 1464 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [sys.catalog]: This master's current role is: FOLLOWER
W20260503 19:53:31.414379 794 consensus_peers.cc:597] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca -> Peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Couldn't send request to peer b0cad11e600c48a3af9a3c56115696d5. Status: Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260503 19:53:31.419381 1463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:31.419667 1463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: This master's current role is: LEADER
I20260503 19:53:31.422384 1464 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:31.422817 1464 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260503 19:53:31.519378 806 catalog_manager.cc:2174] created HMS catalog entry for table [id=f8aedcc7a5c34071a86b38b46361455f]
I20260503 19:53:31.545382 806 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:54616:
table { table_name: "default.Table_f423a73d5c224651bb5eb676ee98c5e3" } new_table_name: "default.Table_90069e6755454566a407bbb659d59d9e" modify_external_catalogs: true
I20260503 19:53:31.640089 809 catalog_manager.cc:2174] created HMS catalog entry for table [id=8d8f5993b50b4a59aaa004b29cf7c6dd]
W20260503 19:53:31.674626 1460 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260503 19:53:31.675156 1460 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260503 19:53:31.675330 1460 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260503 19:53:31.688169 809 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:54616:
table { table_name: "default.Table_9c30ca5293344be3964460b578f9d6df" } new_table_name: "default.Table_90313d18e9a542119fe8e0e296d4396c" modify_external_catalogs: true
W20260503 19:53:31.688748 1460 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260503 19:53:31.688916 1460 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260503 19:53:31.689059 1460 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260503 19:53:31.689209 1460 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260503 19:53:31.704384 1328 tablet_service.cc:1511] Processing CreateTablet for tablet 6841c0c80f814208b598787466df21b7 (DEFAULT_TABLE table=default.table_9c30ca5293344be3964460b578f9d6df [id=8d8f5993b50b4a59aaa004b29cf7c6dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260503 19:53:31.706384 1328 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6841c0c80f814208b598787466df21b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:31.706799 1329 tablet_service.cc:1511] Processing CreateTablet for tablet 84654b8548be48478ea7c6dcf1f1a3f5 (DEFAULT_TABLE table=default.table_f423a73d5c224651bb5eb676ee98c5e3 [id=f8aedcc7a5c34071a86b38b46361455f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260503 19:53:31.707960 1329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84654b8548be48478ea7c6dcf1f1a3f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:31.711895 1049 tablet_service.cc:1511] Processing CreateTablet for tablet 6841c0c80f814208b598787466df21b7 (DEFAULT_TABLE table=default.table_9c30ca5293344be3964460b578f9d6df [id=8d8f5993b50b4a59aaa004b29cf7c6dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260503 19:53:31.713881 1049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6841c0c80f814208b598787466df21b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:31.712829 1050 tablet_service.cc:1511] Processing CreateTablet for tablet 84654b8548be48478ea7c6dcf1f1a3f5 (DEFAULT_TABLE table=default.table_f423a73d5c224651bb5eb676ee98c5e3 [id=f8aedcc7a5c34071a86b38b46361455f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260503 19:53:31.713766 1191 tablet_service.cc:1511] Processing CreateTablet for tablet 84654b8548be48478ea7c6dcf1f1a3f5 (DEFAULT_TABLE table=default.table_f423a73d5c224651bb5eb676ee98c5e3 [id=f8aedcc7a5c34071a86b38b46361455f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260503 19:53:31.714839 1050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84654b8548be48478ea7c6dcf1f1a3f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:31.715181 1190 tablet_service.cc:1511] Processing CreateTablet for tablet 6841c0c80f814208b598787466df21b7 (DEFAULT_TABLE table=default.table_9c30ca5293344be3964460b578f9d6df [id=8d8f5993b50b4a59aaa004b29cf7c6dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260503 19:53:31.715922 1191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84654b8548be48478ea7c6dcf1f1a3f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:31.716406 1190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6841c0c80f814208b598787466df21b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:31.741884 803 catalog_manager.cc:2174] created HMS catalog entry for table [id=da8af64ba75b4650858bb82728b61cd5]
I20260503 19:53:31.743458 1486 tablet_bootstrap.cc:492] T 84654b8548be48478ea7c6dcf1f1a3f5 P b68b1efd1cc44e15b8c69e730229dbdd: Bootstrap starting.
I20260503 19:53:31.748225 1486 tablet_bootstrap.cc:654] T 84654b8548be48478ea7c6dcf1f1a3f5 P b68b1efd1cc44e15b8c69e730229dbdd: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:31.749869 1486 log.cc:826] T 84654b8548be48478ea7c6dcf1f1a3f5 P b68b1efd1cc44e15b8c69e730229dbdd: Log is configured to *not* fsync() on all Append() calls
I20260503 19:53:31.750492 1460 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.105.20:36687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:38395
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.60:42203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.105.60
--webserver_port=36851
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 0ba7f54e15c9f673bf3e350f30b8d5f1a4037b79
build type FASTDEBUG
built by None at 03 May 2026 19:43:25 UTC on bdcb31816ec0
build id 11730
ASAN enabled
I20260503 19:53:31.751235 803 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54616:
name: "default.Table_b578e2eeb11a4b698a7b51e206e1d015"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:31.752910 1460 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260503 19:53:31.755597 1460 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20260503 19:53:31.765897 1486 tablet_bootstrap.cc:492] T 84654b8548be48478ea7c6dcf1f1a3f5 P b68b1efd1cc44e15b8c69e730229dbdd: No bootstrap required, opened a new log
I20260503 19:53:31.766335 1486 ts_tablet_manager.cc:1403] T 84654b8548be48478ea7c6dcf1f1a3f5 P b68b1efd1cc44e15b8c69e730229dbdd: Time spent bootstrapping tablet: real 0.023s user 0.006s sys 0.009s
I20260503 19:53:31.775846 1049 tablet_service.cc:1511] Processing CreateTablet for tablet ed5861fcf94c47199eb1a81d4702f19b (DEFAULT_TABLE table=default.table_dafc7a171f954df18eee7c2911f793f1 [id=da8af64ba75b4650858bb82728b61cd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260503 19:53:31.776384 1049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed5861fcf94c47199eb1a81d4702f19b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:31.776638 1486 raft_consensus.cc:359] T 84654b8548be48478ea7c6dcf1f1a3f5 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } } peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } }
I20260503 19:53:31.777186 1486 raft_consensus.cc:385] T 84654b8548be48478ea7c6dcf1f1a3f5 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:31.777305 1486 raft_consensus.cc:740] T 84654b8548be48478ea7c6dcf1f1a3f5 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b68b1efd1cc44e15b8c69e730229dbdd, State: Initialized, Role: FOLLOWER
I20260503 19:53:31.782176 1329 tablet_service.cc:1511] Processing CreateTablet for tablet ed5861fcf94c47199eb1a81d4702f19b (DEFAULT_TABLE table=default.table_dafc7a171f954df18eee7c2911f793f1 [id=da8af64ba75b4650858bb82728b61cd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260503 19:53:31.782327 1190 tablet_service.cc:1511] Processing CreateTablet for tablet ed5861fcf94c47199eb1a81d4702f19b (DEFAULT_TABLE table=default.table_dafc7a171f954df18eee7c2911f793f1 [id=da8af64ba75b4650858bb82728b61cd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260503 19:53:31.782776 1190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed5861fcf94c47199eb1a81d4702f19b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:31.782775 1329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed5861fcf94c47199eb1a81d4702f19b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:31.789674 1487 tablet_bootstrap.cc:492] T 84654b8548be48478ea7c6dcf1f1a3f5 P fc902de5b2274103ab5e76f967503afb: Bootstrap starting.
I20260503 19:53:31.790673 1488 tablet_bootstrap.cc:492] T 84654b8548be48478ea7c6dcf1f1a3f5 P b8594a7337c54dc58d701259120d361b: Bootstrap starting.
W20260503 19:53:31.792796 1491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260503 19:53:31.797672 1486 consensus_queue.cc:260] T 84654b8548be48478ea7c6dcf1f1a3f5 P b68b1efd1cc44e15b8c69e730229dbdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } } peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } }
I20260503 19:53:31.798081 1487 tablet_bootstrap.cc:654] T 84654b8548be48478ea7c6dcf1f1a3f5 P fc902de5b2274103ab5e76f967503afb: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:31.799831 1487 log.cc:826] T 84654b8548be48478ea7c6dcf1f1a3f5 P fc902de5b2274103ab5e76f967503afb: Log is configured to *not* fsync() on all Append() calls
I20260503 19:53:31.802317 1488 tablet_bootstrap.cc:654] T 84654b8548be48478ea7c6dcf1f1a3f5 P b8594a7337c54dc58d701259120d361b: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:31.804020 1488 log.cc:826] T 84654b8548be48478ea7c6dcf1f1a3f5 P b8594a7337c54dc58d701259120d361b: Log is configured to *not* fsync() on all Append() calls
I20260503 19:53:31.804548 1486 ts_tablet_manager.cc:1434] T 84654b8548be48478ea7c6dcf1f1a3f5 P b68b1efd1cc44e15b8c69e730229dbdd: Time spent starting tablet: real 0.038s user 0.010s sys 0.003s
I20260503 19:53:31.811022 1396 heartbeater.cc:499] Master 127.0.105.61:35345 was elected leader, sending a full tablet report...
I20260503 19:53:31.811192 1486 tablet_bootstrap.cc:492] T 6841c0c80f814208b598787466df21b7 P b68b1efd1cc44e15b8c69e730229dbdd: Bootstrap starting.
I20260503 19:53:31.813118 1486 tablet_bootstrap.cc:654] T 6841c0c80f814208b598787466df21b7 P b68b1efd1cc44e15b8c69e730229dbdd: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:31.814553 1487 tablet_bootstrap.cc:492] T 84654b8548be48478ea7c6dcf1f1a3f5 P fc902de5b2274103ab5e76f967503afb: No bootstrap required, opened a new log
I20260503 19:53:31.816202 1488 tablet_bootstrap.cc:492] T 84654b8548be48478ea7c6dcf1f1a3f5 P b8594a7337c54dc58d701259120d361b: No bootstrap required, opened a new log
I20260503 19:53:31.816725 1487 ts_tablet_manager.cc:1403] T 84654b8548be48478ea7c6dcf1f1a3f5 P fc902de5b2274103ab5e76f967503afb: Time spent bootstrapping tablet: real 0.027s user 0.013s sys 0.000s
I20260503 19:53:31.821063 1488 ts_tablet_manager.cc:1403] T 84654b8548be48478ea7c6dcf1f1a3f5 P b8594a7337c54dc58d701259120d361b: Time spent bootstrapping tablet: real 0.031s user 0.006s sys 0.006s
W20260503 19:53:31.821484 1492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260503 19:53:31.827086 1486 tablet_bootstrap.cc:492] T 6841c0c80f814208b598787466df21b7 P b68b1efd1cc44e15b8c69e730229dbdd: No bootstrap required, opened a new log
I20260503 19:53:31.827381 1486 ts_tablet_manager.cc:1403] T 6841c0c80f814208b598787466df21b7 P b68b1efd1cc44e15b8c69e730229dbdd: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.001s
I20260503 19:53:31.828512 1486 raft_consensus.cc:359] T 6841c0c80f814208b598787466df21b7 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:31.828814 1486 raft_consensus.cc:385] T 6841c0c80f814208b598787466df21b7 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:31.828939 1486 raft_consensus.cc:740] T 6841c0c80f814208b598787466df21b7 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b68b1efd1cc44e15b8c69e730229dbdd, State: Initialized, Role: FOLLOWER
W20260503 19:53:31.836360 1397 tablet.cc:2404] T 84654b8548be48478ea7c6dcf1f1a3f5 P b68b1efd1cc44e15b8c69e730229dbdd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260503 19:53:31.838682 1487 raft_consensus.cc:359] T 84654b8548be48478ea7c6dcf1f1a3f5 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } } peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } }
I20260503 19:53:31.841063 1487 raft_consensus.cc:385] T 84654b8548be48478ea7c6dcf1f1a3f5 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:31.829332 1486 consensus_queue.cc:260] T 6841c0c80f814208b598787466df21b7 P b68b1efd1cc44e15b8c69e730229dbdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:31.842387 1488 raft_consensus.cc:359] T 84654b8548be48478ea7c6dcf1f1a3f5 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } } peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } }
I20260503 19:53:31.843065 1487 raft_consensus.cc:740] T 84654b8548be48478ea7c6dcf1f1a3f5 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc902de5b2274103ab5e76f967503afb, State: Initialized, Role: FOLLOWER
I20260503 19:53:31.844115 1487 consensus_queue.cc:260] T 84654b8548be48478ea7c6dcf1f1a3f5 P fc902de5b2274103ab5e76f967503afb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } } peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } }
I20260503 19:53:31.844388 1488 raft_consensus.cc:385] T 84654b8548be48478ea7c6dcf1f1a3f5 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:31.844523 1488 raft_consensus.cc:740] T 84654b8548be48478ea7c6dcf1f1a3f5 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8594a7337c54dc58d701259120d361b, State: Initialized, Role: FOLLOWER
I20260503 19:53:31.845196 1486 ts_tablet_manager.cc:1434] T 6841c0c80f814208b598787466df21b7 P b68b1efd1cc44e15b8c69e730229dbdd: Time spent starting tablet: real 0.018s user 0.002s sys 0.000s
I20260503 19:53:31.845499 1488 consensus_queue.cc:260] T 84654b8548be48478ea7c6dcf1f1a3f5 P b8594a7337c54dc58d701259120d361b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } } peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } }
I20260503 19:53:31.847211 1117 heartbeater.cc:499] Master 127.0.105.61:35345 was elected leader, sending a full tablet report...
I20260503 19:53:31.848351 1486 tablet_bootstrap.cc:492] T ed5861fcf94c47199eb1a81d4702f19b P b68b1efd1cc44e15b8c69e730229dbdd: Bootstrap starting.
I20260503 19:53:31.850121 1488 ts_tablet_manager.cc:1434] T 84654b8548be48478ea7c6dcf1f1a3f5 P b8594a7337c54dc58d701259120d361b: Time spent starting tablet: real 0.029s user 0.014s sys 0.000s
I20260503 19:53:31.850248 1486 tablet_bootstrap.cc:654] T ed5861fcf94c47199eb1a81d4702f19b P b68b1efd1cc44e15b8c69e730229dbdd: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:31.850667 1488 tablet_bootstrap.cc:492] T 6841c0c80f814208b598787466df21b7 P b8594a7337c54dc58d701259120d361b: Bootstrap starting.
I20260503 19:53:31.852931 1488 tablet_bootstrap.cc:654] T 6841c0c80f814208b598787466df21b7 P b8594a7337c54dc58d701259120d361b: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:31.854157 1487 ts_tablet_manager.cc:1434] T 84654b8548be48478ea7c6dcf1f1a3f5 P fc902de5b2274103ab5e76f967503afb: Time spent starting tablet: real 0.037s user 0.017s sys 0.003s
I20260503 19:53:31.856170 1460 server_base.cc:1061] running on GCE node
I20260503 19:53:31.857625 1486 tablet_bootstrap.cc:492] T ed5861fcf94c47199eb1a81d4702f19b P b68b1efd1cc44e15b8c69e730229dbdd: No bootstrap required, opened a new log
I20260503 19:53:31.857693 1261 heartbeater.cc:499] Master 127.0.105.61:35345 was elected leader, sending a full tablet report...
I20260503 19:53:31.857841 1486 ts_tablet_manager.cc:1403] T ed5861fcf94c47199eb1a81d4702f19b P b68b1efd1cc44e15b8c69e730229dbdd: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20260503 19:53:31.858652 1486 raft_consensus.cc:359] T ed5861fcf94c47199eb1a81d4702f19b P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:31.858876 1486 raft_consensus.cc:385] T ed5861fcf94c47199eb1a81d4702f19b P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:31.858954 1486 raft_consensus.cc:740] T ed5861fcf94c47199eb1a81d4702f19b P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b68b1efd1cc44e15b8c69e730229dbdd, State: Initialized, Role: FOLLOWER
I20260503 19:53:31.859236 1487 tablet_bootstrap.cc:492] T 6841c0c80f814208b598787466df21b7 P fc902de5b2274103ab5e76f967503afb: Bootstrap starting.
I20260503 19:53:31.859266 1486 consensus_queue.cc:260] T ed5861fcf94c47199eb1a81d4702f19b P b68b1efd1cc44e15b8c69e730229dbdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:31.862154 1487 tablet_bootstrap.cc:654] T 6841c0c80f814208b598787466df21b7 P fc902de5b2274103ab5e76f967503afb: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:31.863646 810 catalog_manager.cc:2174] created HMS catalog entry for table [id=54840346b61f4174a481e91d1f745c71]
W20260503 19:53:31.866153 1498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260503 19:53:31.867362 1460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260503 19:53:31.867730 1488 tablet_bootstrap.cc:492] T 6841c0c80f814208b598787466df21b7 P b8594a7337c54dc58d701259120d361b: No bootstrap required, opened a new log
I20260503 19:53:31.867936 1488 ts_tablet_manager.cc:1403] T 6841c0c80f814208b598787466df21b7 P b8594a7337c54dc58d701259120d361b: Time spent bootstrapping tablet: real 0.017s user 0.014s sys 0.000s
I20260503 19:53:31.872795 1486 ts_tablet_manager.cc:1434] T ed5861fcf94c47199eb1a81d4702f19b P b68b1efd1cc44e15b8c69e730229dbdd: Time spent starting tablet: real 0.015s user 0.002s sys 0.000s
I20260503 19:53:31.873468 810 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:54616 (request call id 24) took 1056 ms. Trace:
I20260503 19:53:31.873746 810 rpcz_store.cc:276] 0503 19:53:30.816843 (+ 0us) service_pool.cc:167] Inserting onto call queue
0503 19:53:30.816949 (+ 106us) service_pool.cc:224] Handling call
0503 19:53:30.834641 (+ 17692us) catalog_manager.cc:2101] Acquired catalog manager lock
0503 19:53:30.837038 (+ 2397us) catalog_manager.cc:2147] Created new table and tablet info
0503 19:53:31.863603 (+1026565us) catalog_manager.cc:2173] Created new table in HMS catalog
0503 19:53:31.873032 (+ 9429us) catalog_manager.cc:2207] Wrote table and tablets to system table
0503 19:53:31.873230 (+ 198us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0503 19:53:31.873390 (+ 160us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0503 19:53:31.864586 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0503 19:53:31.864755 (+ 169us) write_op.cc:432] Acquiring schema lock in shared mode
0503 19:53:31.864765 (+ 10us) write_op.cc:435] Acquired schema lock
0503 19:53:31.864769 (+ 4us) tablet.cc:662] Decoding operations
0503 19:53:31.864894 (+ 125us) write_op.cc:620] Acquiring the partition lock for write op
0503 19:53:31.864914 (+ 20us) write_op.cc:641] Partition lock acquired for write op
0503 19:53:31.864922 (+ 8us) tablet.cc:685] Acquiring locks for 2 operations
0503 19:53:31.865028 (+ 106us) tablet.cc:701] Row locks acquired
0503 19:53:31.865032 (+ 4us) write_op.cc:260] PREPARE: finished
0503 19:53:31.865071 (+ 39us) write_op.cc:270] Start()
0503 19:53:31.865180 (+ 109us) write_op.cc:276] Timestamp: P: 1777838011864965 usec, L: 0
0503 19:53:31.865184 (+ 4us) op_driver.cc:348] REPLICATION: starting
0503 19:53:31.865437 (+ 253us) log.cc:844] Serialized 432 byte log entry
0503 19:53:31.871863 (+ 6426us) op_driver.cc:464] REPLICATION: finished
0503 19:53:31.872065 (+ 202us) write_op.cc:301] APPLY: starting
0503 19:53:31.872086 (+ 21us) tablet.cc:1367] starting BulkCheckPresence
0503 19:53:31.872141 (+ 55us) tablet.cc:1370] finished BulkCheckPresence
0503 19:53:31.872191 (+ 50us) tablet.cc:1372] starting ApplyRowOperation cycle
0503 19:53:31.872328 (+ 137us) tablet.cc:1383] finished ApplyRowOperation cycle
0503 19:53:31.872387 (+ 59us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0503 19:53:31.872399 (+ 12us) write_op.cc:312] APPLY: finished
0503 19:53:31.872499 (+ 100us) log.cc:844] Serialized 28 byte log entry
0503 19:53:31.872571 (+ 72us) write_op.cc:489] Releasing partition, row and schema locks
0503 19:53:31.872596 (+ 25us) write_op.cc:454] Released schema lock
0503 19:53:31.872604 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0503 19:53:31.872690 (+ 86us) write_op.cc:489] Releasing partition, row and schema locks
0503 19:53:31.872697 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":904370,"Hive Metastore.run_cpu_time_us":347,"Hive Metastore.run_wall_time_us":121990,"spinlock_wait_cycles":9344,"child_traces":[["op",{"apply.queue_time_us":134,"apply.run_cpu_time_us":614,"apply.run_wall_time_us":613,"num_ops":2,"prepare.queue_time_us":107,"prepare.run_cpu_time_us":1039,"prepare.run_wall_time_us":1073,"raft.queue_time_us":1227,"raft.run_cpu_time_us":300,"raft.run_wall_time_us":300,"replication_time_us":6631}]]}
I20260503 19:53:31.893986 1050 tablet_service.cc:1511] Processing CreateTablet for tablet 47b6e48ca8c44c7e8bda3c62201b9dd1 (DEFAULT_TABLE table=default.table_0fc3df0b61324d7eac247468c05ce87f [id=54840346b61f4174a481e91d1f745c71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260503 19:53:31.894481 1050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47b6e48ca8c44c7e8bda3c62201b9dd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:31.894603 1191 tablet_service.cc:1511] Processing CreateTablet for tablet 47b6e48ca8c44c7e8bda3c62201b9dd1 (DEFAULT_TABLE table=default.table_0fc3df0b61324d7eac247468c05ce87f [id=54840346b61f4174a481e91d1f745c71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260503 19:53:31.894999 1191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47b6e48ca8c44c7e8bda3c62201b9dd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:31.897452 1329 tablet_service.cc:1511] Processing CreateTablet for tablet 47b6e48ca8c44c7e8bda3c62201b9dd1 (DEFAULT_TABLE table=default.table_0fc3df0b61324d7eac247468c05ce87f [id=54840346b61f4174a481e91d1f745c71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260503 19:53:31.897846 1329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47b6e48ca8c44c7e8bda3c62201b9dd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:31.902827 1487 tablet_bootstrap.cc:492] T 6841c0c80f814208b598787466df21b7 P fc902de5b2274103ab5e76f967503afb: No bootstrap required, opened a new log
I20260503 19:53:31.903004 1487 ts_tablet_manager.cc:1403] T 6841c0c80f814208b598787466df21b7 P fc902de5b2274103ab5e76f967503afb: Time spent bootstrapping tablet: real 0.044s user 0.013s sys 0.011s
I20260503 19:53:31.904042 1487 raft_consensus.cc:359] T 6841c0c80f814208b598787466df21b7 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:31.904253 1487 raft_consensus.cc:385] T 6841c0c80f814208b598787466df21b7 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:31.904356 1487 raft_consensus.cc:740] T 6841c0c80f814208b598787466df21b7 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc902de5b2274103ab5e76f967503afb, State: Initialized, Role: FOLLOWER
I20260503 19:53:31.904690 1487 consensus_queue.cc:260] T 6841c0c80f814208b598787466df21b7 P fc902de5b2274103ab5e76f967503afb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:31.909217 1487 ts_tablet_manager.cc:1434] T 6841c0c80f814208b598787466df21b7 P fc902de5b2274103ab5e76f967503afb: Time spent starting tablet: real 0.006s user 0.002s sys 0.001s
I20260503 19:53:31.912744 1486 tablet_bootstrap.cc:492] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b68b1efd1cc44e15b8c69e730229dbdd: Bootstrap starting.
I20260503 19:53:31.913779 1487 tablet_bootstrap.cc:492] T ed5861fcf94c47199eb1a81d4702f19b P fc902de5b2274103ab5e76f967503afb: Bootstrap starting.
I20260503 19:53:31.914588 1486 tablet_bootstrap.cc:654] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b68b1efd1cc44e15b8c69e730229dbdd: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:31.915571 1487 tablet_bootstrap.cc:654] T ed5861fcf94c47199eb1a81d4702f19b P fc902de5b2274103ab5e76f967503afb: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:31.916565 1486 tablet_bootstrap.cc:492] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b68b1efd1cc44e15b8c69e730229dbdd: No bootstrap required, opened a new log
I20260503 19:53:31.916723 1486 ts_tablet_manager.cc:1403] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b68b1efd1cc44e15b8c69e730229dbdd: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260503 19:53:31.917625 1486 raft_consensus.cc:359] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:31.917833 1486 raft_consensus.cc:385] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:31.917929 1486 raft_consensus.cc:740] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b68b1efd1cc44e15b8c69e730229dbdd, State: Initialized, Role: FOLLOWER
I20260503 19:53:31.918187 1486 consensus_queue.cc:260] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b68b1efd1cc44e15b8c69e730229dbdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:31.918493 1488 raft_consensus.cc:359] T 6841c0c80f814208b598787466df21b7 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:31.919613 1486 ts_tablet_manager.cc:1434] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b68b1efd1cc44e15b8c69e730229dbdd: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260503 19:53:31.920418 1488 raft_consensus.cc:385] T 6841c0c80f814208b598787466df21b7 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:31.921175 1488 raft_consensus.cc:740] T 6841c0c80f814208b598787466df21b7 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8594a7337c54dc58d701259120d361b, State: Initialized, Role: FOLLOWER
I20260503 19:53:31.921571 1488 consensus_queue.cc:260] T 6841c0c80f814208b598787466df21b7 P b8594a7337c54dc58d701259120d361b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:31.922744 1488 ts_tablet_manager.cc:1434] T 6841c0c80f814208b598787466df21b7 P b8594a7337c54dc58d701259120d361b: Time spent starting tablet: real 0.055s user 0.002s sys 0.000s
I20260503 19:53:31.925318 1488 tablet_bootstrap.cc:492] T ed5861fcf94c47199eb1a81d4702f19b P b8594a7337c54dc58d701259120d361b: Bootstrap starting.
I20260503 19:53:31.927131 1488 tablet_bootstrap.cc:654] T ed5861fcf94c47199eb1a81d4702f19b P b8594a7337c54dc58d701259120d361b: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:31.934312 1460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260503 19:53:31.941819 1460 hybrid_clock.cc:648] HybridClock initialized: now 1777838011941644 us; error 177 us; skew 500 ppm
I20260503 19:53:31.942376 1460 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260503 19:53:31.944072 810 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:54616:
table { table_name: "default.Table_dafc7a171f954df18eee7c2911f793f1" } new_table_name: "default.Table_d2f4bbb38f1d4d0191643de99eeb35b6" modify_external_catalogs: true
I20260503 19:53:31.944590 1488 tablet_bootstrap.cc:492] T ed5861fcf94c47199eb1a81d4702f19b P b8594a7337c54dc58d701259120d361b: No bootstrap required, opened a new log
I20260503 19:53:31.944811 1488 ts_tablet_manager.cc:1403] T ed5861fcf94c47199eb1a81d4702f19b P b8594a7337c54dc58d701259120d361b: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.000s
I20260503 19:53:31.945681 1488 raft_consensus.cc:359] T ed5861fcf94c47199eb1a81d4702f19b P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:31.945883 1488 raft_consensus.cc:385] T ed5861fcf94c47199eb1a81d4702f19b P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:31.945988 1488 raft_consensus.cc:740] T ed5861fcf94c47199eb1a81d4702f19b P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8594a7337c54dc58d701259120d361b, State: Initialized, Role: FOLLOWER
I20260503 19:53:31.946368 1488 consensus_queue.cc:260] T ed5861fcf94c47199eb1a81d4702f19b P b8594a7337c54dc58d701259120d361b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:31.947170 1488 ts_tablet_manager.cc:1434] T ed5861fcf94c47199eb1a81d4702f19b P b8594a7337c54dc58d701259120d361b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260503 19:53:31.947314 1487 tablet_bootstrap.cc:492] T ed5861fcf94c47199eb1a81d4702f19b P fc902de5b2274103ab5e76f967503afb: No bootstrap required, opened a new log
I20260503 19:53:31.947741 1488 tablet_bootstrap.cc:492] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b8594a7337c54dc58d701259120d361b: Bootstrap starting.
I20260503 19:53:31.949461 1488 tablet_bootstrap.cc:654] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b8594a7337c54dc58d701259120d361b: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:31.950517 1487 ts_tablet_manager.cc:1403] T ed5861fcf94c47199eb1a81d4702f19b P fc902de5b2274103ab5e76f967503afb: Time spent bootstrapping tablet: real 0.037s user 0.007s sys 0.004s
I20260503 19:53:31.951566 1487 raft_consensus.cc:359] T ed5861fcf94c47199eb1a81d4702f19b P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:31.951820 1487 raft_consensus.cc:385] T ed5861fcf94c47199eb1a81d4702f19b P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:31.951918 1487 raft_consensus.cc:740] T ed5861fcf94c47199eb1a81d4702f19b P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc902de5b2274103ab5e76f967503afb, State: Initialized, Role: FOLLOWER
I20260503 19:53:31.952293 1487 consensus_queue.cc:260] T ed5861fcf94c47199eb1a81d4702f19b P fc902de5b2274103ab5e76f967503afb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:31.959611 1487 ts_tablet_manager.cc:1434] T ed5861fcf94c47199eb1a81d4702f19b P fc902de5b2274103ab5e76f967503afb: Time spent starting tablet: real 0.009s user 0.002s sys 0.001s
I20260503 19:53:31.963459 1487 tablet_bootstrap.cc:492] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P fc902de5b2274103ab5e76f967503afb: Bootstrap starting.
I20260503 19:53:31.966146 1487 tablet_bootstrap.cc:654] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P fc902de5b2274103ab5e76f967503afb: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:31.967914 1488 tablet_bootstrap.cc:492] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b8594a7337c54dc58d701259120d361b: No bootstrap required, opened a new log
I20260503 19:53:31.968158 1488 ts_tablet_manager.cc:1403] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b8594a7337c54dc58d701259120d361b: Time spent bootstrapping tablet: real 0.021s user 0.004s sys 0.000s
I20260503 19:53:31.969341 1488 raft_consensus.cc:359] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:31.969714 1488 raft_consensus.cc:385] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:31.969875 1488 raft_consensus.cc:740] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8594a7337c54dc58d701259120d361b, State: Initialized, Role: FOLLOWER
I20260503 19:53:31.970265 1488 consensus_queue.cc:260] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b8594a7337c54dc58d701259120d361b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:31.971189 1488 ts_tablet_manager.cc:1434] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b8594a7337c54dc58d701259120d361b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260503 19:53:31.971724 1460 webserver.cc:492] Webserver started at http://127.0.105.60:36851/ using document root <none> and password file <none>
I20260503 19:53:31.972908 1460 fs_manager.cc:362] Metadata directory not provided
I20260503 19:53:31.973073 1460 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260503 19:53:31.973420 1487 tablet_bootstrap.cc:492] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P fc902de5b2274103ab5e76f967503afb: No bootstrap required, opened a new log
I20260503 19:53:31.973675 1487 ts_tablet_manager.cc:1403] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P fc902de5b2274103ab5e76f967503afb: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.007s
I20260503 19:53:31.974839 1487 raft_consensus.cc:359] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:31.975116 1487 raft_consensus.cc:385] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:31.975266 1487 raft_consensus.cc:740] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc902de5b2274103ab5e76f967503afb, State: Initialized, Role: FOLLOWER
I20260503 19:53:31.975569 1487 consensus_queue.cc:260] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P fc902de5b2274103ab5e76f967503afb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:31.978072 1487 ts_tablet_manager.cc:1434] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P fc902de5b2274103ab5e76f967503afb: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260503 19:53:31.984525 1460 fs_manager.cc:714] Time spent opening directory manager: real 0.009s user 0.005s sys 0.005s
I20260503 19:53:31.990765 1515 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260503 19:53:31.992569 1460 fs_manager.cc:730] Time spent opening block manager: real 0.006s user 0.000s sys 0.005s
I20260503 19:53:31.992789 1460 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/data,/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/wal
uuid: "b0cad11e600c48a3af9a3c56115696d5"
format_stamp: "Formatted at 2026-05-03 19:53:27 on dist-test-slave-gl77"
I20260503 19:53:31.993836 1460 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260503 19:53:31.996500 1501 raft_consensus.cc:493] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:53:31.996765 1501 raft_consensus.cc:515] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:31.998898 1501 leader_election.cc:290] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b8594a7337c54dc58d701259120d361b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc902de5b2274103ab5e76f967503afb (127.0.105.2:46339), b68b1efd1cc44e15b8c69e730229dbdd (127.0.105.3:41575)
I20260503 19:53:32.025552 1211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b6e48ca8c44c7e8bda3c62201b9dd1" candidate_uuid: "b8594a7337c54dc58d701259120d361b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc902de5b2274103ab5e76f967503afb" is_pre_election: true
I20260503 19:53:32.025622 1349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b6e48ca8c44c7e8bda3c62201b9dd1" candidate_uuid: "b8594a7337c54dc58d701259120d361b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" is_pre_election: true
I20260503 19:53:32.026124 1211 raft_consensus.cc:2468] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8594a7337c54dc58d701259120d361b in term 0.
I20260503 19:53:32.026171 1349 raft_consensus.cc:2468] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8594a7337c54dc58d701259120d361b in term 0.
I20260503 19:53:32.027040 1005 leader_election.cc:304] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b8594a7337c54dc58d701259120d361b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b68b1efd1cc44e15b8c69e730229dbdd, b8594a7337c54dc58d701259120d361b; no voters:
I20260503 19:53:32.028246 1501 raft_consensus.cc:2804] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260503 19:53:32.028427 1501 raft_consensus.cc:493] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260503 19:53:32.028537 1501 raft_consensus.cc:3060] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Advancing to term 1
I20260503 19:53:32.031599 1501 raft_consensus.cc:515] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b8594a7337c54dc58d701259120d361b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.032760 1211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b6e48ca8c44c7e8bda3c62201b9dd1" candidate_uuid: "b8594a7337c54dc58d701259120d361b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc902de5b2274103ab5e76f967503afb"
I20260503 19:53:32.033000 1211 raft_consensus.cc:3060] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Advancing to term 1
I20260503 19:53:32.036238 1211 raft_consensus.cc:2468] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P fc902de5b2274103ab5e76f967503afb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8594a7337c54dc58d701259120d361b in term 1.
I20260503 19:53:32.036799 1005 leader_election.cc:304] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b8594a7337c54dc58d701259120d361b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8594a7337c54dc58d701259120d361b, fc902de5b2274103ab5e76f967503afb; no voters:
I20260503 19:53:32.037485 1501 leader_election.cc:290] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b8594a7337c54dc58d701259120d361b [CANDIDATE]: Term 1 election: Requested vote from peers fc902de5b2274103ab5e76f967503afb (127.0.105.2:46339), b68b1efd1cc44e15b8c69e730229dbdd (127.0.105.3:41575)
I20260503 19:53:32.037910 1501 raft_consensus.cc:2804] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b8594a7337c54dc58d701259120d361b [term 1 FOLLOWER]: Leader election won for term 1
I20260503 19:53:32.038138 1501 raft_consensus.cc:697] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b8594a7337c54dc58d701259120d361b [term 1 LEADER]: Becoming Leader. State: Replica: b8594a7337c54dc58d701259120d361b, State: Running, Role: LEADER
I20260503 19:53:32.038865 1501 consensus_queue.cc:237] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b8594a7337c54dc58d701259120d361b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.059120 1349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b6e48ca8c44c7e8bda3c62201b9dd1" candidate_uuid: "b8594a7337c54dc58d701259120d361b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b68b1efd1cc44e15b8c69e730229dbdd"
I20260503 19:53:32.059446 1349 raft_consensus.cc:3060] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Advancing to term 1
W20260503 19:53:32.059857 1265 tablet.cc:2404] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P fc902de5b2274103ab5e76f967503afb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260503 19:53:32.062666 1349 raft_consensus.cc:2468] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b68b1efd1cc44e15b8c69e730229dbdd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8594a7337c54dc58d701259120d361b in term 1.
I20260503 19:53:32.062961 1460 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260503 19:53:32.064242 1460 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260503 19:53:32.064656 1460 kserver.cc:163] Server-wide thread pool size limit: 3276
W20260503 19:53:32.100668 1118 tablet.cc:2404] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b8594a7337c54dc58d701259120d361b: Can't schedule compaction. Clean time has not been advanced past its initial value.
19:53:32.190 [HMS - ERROR - pool-6-thread-3] (RetryingHMSHandler.java:201) AlreadyExistsException(message:Table hive.default.table_cb2fc46da15e4d3cbc25e42aac22a26c already exists)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1839)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1786)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:2035)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
at jdk.proxy2/jdk.proxy2.$Proxy27.create_table_with_environment_context(Unknown Source)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14800)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14784)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:104)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
I20260503 19:53:32.191855 1521 raft_consensus.cc:493] T 84654b8548be48478ea7c6dcf1f1a3f5 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:53:32.192122 1521 raft_consensus.cc:515] T 84654b8548be48478ea7c6dcf1f1a3f5 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } } peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } }
I20260503 19:53:32.193040 1521 leader_election.cc:290] T 84654b8548be48478ea7c6dcf1f1a3f5 P b8594a7337c54dc58d701259120d361b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc902de5b2274103ab5e76f967503afb (127.0.105.2:46339), b68b1efd1cc44e15b8c69e730229dbdd (127.0.105.3:41575)
W20260503 19:53:32.193763 802 catalog_manager.cc:2170] Already present: failed to create HMS catalog entry for table [id=8a9b091809bb45e4aa188a8b74379c39]: failed to create Hive MetaStore table: TException - service has thrown: AlreadyExistsException(message=Table hive.default.table_cb2fc46da15e4d3cbc25e42aac22a26c already exists)
I20260503 19:53:32.194283 802 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:54616 (request call id 31) took 1375 ms. Trace:
I20260503 19:53:32.194274 1211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84654b8548be48478ea7c6dcf1f1a3f5" candidate_uuid: "b8594a7337c54dc58d701259120d361b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc902de5b2274103ab5e76f967503afb" is_pre_election: true
I20260503 19:53:32.194419 802 rpcz_store.cc:276] 0503 19:53:30.818349 (+ 0us) service_pool.cc:167] Inserting onto call queue
0503 19:53:30.818598 (+ 249us) service_pool.cc:224] Handling call
0503 19:53:30.835050 (+ 16452us) catalog_manager.cc:2101] Acquired catalog manager lock
0503 19:53:30.837417 (+ 2367us) catalog_manager.cc:2147] Created new table and tablet info
0503 19:53:32.194236 (+1356819us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":1026052,"Hive Metastore.run_cpu_time_us":1142,"Hive Metastore.run_wall_time_us":329933}
I20260503 19:53:32.194617 1211 raft_consensus.cc:2468] T 84654b8548be48478ea7c6dcf1f1a3f5 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8594a7337c54dc58d701259120d361b in term 0.
I20260503 19:53:32.194996 1349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84654b8548be48478ea7c6dcf1f1a3f5" candidate_uuid: "b8594a7337c54dc58d701259120d361b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" is_pre_election: true
I20260503 19:53:32.195221 1349 raft_consensus.cc:2468] T 84654b8548be48478ea7c6dcf1f1a3f5 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8594a7337c54dc58d701259120d361b in term 0.
I20260503 19:53:32.197023 1005 leader_election.cc:304] T 84654b8548be48478ea7c6dcf1f1a3f5 P b8594a7337c54dc58d701259120d361b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8594a7337c54dc58d701259120d361b, fc902de5b2274103ab5e76f967503afb; no voters:
I20260503 19:53:32.197193 802 master_service.cc:954] ReplaceTablet: received request to replace tablet 84654b8548be48478ea7c6dcf1f1a3f5 from {username='slave'} at 127.0.0.1:54616
I20260503 19:53:32.197875 1521 raft_consensus.cc:2804] T 84654b8548be48478ea7c6dcf1f1a3f5 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260503 19:53:32.197989 1521 raft_consensus.cc:493] T 84654b8548be48478ea7c6dcf1f1a3f5 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260503 19:53:32.198074 1521 raft_consensus.cc:3060] T 84654b8548be48478ea7c6dcf1f1a3f5 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Advancing to term 1
I20260503 19:53:32.208132 802 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: Sending DeleteTablet for 3 replicas of tablet 84654b8548be48478ea7c6dcf1f1a3f5
I20260503 19:53:32.208875 802 catalog_manager.cc:6741] ReplaceTablet: tablet 84654b8548be48478ea7c6dcf1f1a3f5 deleted and replaced by tablet 1e0cc6e8b2734484a67fbb9572eb68e7
I20260503 19:53:32.211833 1329 tablet_service.cc:1558] Processing DeleteTablet for tablet 84654b8548be48478ea7c6dcf1f1a3f5 with delete_type TABLET_DATA_DELETED (replaced by tablet 1e0cc6e8b2734484a67fbb9572eb68e7) from {username='slave'} at 127.0.0.1:36426
I20260503 19:53:32.212766 1548 tablet_replica.cc:333] T 84654b8548be48478ea7c6dcf1f1a3f5 P b68b1efd1cc44e15b8c69e730229dbdd: stopping tablet replica
I20260503 19:53:32.212966 1191 tablet_service.cc:1558] Processing DeleteTablet for tablet 84654b8548be48478ea7c6dcf1f1a3f5 with delete_type TABLET_DATA_DELETED (replaced by tablet 1e0cc6e8b2734484a67fbb9572eb68e7) from {username='slave'} at 127.0.0.1:58604
I20260503 19:53:32.213284 1548 raft_consensus.cc:2243] T 84654b8548be48478ea7c6dcf1f1a3f5 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Raft consensus shutting down.
I20260503 19:53:32.213717 1548 raft_consensus.cc:2272] T 84654b8548be48478ea7c6dcf1f1a3f5 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Raft consensus is shut down!
I20260503 19:53:32.214509 1549 tablet_replica.cc:333] T 84654b8548be48478ea7c6dcf1f1a3f5 P fc902de5b2274103ab5e76f967503afb: stopping tablet replica
I20260503 19:53:32.214944 1549 raft_consensus.cc:2243] T 84654b8548be48478ea7c6dcf1f1a3f5 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260503 19:53:32.215314 1549 raft_consensus.cc:2272] T 84654b8548be48478ea7c6dcf1f1a3f5 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260503 19:53:32.217361 802 catalog_manager.cc:5671] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b8594a7337c54dc58d701259120d361b reported cstate change: term changed from 0 to 1, leader changed from <none> to b8594a7337c54dc58d701259120d361b (127.0.105.1). New cstate: current_term: 1 leader_uuid: "b8594a7337c54dc58d701259120d361b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } health_report { overall_health: UNKNOWN } } }
I20260503 19:53:32.217973 1549 ts_tablet_manager.cc:1916] T 84654b8548be48478ea7c6dcf1f1a3f5 P fc902de5b2274103ab5e76f967503afb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260503 19:53:32.218201 1500 raft_consensus.cc:493] T 6841c0c80f814208b598787466df21b7 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:53:32.218371 1500 raft_consensus.cc:515] T 6841c0c80f814208b598787466df21b7 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.220636 1500 leader_election.cc:290] T 6841c0c80f814208b598787466df21b7 P fc902de5b2274103ab5e76f967503afb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8594a7337c54dc58d701259120d361b (127.0.105.1:40289), b68b1efd1cc44e15b8c69e730229dbdd (127.0.105.3:41575)
I20260503 19:53:32.222802 1548 ts_tablet_manager.cc:1916] T 84654b8548be48478ea7c6dcf1f1a3f5 P b68b1efd1cc44e15b8c69e730229dbdd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260503 19:53:32.232381 1349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6841c0c80f814208b598787466df21b7" candidate_uuid: "fc902de5b2274103ab5e76f967503afb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" is_pre_election: true
I20260503 19:53:32.232679 1349 raft_consensus.cc:2468] T 6841c0c80f814208b598787466df21b7 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc902de5b2274103ab5e76f967503afb in term 0.
I20260503 19:53:32.233606 1147 leader_election.cc:304] T 6841c0c80f814208b598787466df21b7 P fc902de5b2274103ab5e76f967503afb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b68b1efd1cc44e15b8c69e730229dbdd, fc902de5b2274103ab5e76f967503afb; no voters:
I20260503 19:53:32.234261 1500 raft_consensus.cc:2804] T 6841c0c80f814208b598787466df21b7 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260503 19:53:32.234431 1500 raft_consensus.cc:493] T 6841c0c80f814208b598787466df21b7 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260503 19:53:32.234557 1500 raft_consensus.cc:3060] T 6841c0c80f814208b598787466df21b7 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Advancing to term 1
I20260503 19:53:32.237572 1500 raft_consensus.cc:515] T 6841c0c80f814208b598787466df21b7 P fc902de5b2274103ab5e76f967503afb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.210649 1050 tablet_service.cc:1558] Processing DeleteTablet for tablet 84654b8548be48478ea7c6dcf1f1a3f5 with delete_type TABLET_DATA_DELETED (replaced by tablet 1e0cc6e8b2734484a67fbb9572eb68e7) from {username='slave'} at 127.0.0.1:47844
I20260503 19:53:32.200914 1521 raft_consensus.cc:515] T 84654b8548be48478ea7c6dcf1f1a3f5 P b8594a7337c54dc58d701259120d361b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } } peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } }
I20260503 19:53:32.245476 1070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6841c0c80f814208b598787466df21b7" candidate_uuid: "fc902de5b2274103ab5e76f967503afb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8594a7337c54dc58d701259120d361b" is_pre_election: true
I20260503 19:53:32.245908 1070 raft_consensus.cc:2468] T 6841c0c80f814208b598787466df21b7 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc902de5b2274103ab5e76f967503afb in term 0.
I20260503 19:53:32.249588 1548 ts_tablet_manager.cc:1929] T 84654b8548be48478ea7c6dcf1f1a3f5 P b68b1efd1cc44e15b8c69e730229dbdd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260503 19:53:32.250591 1557 tablet_replica.cc:333] T 84654b8548be48478ea7c6dcf1f1a3f5 P b8594a7337c54dc58d701259120d361b: stopping tablet replica
I20260503 19:53:32.249972 1493 raft_consensus.cc:493] T ed5861fcf94c47199eb1a81d4702f19b P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:53:32.250990 1493 raft_consensus.cc:515] T ed5861fcf94c47199eb1a81d4702f19b P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.252907 1493 leader_election.cc:290] T ed5861fcf94c47199eb1a81d4702f19b P b68b1efd1cc44e15b8c69e730229dbdd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8594a7337c54dc58d701259120d361b (127.0.105.1:40289), fc902de5b2274103ab5e76f967503afb (127.0.105.2:46339)
I20260503 19:53:32.253365 1557 raft_consensus.cc:2243] T 84654b8548be48478ea7c6dcf1f1a3f5 P b8594a7337c54dc58d701259120d361b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260503 19:53:32.253767 1557 raft_consensus.cc:2272] T 84654b8548be48478ea7c6dcf1f1a3f5 P b8594a7337c54dc58d701259120d361b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260503 19:53:32.254832 1049 tablet_service.cc:1511] Processing CreateTablet for tablet 1e0cc6e8b2734484a67fbb9572eb68e7 (DEFAULT_TABLE table=default.table_f423a73d5c224651bb5eb676ee98c5e3 [id=f8aedcc7a5c34071a86b38b46361455f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260503 19:53:32.255250 1049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e0cc6e8b2734484a67fbb9572eb68e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:32.257812 1329 tablet_service.cc:1511] Processing CreateTablet for tablet 1e0cc6e8b2734484a67fbb9572eb68e7 (DEFAULT_TABLE table=default.table_f423a73d5c224651bb5eb676ee98c5e3 [id=f8aedcc7a5c34071a86b38b46361455f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260503 19:53:32.258224 1329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e0cc6e8b2734484a67fbb9572eb68e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:32.261332 802 master_service.cc:954] ReplaceTablet: received request to replace tablet 6841c0c80f814208b598787466df21b7 from {username='slave'} at 127.0.0.1:54616
I20260503 19:53:32.266420 1488 tablet_bootstrap.cc:492] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b8594a7337c54dc58d701259120d361b: Bootstrap starting.
I20260503 19:53:32.268282 1488 tablet_bootstrap.cc:654] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b8594a7337c54dc58d701259120d361b: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:32.268237 1349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84654b8548be48478ea7c6dcf1f1a3f5" candidate_uuid: "b8594a7337c54dc58d701259120d361b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b68b1efd1cc44e15b8c69e730229dbdd"
I20260503 19:53:32.269497 1070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed5861fcf94c47199eb1a81d4702f19b" candidate_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8594a7337c54dc58d701259120d361b" is_pre_election: true
I20260503 19:53:32.269717 1070 raft_consensus.cc:2468] T ed5861fcf94c47199eb1a81d4702f19b P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b68b1efd1cc44e15b8c69e730229dbdd in term 0.
W20260503 19:53:32.269632 1005 leader_election.cc:343] T 84654b8548be48478ea7c6dcf1f1a3f5 P b8594a7337c54dc58d701259120d361b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b68b1efd1cc44e15b8c69e730229dbdd (127.0.105.3:41575): Illegal state: Tablet not RUNNING: STOPPED
I20260503 19:53:32.270407 1284 leader_election.cc:304] T ed5861fcf94c47199eb1a81d4702f19b P b68b1efd1cc44e15b8c69e730229dbdd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b68b1efd1cc44e15b8c69e730229dbdd, b8594a7337c54dc58d701259120d361b; no voters:
I20260503 19:53:32.270926 1493 raft_consensus.cc:2804] T ed5861fcf94c47199eb1a81d4702f19b P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260503 19:53:32.271044 1493 raft_consensus.cc:493] T ed5861fcf94c47199eb1a81d4702f19b P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260503 19:53:32.271126 1493 raft_consensus.cc:3060] T ed5861fcf94c47199eb1a81d4702f19b P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Advancing to term 1
I20260503 19:53:32.276453 1486 tablet_bootstrap.cc:492] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b68b1efd1cc44e15b8c69e730229dbdd: Bootstrap starting.
I20260503 19:53:32.276473 802 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: Sending DeleteTablet for 3 replicas of tablet 6841c0c80f814208b598787466df21b7
I20260503 19:53:32.277225 1050 tablet_service.cc:1558] Processing DeleteTablet for tablet 6841c0c80f814208b598787466df21b7 with delete_type TABLET_DATA_DELETED (replaced by tablet 65f0f8cedacf4024ae7bbb741a25c2f8) from {username='slave'} at 127.0.0.1:47844
I20260503 19:53:32.277444 1521 leader_election.cc:290] T 84654b8548be48478ea7c6dcf1f1a3f5 P b8594a7337c54dc58d701259120d361b [CANDIDATE]: Term 1 election: Requested vote from peers fc902de5b2274103ab5e76f967503afb (127.0.105.2:46339), b68b1efd1cc44e15b8c69e730229dbdd (127.0.105.3:41575)
I20260503 19:53:32.277755 1493 raft_consensus.cc:515] T ed5861fcf94c47199eb1a81d4702f19b P b68b1efd1cc44e15b8c69e730229dbdd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.278347 1486 tablet_bootstrap.cc:654] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b68b1efd1cc44e15b8c69e730229dbdd: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:32.278942 1070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed5861fcf94c47199eb1a81d4702f19b" candidate_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8594a7337c54dc58d701259120d361b"
I20260503 19:53:32.279135 1070 raft_consensus.cc:3060] T ed5861fcf94c47199eb1a81d4702f19b P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Advancing to term 1
I20260503 19:53:32.280236 1069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6841c0c80f814208b598787466df21b7" candidate_uuid: "fc902de5b2274103ab5e76f967503afb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8594a7337c54dc58d701259120d361b"
I20260503 19:53:32.280536 1069 raft_consensus.cc:3060] T 6841c0c80f814208b598787466df21b7 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Advancing to term 1
I20260503 19:53:32.281253 1328 tablet_service.cc:1558] Processing DeleteTablet for tablet 6841c0c80f814208b598787466df21b7 with delete_type TABLET_DATA_DELETED (replaced by tablet 65f0f8cedacf4024ae7bbb741a25c2f8) from {username='slave'} at 127.0.0.1:36426
I20260503 19:53:32.282018 1557 ts_tablet_manager.cc:1916] T 84654b8548be48478ea7c6dcf1f1a3f5 P b8594a7337c54dc58d701259120d361b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260503 19:53:32.282405 1070 raft_consensus.cc:2468] T ed5861fcf94c47199eb1a81d4702f19b P b8594a7337c54dc58d701259120d361b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b68b1efd1cc44e15b8c69e730229dbdd in term 1.
I20260503 19:53:32.282932 1284 leader_election.cc:304] T ed5861fcf94c47199eb1a81d4702f19b P b68b1efd1cc44e15b8c69e730229dbdd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b68b1efd1cc44e15b8c69e730229dbdd, b8594a7337c54dc58d701259120d361b; no voters:
I20260503 19:53:32.283591 802 catalog_manager.cc:6741] ReplaceTablet: tablet 6841c0c80f814208b598787466df21b7 deleted and replaced by tablet 65f0f8cedacf4024ae7bbb741a25c2f8
I20260503 19:53:32.283603 1069 raft_consensus.cc:2468] T 6841c0c80f814208b598787466df21b7 P b8594a7337c54dc58d701259120d361b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc902de5b2274103ab5e76f967503afb in term 1.
I20260503 19:53:32.284036 1146 leader_election.cc:304] T 6841c0c80f814208b598787466df21b7 P fc902de5b2274103ab5e76f967503afb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8594a7337c54dc58d701259120d361b, fc902de5b2274103ab5e76f967503afb; no voters:
I20260503 19:53:32.284248 802 master_service.cc:954] ReplaceTablet: received request to replace tablet 84654b8548be48478ea7c6dcf1f1a3f5 from {username='slave'} at 127.0.0.1:54616
I20260503 19:53:32.285722 1500 leader_election.cc:290] T 6841c0c80f814208b598787466df21b7 P fc902de5b2274103ab5e76f967503afb [CANDIDATE]: Term 1 election: Requested vote from peers b8594a7337c54dc58d701259120d361b (127.0.105.1:40289), b68b1efd1cc44e15b8c69e730229dbdd (127.0.105.3:41575)
I20260503 19:53:32.289153 1050 tablet_service.cc:1511] Processing CreateTablet for tablet 65f0f8cedacf4024ae7bbb741a25c2f8 (DEFAULT_TABLE table=default.table_9c30ca5293344be3964460b578f9d6df [id=8d8f5993b50b4a59aaa004b29cf7c6dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260503 19:53:32.289640 1050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65f0f8cedacf4024ae7bbb741a25c2f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:32.294677 1493 leader_election.cc:290] T ed5861fcf94c47199eb1a81d4702f19b P b68b1efd1cc44e15b8c69e730229dbdd [CANDIDATE]: Term 1 election: Requested vote from peers b8594a7337c54dc58d701259120d361b (127.0.105.1:40289), fc902de5b2274103ab5e76f967503afb (127.0.105.2:46339)
I20260503 19:53:32.295046 1493 raft_consensus.cc:2804] T ed5861fcf94c47199eb1a81d4702f19b P b68b1efd1cc44e15b8c69e730229dbdd [term 1 FOLLOWER]: Leader election won for term 1
I20260503 19:53:32.295249 1493 raft_consensus.cc:697] T ed5861fcf94c47199eb1a81d4702f19b P b68b1efd1cc44e15b8c69e730229dbdd [term 1 LEADER]: Becoming Leader. State: Replica: b68b1efd1cc44e15b8c69e730229dbdd, State: Running, Role: LEADER
I20260503 19:53:32.295846 1493 consensus_queue.cc:237] T ed5861fcf94c47199eb1a81d4702f19b P b68b1efd1cc44e15b8c69e730229dbdd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.298720 1189 tablet_service.cc:1511] Processing CreateTablet for tablet 65f0f8cedacf4024ae7bbb741a25c2f8 (DEFAULT_TABLE table=default.table_9c30ca5293344be3964460b578f9d6df [id=8d8f5993b50b4a59aaa004b29cf7c6dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260503 19:53:32.299417 1189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65f0f8cedacf4024ae7bbb741a25c2f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:32.299710 1500 raft_consensus.cc:2804] T 6841c0c80f814208b598787466df21b7 P fc902de5b2274103ab5e76f967503afb [term 1 FOLLOWER]: Leader election won for term 1
I20260503 19:53:32.299901 1500 raft_consensus.cc:697] T 6841c0c80f814208b598787466df21b7 P fc902de5b2274103ab5e76f967503afb [term 1 LEADER]: Becoming Leader. State: Replica: fc902de5b2274103ab5e76f967503afb, State: Running, Role: LEADER
I20260503 19:53:32.300433 1500 consensus_queue.cc:237] T 6841c0c80f814208b598787466df21b7 P fc902de5b2274103ab5e76f967503afb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.301520 802 master_service.cc:954] ReplaceTablet: received request to replace tablet 84654b8548be48478ea7c6dcf1f1a3f5 from {username='slave'} at 127.0.0.1:54616
I20260503 19:53:32.303589 802 master_service.cc:954] ReplaceTablet: received request to replace tablet 84654b8548be48478ea7c6dcf1f1a3f5 from {username='slave'} at 127.0.0.1:54616
I20260503 19:53:32.305440 1557 ts_tablet_manager.cc:1929] T 84654b8548be48478ea7c6dcf1f1a3f5 P b8594a7337c54dc58d701259120d361b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260503 19:53:32.306670 802 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:54616:
table { table_name: "default.Table_0fc3df0b61324d7eac247468c05ce87f" } new_table_name: "default.Table_dd74e45a321f4e7f92e8b514d7378c86" modify_external_catalogs: true
I20260503 19:53:32.307017 1487 tablet_bootstrap.cc:492] T 65f0f8cedacf4024ae7bbb741a25c2f8 P fc902de5b2274103ab5e76f967503afb: Bootstrap starting.
I20260503 19:53:32.308911 1487 tablet_bootstrap.cc:654] T 65f0f8cedacf4024ae7bbb741a25c2f8 P fc902de5b2274103ab5e76f967503afb: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:32.309446 1557 log.cc:1199] T 84654b8548be48478ea7c6dcf1f1a3f5 P b8594a7337c54dc58d701259120d361b: Deleting WAL directory at /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-0/wal/wals/84654b8548be48478ea7c6dcf1f1a3f5
I20260503 19:53:32.310025 1557 ts_tablet_manager.cc:1950] T 84654b8548be48478ea7c6dcf1f1a3f5 P b8594a7337c54dc58d701259120d361b: Deleting consensus metadata
I20260503 19:53:32.250424 1549 ts_tablet_manager.cc:1929] T 84654b8548be48478ea7c6dcf1f1a3f5 P fc902de5b2274103ab5e76f967503afb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260503 19:53:32.257444 1191 tablet_service.cc:1511] Processing CreateTablet for tablet 1e0cc6e8b2734484a67fbb9572eb68e7 (DEFAULT_TABLE table=default.table_f423a73d5c224651bb5eb676ee98c5e3 [id=f8aedcc7a5c34071a86b38b46361455f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260503 19:53:32.311625 1191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e0cc6e8b2734484a67fbb9572eb68e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:32.313823 1557 tablet_replica.cc:333] T 6841c0c80f814208b598787466df21b7 P b8594a7337c54dc58d701259120d361b: stopping tablet replica
I20260503 19:53:32.267462 1211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84654b8548be48478ea7c6dcf1f1a3f5" candidate_uuid: "b8594a7337c54dc58d701259120d361b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc902de5b2274103ab5e76f967503afb"
I20260503 19:53:32.316228 796 catalog_manager.cc:5002] TS b8594a7337c54dc58d701259120d361b (127.0.105.1:40289): tablet 84654b8548be48478ea7c6dcf1f1a3f5 (table default.table_f423a73d5c224651bb5eb676ee98c5e3 [id=f8aedcc7a5c34071a86b38b46361455f]) successfully deleted
I20260503 19:53:32.316469 1557 raft_consensus.cc:2243] T 6841c0c80f814208b598787466df21b7 P b8594a7337c54dc58d701259120d361b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260503 19:53:32.278496 1190 tablet_service.cc:1558] Processing DeleteTablet for tablet 6841c0c80f814208b598787466df21b7 with delete_type TABLET_DATA_DELETED (replaced by tablet 65f0f8cedacf4024ae7bbb741a25c2f8) from {username='slave'} at 127.0.0.1:58604
I20260503 19:53:32.316666 1557 raft_consensus.cc:2272] T 6841c0c80f814208b598787466df21b7 P b8594a7337c54dc58d701259120d361b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260503 19:53:32.316717 1487 tablet_bootstrap.cc:492] T 65f0f8cedacf4024ae7bbb741a25c2f8 P fc902de5b2274103ab5e76f967503afb: No bootstrap required, opened a new log
I20260503 19:53:32.316898 1487 ts_tablet_manager.cc:1403] T 65f0f8cedacf4024ae7bbb741a25c2f8 P fc902de5b2274103ab5e76f967503afb: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
W20260503 19:53:32.317215 1005 leader_election.cc:343] T 84654b8548be48478ea7c6dcf1f1a3f5 P b8594a7337c54dc58d701259120d361b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fc902de5b2274103ab5e76f967503afb (127.0.105.2:46339): Illegal state: Tablet not RUNNING: STOPPED
I20260503 19:53:32.317343 1005 leader_election.cc:304] T 84654b8548be48478ea7c6dcf1f1a3f5 P b8594a7337c54dc58d701259120d361b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8594a7337c54dc58d701259120d361b; no voters: b68b1efd1cc44e15b8c69e730229dbdd, fc902de5b2274103ab5e76f967503afb
W20260503 19:53:32.317579 1005 raft_consensus.cc:2686] T 84654b8548be48478ea7c6dcf1f1a3f5 P b8594a7337c54dc58d701259120d361b: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260503 19:53:32.317802 1487 raft_consensus.cc:359] T 65f0f8cedacf4024ae7bbb741a25c2f8 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.314743 1488 tablet_bootstrap.cc:492] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b8594a7337c54dc58d701259120d361b: No bootstrap required, opened a new log
I20260503 19:53:32.318322 1488 ts_tablet_manager.cc:1403] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b8594a7337c54dc58d701259120d361b: Time spent bootstrapping tablet: real 0.052s user 0.012s sys 0.006s
I20260503 19:53:32.319139 1488 raft_consensus.cc:359] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.319335 1488 raft_consensus.cc:385] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:32.319406 1488 raft_consensus.cc:740] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8594a7337c54dc58d701259120d361b, State: Initialized, Role: FOLLOWER
I20260503 19:53:32.319639 1488 consensus_queue.cc:260] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b8594a7337c54dc58d701259120d361b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.320485 1487 raft_consensus.cc:385] T 65f0f8cedacf4024ae7bbb741a25c2f8 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:32.320605 1487 raft_consensus.cc:740] T 65f0f8cedacf4024ae7bbb741a25c2f8 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc902de5b2274103ab5e76f967503afb, State: Initialized, Role: FOLLOWER
I20260503 19:53:32.320740 1488 ts_tablet_manager.cc:1434] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b8594a7337c54dc58d701259120d361b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260503 19:53:32.320881 1487 consensus_queue.cc:260] T 65f0f8cedacf4024ae7bbb741a25c2f8 P fc902de5b2274103ab5e76f967503afb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.321112 1488 tablet_bootstrap.cc:492] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b8594a7337c54dc58d701259120d361b: Bootstrap starting.
I20260503 19:53:32.321573 1487 ts_tablet_manager.cc:1434] T 65f0f8cedacf4024ae7bbb741a25c2f8 P fc902de5b2274103ab5e76f967503afb: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260503 19:53:32.322840 1488 tablet_bootstrap.cc:654] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b8594a7337c54dc58d701259120d361b: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:32.333726 1557 ts_tablet_manager.cc:1916] T 6841c0c80f814208b598787466df21b7 P b8594a7337c54dc58d701259120d361b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260503 19:53:32.337143 1549 log.cc:1199] T 84654b8548be48478ea7c6dcf1f1a3f5 P fc902de5b2274103ab5e76f967503afb: Deleting WAL directory at /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-1/wal/wals/84654b8548be48478ea7c6dcf1f1a3f5
I20260503 19:53:32.340911 1487 tablet_bootstrap.cc:492] T 1e0cc6e8b2734484a67fbb9572eb68e7 P fc902de5b2274103ab5e76f967503afb: Bootstrap starting.
I20260503 19:53:32.338847 1549 ts_tablet_manager.cc:1950] T 84654b8548be48478ea7c6dcf1f1a3f5 P fc902de5b2274103ab5e76f967503afb: Deleting consensus metadata
I20260503 19:53:32.342816 1487 tablet_bootstrap.cc:654] T 1e0cc6e8b2734484a67fbb9572eb68e7 P fc902de5b2274103ab5e76f967503afb: Neither blocks nor log segments found. Creating new log.
W20260503 19:53:32.343007 1284 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.1:40289, user_credentials={real_user=slave}} blocked reactor thread for 60073.9us
I20260503 19:53:32.343116 797 catalog_manager.cc:5002] TS fc902de5b2274103ab5e76f967503afb (127.0.105.2:46339): tablet 84654b8548be48478ea7c6dcf1f1a3f5 (table default.table_f423a73d5c224651bb5eb676ee98c5e3 [id=f8aedcc7a5c34071a86b38b46361455f]) successfully deleted
I20260503 19:53:32.343780 1329 tablet_service.cc:1511] Processing CreateTablet for tablet 65f0f8cedacf4024ae7bbb741a25c2f8 (DEFAULT_TABLE table=default.table_9c30ca5293344be3964460b578f9d6df [id=8d8f5993b50b4a59aaa004b29cf7c6dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260503 19:53:32.344024 1549 tablet_replica.cc:333] T 6841c0c80f814208b598787466df21b7 P fc902de5b2274103ab5e76f967503afb: stopping tablet replica
I20260503 19:53:32.344180 1329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65f0f8cedacf4024ae7bbb741a25c2f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:32.344295 1549 raft_consensus.cc:2243] T 6841c0c80f814208b598787466df21b7 P fc902de5b2274103ab5e76f967503afb [term 1 LEADER]: Raft consensus shutting down.
I20260503 19:53:32.344883 1549 pending_rounds.cc:70] T 6841c0c80f814208b598787466df21b7 P fc902de5b2274103ab5e76f967503afb: Trying to abort 1 pending ops.
I20260503 19:53:32.345130 1549 pending_rounds.cc:77] T 6841c0c80f814208b598787466df21b7 P fc902de5b2274103ab5e76f967503afb: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7282024498414993408 op_type: NO_OP noop_request { }
I20260503 19:53:32.345333 1549 raft_consensus.cc:2889] T 6841c0c80f814208b598787466df21b7 P fc902de5b2274103ab5e76f967503afb [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260503 19:53:32.346026 1549 raft_consensus.cc:2272] T 6841c0c80f814208b598787466df21b7 P fc902de5b2274103ab5e76f967503afb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260503 19:53:32.346412 1565 consensus_queue.cc:1175] T 6841c0c80f814208b598787466df21b7 P fc902de5b2274103ab5e76f967503afb [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "fc902de5b2274103ab5e76f967503afb" status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
I20260503 19:53:32.348307 1549 ts_tablet_manager.cc:1916] T 6841c0c80f814208b598787466df21b7 P fc902de5b2274103ab5e76f967503afb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260503 19:53:32.349781 1349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6841c0c80f814208b598787466df21b7" candidate_uuid: "fc902de5b2274103ab5e76f967503afb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b68b1efd1cc44e15b8c69e730229dbdd"
I20260503 19:53:32.350001 1349 raft_consensus.cc:3060] T 6841c0c80f814208b598787466df21b7 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Advancing to term 1
I20260503 19:53:32.353564 1349 raft_consensus.cc:2468] T 6841c0c80f814208b598787466df21b7 P b68b1efd1cc44e15b8c69e730229dbdd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc902de5b2274103ab5e76f967503afb in term 1.
I20260503 19:53:32.354476 1487 tablet_bootstrap.cc:492] T 1e0cc6e8b2734484a67fbb9572eb68e7 P fc902de5b2274103ab5e76f967503afb: No bootstrap required, opened a new log
I20260503 19:53:32.354914 1487 ts_tablet_manager.cc:1403] T 1e0cc6e8b2734484a67fbb9572eb68e7 P fc902de5b2274103ab5e76f967503afb: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.005s
I20260503 19:53:32.355125 1211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed5861fcf94c47199eb1a81d4702f19b" candidate_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc902de5b2274103ab5e76f967503afb" is_pre_election: true
I20260503 19:53:32.355360 1548 log.cc:1199] T 84654b8548be48478ea7c6dcf1f1a3f5 P b68b1efd1cc44e15b8c69e730229dbdd: Deleting WAL directory at /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-2/wal/wals/84654b8548be48478ea7c6dcf1f1a3f5
I20260503 19:53:32.355384 1211 raft_consensus.cc:2468] T ed5861fcf94c47199eb1a81d4702f19b P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b68b1efd1cc44e15b8c69e730229dbdd in term 0.
I20260503 19:53:32.355896 1548 ts_tablet_manager.cc:1950] T 84654b8548be48478ea7c6dcf1f1a3f5 P b68b1efd1cc44e15b8c69e730229dbdd: Deleting consensus metadata
I20260503 19:53:32.355891 1210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed5861fcf94c47199eb1a81d4702f19b" candidate_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc902de5b2274103ab5e76f967503afb"
I20260503 19:53:32.356156 1210 raft_consensus.cc:3060] T ed5861fcf94c47199eb1a81d4702f19b P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Advancing to term 1
I20260503 19:53:32.356617 1487 raft_consensus.cc:359] T 1e0cc6e8b2734484a67fbb9572eb68e7 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.356916 1487 raft_consensus.cc:385] T 1e0cc6e8b2734484a67fbb9572eb68e7 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:32.357069 1487 raft_consensus.cc:740] T 1e0cc6e8b2734484a67fbb9572eb68e7 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc902de5b2274103ab5e76f967503afb, State: Initialized, Role: FOLLOWER
I20260503 19:53:32.357494 1487 consensus_queue.cc:260] T 1e0cc6e8b2734484a67fbb9572eb68e7 P fc902de5b2274103ab5e76f967503afb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.358951 1487 ts_tablet_manager.cc:1434] T 1e0cc6e8b2734484a67fbb9572eb68e7 P fc902de5b2274103ab5e76f967503afb: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260503 19:53:32.359336 1210 raft_consensus.cc:2468] T ed5861fcf94c47199eb1a81d4702f19b P fc902de5b2274103ab5e76f967503afb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b68b1efd1cc44e15b8c69e730229dbdd in term 1.
I20260503 19:53:32.362035 1486 tablet_bootstrap.cc:492] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b68b1efd1cc44e15b8c69e730229dbdd: No bootstrap required, opened a new log
I20260503 19:53:32.362226 1486 ts_tablet_manager.cc:1403] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b68b1efd1cc44e15b8c69e730229dbdd: Time spent bootstrapping tablet: real 0.086s user 0.011s sys 0.012s
I20260503 19:53:32.363063 1486 raft_consensus.cc:359] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.363216 1557 ts_tablet_manager.cc:1929] T 6841c0c80f814208b598787466df21b7 P b8594a7337c54dc58d701259120d361b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260503 19:53:32.363286 1486 raft_consensus.cc:385] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:32.363411 1557 log.cc:1199] T 6841c0c80f814208b598787466df21b7 P b8594a7337c54dc58d701259120d361b: Deleting WAL directory at /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-0/wal/wals/6841c0c80f814208b598787466df21b7
I20260503 19:53:32.363379 1486 raft_consensus.cc:740] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b68b1efd1cc44e15b8c69e730229dbdd, State: Initialized, Role: FOLLOWER
I20260503 19:53:32.363806 1486 consensus_queue.cc:260] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b68b1efd1cc44e15b8c69e730229dbdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.364058 1557 ts_tablet_manager.cc:1950] T 6841c0c80f814208b598787466df21b7 P b8594a7337c54dc58d701259120d361b: Deleting consensus metadata
I20260503 19:53:32.364368 1486 ts_tablet_manager.cc:1434] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b68b1efd1cc44e15b8c69e730229dbdd: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260503 19:53:32.368840 1548 tablet_replica.cc:333] T 6841c0c80f814208b598787466df21b7 P b68b1efd1cc44e15b8c69e730229dbdd: stopping tablet replica
I20260503 19:53:32.369055 1548 raft_consensus.cc:2243] T 6841c0c80f814208b598787466df21b7 P b68b1efd1cc44e15b8c69e730229dbdd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260503 19:53:32.369237 1548 raft_consensus.cc:2272] T 6841c0c80f814208b598787466df21b7 P b68b1efd1cc44e15b8c69e730229dbdd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260503 19:53:32.369730 797 catalog_manager.cc:5002] TS b68b1efd1cc44e15b8c69e730229dbdd (127.0.105.3:41575): tablet 84654b8548be48478ea7c6dcf1f1a3f5 (table default.table_f423a73d5c224651bb5eb676ee98c5e3 [id=f8aedcc7a5c34071a86b38b46361455f]) successfully deleted
I20260503 19:53:32.371708 1486 tablet_bootstrap.cc:492] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b68b1efd1cc44e15b8c69e730229dbdd: Bootstrap starting.
I20260503 19:53:32.373989 1486 tablet_bootstrap.cc:654] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b68b1efd1cc44e15b8c69e730229dbdd: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:32.374214 1548 ts_tablet_manager.cc:1916] T 6841c0c80f814208b598787466df21b7 P b68b1efd1cc44e15b8c69e730229dbdd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260503 19:53:32.377797 796 catalog_manager.cc:5002] TS b8594a7337c54dc58d701259120d361b (127.0.105.1:40289): tablet 6841c0c80f814208b598787466df21b7 (table default.table_9c30ca5293344be3964460b578f9d6df [id=8d8f5993b50b4a59aaa004b29cf7c6dd]) successfully deleted
I20260503 19:53:32.378623 1488 tablet_bootstrap.cc:492] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b8594a7337c54dc58d701259120d361b: No bootstrap required, opened a new log
I20260503 19:53:32.378827 1488 ts_tablet_manager.cc:1403] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b8594a7337c54dc58d701259120d361b: Time spent bootstrapping tablet: real 0.058s user 0.005s sys 0.006s
I20260503 19:53:32.379657 1488 raft_consensus.cc:359] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.379921 1488 raft_consensus.cc:385] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:32.380024 1488 raft_consensus.cc:740] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8594a7337c54dc58d701259120d361b, State: Initialized, Role: FOLLOWER
I20260503 19:53:32.380323 1488 consensus_queue.cc:260] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b8594a7337c54dc58d701259120d361b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.376981 1549 ts_tablet_manager.cc:1929] T 6841c0c80f814208b598787466df21b7 P fc902de5b2274103ab5e76f967503afb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260503 19:53:32.383981 1549 log.cc:1199] T 6841c0c80f814208b598787466df21b7 P fc902de5b2274103ab5e76f967503afb: Deleting WAL directory at /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-1/wal/wals/6841c0c80f814208b598787466df21b7
I20260503 19:53:32.384555 1549 ts_tablet_manager.cc:1950] T 6841c0c80f814208b598787466df21b7 P fc902de5b2274103ab5e76f967503afb: Deleting consensus metadata
I20260503 19:53:32.384541 1488 ts_tablet_manager.cc:1434] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b8594a7337c54dc58d701259120d361b: Time spent starting tablet: real 0.006s user 0.002s sys 0.001s
I20260503 19:53:32.386044 797 catalog_manager.cc:5002] TS fc902de5b2274103ab5e76f967503afb (127.0.105.2:46339): tablet 6841c0c80f814208b598787466df21b7 (table default.table_9c30ca5293344be3964460b578f9d6df [id=8d8f5993b50b4a59aaa004b29cf7c6dd]) successfully deleted
I20260503 19:53:32.387614 1486 tablet_bootstrap.cc:492] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b68b1efd1cc44e15b8c69e730229dbdd: No bootstrap required, opened a new log
I20260503 19:53:32.387811 1486 ts_tablet_manager.cc:1403] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b68b1efd1cc44e15b8c69e730229dbdd: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.004s
I20260503 19:53:32.393231 1486 raft_consensus.cc:359] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.393622 1486 raft_consensus.cc:385] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:32.393787 1486 raft_consensus.cc:740] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b68b1efd1cc44e15b8c69e730229dbdd, State: Initialized, Role: FOLLOWER
I20260503 19:53:32.394141 1486 consensus_queue.cc:260] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b68b1efd1cc44e15b8c69e730229dbdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.394773 1486 ts_tablet_manager.cc:1434] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b68b1efd1cc44e15b8c69e730229dbdd: Time spent starting tablet: real 0.007s user 0.002s sys 0.001s
I20260503 19:53:32.400321 1548 ts_tablet_manager.cc:1929] T 6841c0c80f814208b598787466df21b7 P b68b1efd1cc44e15b8c69e730229dbdd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260503 19:53:32.400508 1548 log.cc:1199] T 6841c0c80f814208b598787466df21b7 P b68b1efd1cc44e15b8c69e730229dbdd: Deleting WAL directory at /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-2/wal/wals/6841c0c80f814208b598787466df21b7
I20260503 19:53:32.400906 1548 ts_tablet_manager.cc:1950] T 6841c0c80f814208b598787466df21b7 P b68b1efd1cc44e15b8c69e730229dbdd: Deleting consensus metadata
I20260503 19:53:32.402307 797 catalog_manager.cc:5002] TS b68b1efd1cc44e15b8c69e730229dbdd (127.0.105.3:41575): tablet 6841c0c80f814208b598787466df21b7 (table default.table_9c30ca5293344be3964460b578f9d6df [id=8d8f5993b50b4a59aaa004b29cf7c6dd]) successfully deleted
I20260503 19:53:32.441931 801 catalog_manager.cc:2174] created HMS catalog entry for table [id=dce4d9d47ba04a3187de97305003ba5d]
I20260503 19:53:32.449470 801 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:54616 (request call id 32) took 1630 ms. Trace:
I20260503 19:53:32.449640 801 rpcz_store.cc:276] 0503 19:53:30.818508 (+ 0us) service_pool.cc:167] Inserting onto call queue
0503 19:53:30.818578 (+ 70us) service_pool.cc:224] Handling call
0503 19:53:30.837785 (+ 19207us) catalog_manager.cc:2101] Acquired catalog manager lock
0503 19:53:30.838364 (+ 579us) catalog_manager.cc:2147] Created new table and tablet info
0503 19:53:32.441858 (+1603494us) catalog_manager.cc:2173] Created new table in HMS catalog
0503 19:53:32.449169 (+ 7311us) catalog_manager.cc:2207] Wrote table and tablets to system table
0503 19:53:32.449313 (+ 144us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0503 19:53:32.449425 (+ 112us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0503 19:53:32.442825 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0503 19:53:32.442990 (+ 165us) write_op.cc:432] Acquiring schema lock in shared mode
0503 19:53:32.442998 (+ 8us) write_op.cc:435] Acquired schema lock
0503 19:53:32.443000 (+ 2us) tablet.cc:662] Decoding operations
0503 19:53:32.443105 (+ 105us) write_op.cc:620] Acquiring the partition lock for write op
0503 19:53:32.443119 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0503 19:53:32.443125 (+ 6us) tablet.cc:685] Acquiring locks for 2 operations
0503 19:53:32.443223 (+ 98us) tablet.cc:701] Row locks acquired
0503 19:53:32.443226 (+ 3us) write_op.cc:260] PREPARE: finished
0503 19:53:32.443268 (+ 42us) write_op.cc:270] Start()
0503 19:53:32.443305 (+ 37us) write_op.cc:276] Timestamp: P: 1777838012443242 usec, L: 0
0503 19:53:32.443309 (+ 4us) op_driver.cc:348] REPLICATION: starting
0503 19:53:32.443531 (+ 222us) log.cc:844] Serialized 432 byte log entry
0503 19:53:32.447479 (+ 3948us) op_driver.cc:464] REPLICATION: finished
0503 19:53:32.447626 (+ 147us) write_op.cc:301] APPLY: starting
0503 19:53:32.447643 (+ 17us) tablet.cc:1367] starting BulkCheckPresence
0503 19:53:32.447714 (+ 71us) tablet.cc:1370] finished BulkCheckPresence
0503 19:53:32.447742 (+ 28us) tablet.cc:1372] starting ApplyRowOperation cycle
0503 19:53:32.448076 (+ 334us) tablet.cc:1383] finished ApplyRowOperation cycle
0503 19:53:32.448118 (+ 42us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0503 19:53:32.448125 (+ 7us) write_op.cc:312] APPLY: finished
0503 19:53:32.448210 (+ 85us) log.cc:844] Serialized 28 byte log entry
0503 19:53:32.448276 (+ 66us) write_op.cc:489] Releasing partition, row and schema locks
0503 19:53:32.448297 (+ 21us) write_op.cc:454] Released schema lock
0503 19:53:32.448304 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0503 19:53:32.448374 (+ 70us) write_op.cc:489] Releasing partition, row and schema locks
0503 19:53:32.448377 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":1355135,"Hive Metastore.run_cpu_time_us":355,"Hive Metastore.run_wall_time_us":248200,"child_traces":[["op",{"apply.queue_time_us":97,"apply.run_cpu_time_us":741,"apply.run_wall_time_us":740,"num_ops":2,"prepare.queue_time_us":65,"prepare.run_cpu_time_us":829,"prepare.run_wall_time_us":828,"raft.queue_time_us":1018,"raft.run_cpu_time_us":282,"raft.run_wall_time_us":283,"replication_time_us":4136}]]}
I20260503 19:53:32.456975 1521 raft_consensus.cc:493] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:53:32.457193 1521 raft_consensus.cc:515] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.457919 1521 leader_election.cc:290] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b8594a7337c54dc58d701259120d361b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc902de5b2274103ab5e76f967503afb (127.0.105.2:46339), b68b1efd1cc44e15b8c69e730229dbdd (127.0.105.3:41575)
I20260503 19:53:32.458964 1210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e0cc6e8b2734484a67fbb9572eb68e7" candidate_uuid: "b8594a7337c54dc58d701259120d361b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc902de5b2274103ab5e76f967503afb" is_pre_election: true
I20260503 19:53:32.459211 1210 raft_consensus.cc:2468] T 1e0cc6e8b2734484a67fbb9572eb68e7 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8594a7337c54dc58d701259120d361b in term 0.
I20260503 19:53:32.459609 1349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e0cc6e8b2734484a67fbb9572eb68e7" candidate_uuid: "b8594a7337c54dc58d701259120d361b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" is_pre_election: true
I20260503 19:53:32.459852 1349 raft_consensus.cc:2468] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8594a7337c54dc58d701259120d361b in term 0.
I20260503 19:53:32.460304 1005 leader_election.cc:304] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b8594a7337c54dc58d701259120d361b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b68b1efd1cc44e15b8c69e730229dbdd, b8594a7337c54dc58d701259120d361b; no voters:
I20260503 19:53:32.461040 1521 raft_consensus.cc:2804] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260503 19:53:32.461205 1521 raft_consensus.cc:493] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260503 19:53:32.461313 1521 raft_consensus.cc:3060] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Advancing to term 1
I20260503 19:53:32.463973 1329 tablet_service.cc:1511] Processing CreateTablet for tablet df35135905b04fea927000d21e9994e6 (DEFAULT_TABLE table=default.table_1148c991089047a88d36862e99224cac [id=dce4d9d47ba04a3187de97305003ba5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260503 19:53:32.463973 1191 tablet_service.cc:1511] Processing CreateTablet for tablet df35135905b04fea927000d21e9994e6 (DEFAULT_TABLE table=default.table_1148c991089047a88d36862e99224cac [id=dce4d9d47ba04a3187de97305003ba5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260503 19:53:32.464434 1191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df35135905b04fea927000d21e9994e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:32.464447 1329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df35135905b04fea927000d21e9994e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:32.465827 1585 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.105.60:42203 every 8 connection(s)
I20260503 19:53:32.466161 1460 rpc_server.cc:307] RPC server started. Bound to: 127.0.105.60:42203
I20260503 19:53:32.468654 1460 server_base.cc:1193] Dumped server information to /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/data/info.pb
I20260503 19:53:32.468961 1050 tablet_service.cc:1511] Processing CreateTablet for tablet df35135905b04fea927000d21e9994e6 (DEFAULT_TABLE table=default.table_1148c991089047a88d36862e99224cac [id=dce4d9d47ba04a3187de97305003ba5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260503 19:53:32.469445 1050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df35135905b04fea927000d21e9994e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:32.473189 1190 tablet_service.cc:1558] Processing DeleteTablet for tablet 6841c0c80f814208b598787466df21b7 with delete_type TABLET_DATA_DELETED (replaced by tablet 65f0f8cedacf4024ae7bbb741a25c2f8) from {username='slave'} at 127.0.0.1:58604
W20260503 19:53:32.474282 797 catalog_manager.cc:4969] TS fc902de5b2274103ab5e76f967503afb (127.0.105.2:46339): delete failed for tablet 6841c0c80f814208b598787466df21b7 because the tablet was not found. No further retry: Not found: Tablet not found: 6841c0c80f814208b598787466df21b7
I20260503 19:53:32.475975 1521 raft_consensus.cc:515] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b8594a7337c54dc58d701259120d361b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.476280 801 catalog_manager.cc:5671] T ed5861fcf94c47199eb1a81d4702f19b P b68b1efd1cc44e15b8c69e730229dbdd reported cstate change: term changed from 0 to 1, leader changed from <none> to b68b1efd1cc44e15b8c69e730229dbdd (127.0.105.3). New cstate: current_term: 1 leader_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } health_report { overall_health: HEALTHY } } }
I20260503 19:53:32.477063 1210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e0cc6e8b2734484a67fbb9572eb68e7" candidate_uuid: "b8594a7337c54dc58d701259120d361b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc902de5b2274103ab5e76f967503afb"
I20260503 19:53:32.477310 1210 raft_consensus.cc:3060] T 1e0cc6e8b2734484a67fbb9572eb68e7 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Advancing to term 1
I20260503 19:53:32.477564 420 external_mini_cluster.cc:1428] Started /tmp/dist-test-task12vCql/build/asan/bin/kudu as pid 1460
I20260503 19:53:32.480465 1210 raft_consensus.cc:2468] T 1e0cc6e8b2734484a67fbb9572eb68e7 P fc902de5b2274103ab5e76f967503afb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8594a7337c54dc58d701259120d361b in term 1.
I20260503 19:53:32.480949 1005 leader_election.cc:304] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b8594a7337c54dc58d701259120d361b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8594a7337c54dc58d701259120d361b, fc902de5b2274103ab5e76f967503afb; no voters:
I20260503 19:53:32.481329 1501 raft_consensus.cc:2804] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b8594a7337c54dc58d701259120d361b [term 1 FOLLOWER]: Leader election won for term 1
I20260503 19:53:32.486239 1486 tablet_bootstrap.cc:492] T df35135905b04fea927000d21e9994e6 P b68b1efd1cc44e15b8c69e730229dbdd: Bootstrap starting.
I20260503 19:53:32.488279 1486 tablet_bootstrap.cc:654] T df35135905b04fea927000d21e9994e6 P b68b1efd1cc44e15b8c69e730229dbdd: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:32.491361 1349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e0cc6e8b2734484a67fbb9572eb68e7" candidate_uuid: "b8594a7337c54dc58d701259120d361b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b68b1efd1cc44e15b8c69e730229dbdd"
I20260503 19:53:32.491601 1349 raft_consensus.cc:3060] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Advancing to term 1
I20260503 19:53:32.494735 1349 raft_consensus.cc:2468] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b68b1efd1cc44e15b8c69e730229dbdd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8594a7337c54dc58d701259120d361b in term 1.
I20260503 19:53:32.495679 1521 leader_election.cc:290] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b8594a7337c54dc58d701259120d361b [CANDIDATE]: Term 1 election: Requested vote from peers fc902de5b2274103ab5e76f967503afb (127.0.105.2:46339), b68b1efd1cc44e15b8c69e730229dbdd (127.0.105.3:41575)
I20260503 19:53:32.498128 1487 tablet_bootstrap.cc:492] T df35135905b04fea927000d21e9994e6 P fc902de5b2274103ab5e76f967503afb: Bootstrap starting.
I20260503 19:53:32.499884 1487 tablet_bootstrap.cc:654] T df35135905b04fea927000d21e9994e6 P fc902de5b2274103ab5e76f967503afb: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:32.509053 1488 tablet_bootstrap.cc:492] T df35135905b04fea927000d21e9994e6 P b8594a7337c54dc58d701259120d361b: Bootstrap starting.
I20260503 19:53:32.510989 1488 tablet_bootstrap.cc:654] T df35135905b04fea927000d21e9994e6 P b8594a7337c54dc58d701259120d361b: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:32.513106 801 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54616:
name: "default.Table_e9dee4f0693a4aca81d75d2c25cefd09"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:32.534969 1488 tablet_bootstrap.cc:492] T df35135905b04fea927000d21e9994e6 P b8594a7337c54dc58d701259120d361b: No bootstrap required, opened a new log
I20260503 19:53:32.535686 1488 ts_tablet_manager.cc:1403] T df35135905b04fea927000d21e9994e6 P b8594a7337c54dc58d701259120d361b: Time spent bootstrapping tablet: real 0.026s user 0.000s sys 0.011s
I20260503 19:53:32.536607 1488 raft_consensus.cc:359] T df35135905b04fea927000d21e9994e6 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.536857 1488 raft_consensus.cc:385] T df35135905b04fea927000d21e9994e6 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:32.536935 1488 raft_consensus.cc:740] T df35135905b04fea927000d21e9994e6 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8594a7337c54dc58d701259120d361b, State: Initialized, Role: FOLLOWER
I20260503 19:53:32.538403 1487 tablet_bootstrap.cc:492] T df35135905b04fea927000d21e9994e6 P fc902de5b2274103ab5e76f967503afb: No bootstrap required, opened a new log
I20260503 19:53:32.538589 1487 ts_tablet_manager.cc:1403] T df35135905b04fea927000d21e9994e6 P fc902de5b2274103ab5e76f967503afb: Time spent bootstrapping tablet: real 0.041s user 0.004s sys 0.008s
I20260503 19:53:32.539422 1487 raft_consensus.cc:359] T df35135905b04fea927000d21e9994e6 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.539640 1487 raft_consensus.cc:385] T df35135905b04fea927000d21e9994e6 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:32.539740 1487 raft_consensus.cc:740] T df35135905b04fea927000d21e9994e6 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc902de5b2274103ab5e76f967503afb, State: Initialized, Role: FOLLOWER
I20260503 19:53:32.540999 1586 sys_catalog.cc:263] Verifying existing consensus state
I20260503 19:53:32.543947 1487 consensus_queue.cc:260] T df35135905b04fea927000d21e9994e6 P fc902de5b2274103ab5e76f967503afb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.544692 1487 ts_tablet_manager.cc:1434] T df35135905b04fea927000d21e9994e6 P fc902de5b2274103ab5e76f967503afb: Time spent starting tablet: real 0.006s user 0.002s sys 0.001s
I20260503 19:53:32.545185 1501 raft_consensus.cc:697] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b8594a7337c54dc58d701259120d361b [term 1 LEADER]: Becoming Leader. State: Replica: b8594a7337c54dc58d701259120d361b, State: Running, Role: LEADER
I20260503 19:53:32.545503 1501 consensus_queue.cc:237] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b8594a7337c54dc58d701259120d361b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.546279 1486 tablet_bootstrap.cc:492] T df35135905b04fea927000d21e9994e6 P b68b1efd1cc44e15b8c69e730229dbdd: No bootstrap required, opened a new log
I20260503 19:53:32.544920 1488 consensus_queue.cc:260] T df35135905b04fea927000d21e9994e6 P b8594a7337c54dc58d701259120d361b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.547063 1488 ts_tablet_manager.cc:1434] T df35135905b04fea927000d21e9994e6 P b8594a7337c54dc58d701259120d361b: Time spent starting tablet: real 0.011s user 0.003s sys 0.000s
I20260503 19:53:32.546546 1486 ts_tablet_manager.cc:1403] T df35135905b04fea927000d21e9994e6 P b68b1efd1cc44e15b8c69e730229dbdd: Time spent bootstrapping tablet: real 0.060s user 0.013s sys 0.009s
I20260503 19:53:32.548970 1486 raft_consensus.cc:359] T df35135905b04fea927000d21e9994e6 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.549335 1486 raft_consensus.cc:385] T df35135905b04fea927000d21e9994e6 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:32.549480 1486 raft_consensus.cc:740] T df35135905b04fea927000d21e9994e6 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b68b1efd1cc44e15b8c69e730229dbdd, State: Initialized, Role: FOLLOWER
I20260503 19:53:32.549847 1486 consensus_queue.cc:260] T df35135905b04fea927000d21e9994e6 P b68b1efd1cc44e15b8c69e730229dbdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.550945 1486 ts_tablet_manager.cc:1434] T df35135905b04fea927000d21e9994e6 P b68b1efd1cc44e15b8c69e730229dbdd: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260503 19:53:32.573107 1586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5: Bootstrap starting.
I20260503 19:53:32.591773 1115 heartbeater.cc:344] Connected to a master server at 127.0.105.60:42203
I20260503 19:53:32.591773 1256 heartbeater.cc:344] Connected to a master server at 127.0.105.60:42203
I20260503 19:53:32.602950 1394 heartbeater.cc:344] Connected to a master server at 127.0.105.60:42203
I20260503 19:53:32.603185 1394 heartbeater.cc:499] Master 127.0.105.60:42203 was elected leader, sending a full tablet report...
I20260503 19:53:32.603484 1586 log.cc:826] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5: Log is configured to *not* fsync() on all Append() calls
I20260503 19:53:32.605890 1394 heartbeater.cc:499] Master 127.0.105.60:42203 was elected leader, sending a full tablet report...
I20260503 19:53:32.621414 1586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260503 19:53:32.622263 1586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5: Bootstrap complete.
I20260503 19:53:32.626921 808 catalog_manager.cc:2174] created HMS catalog entry for table [id=40f19f1e7eaa4640b8c54d5e9e01c320]
I20260503 19:53:32.633499 808 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:54616 (request call id 25) took 1816 ms. Trace:
I20260503 19:53:32.632169 1586 raft_consensus.cc:359] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:32.633713 1586 raft_consensus.cc:740] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0cad11e600c48a3af9a3c56115696d5, State: Initialized, Role: FOLLOWER
I20260503 19:53:32.633651 808 rpcz_store.cc:276] 0503 19:53:30.817024 (+ 0us) service_pool.cc:167] Inserting onto call queue
0503 19:53:30.817084 (+ 60us) service_pool.cc:224] Handling call
0503 19:53:30.835196 (+ 18112us) catalog_manager.cc:2101] Acquired catalog manager lock
0503 19:53:30.838915 (+ 3719us) catalog_manager.cc:2147] Created new table and tablet info
0503 19:53:32.626847 (+1787932us) catalog_manager.cc:2173] Created new table in HMS catalog
0503 19:53:32.633154 (+ 6307us) catalog_manager.cc:2207] Wrote table and tablets to system table
0503 19:53:32.633324 (+ 170us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0503 19:53:32.633452 (+ 128us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0503 19:53:32.627814 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0503 19:53:32.627949 (+ 135us) write_op.cc:432] Acquiring schema lock in shared mode
0503 19:53:32.627955 (+ 6us) write_op.cc:435] Acquired schema lock
0503 19:53:32.627957 (+ 2us) tablet.cc:662] Decoding operations
0503 19:53:32.628047 (+ 90us) write_op.cc:620] Acquiring the partition lock for write op
0503 19:53:32.628061 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0503 19:53:32.628066 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0503 19:53:32.628144 (+ 78us) tablet.cc:701] Row locks acquired
0503 19:53:32.628147 (+ 3us) write_op.cc:260] PREPARE: finished
0503 19:53:32.628184 (+ 37us) write_op.cc:270] Start()
0503 19:53:32.628218 (+ 34us) write_op.cc:276] Timestamp: P: 1777838012628124 usec, L: 0
0503 19:53:32.628221 (+ 3us) op_driver.cc:348] REPLICATION: starting
0503 19:53:32.628409 (+ 188us) log.cc:844] Serialized 432 byte log entry
0503 19:53:32.632427 (+ 4018us) op_driver.cc:464] REPLICATION: finished
0503 19:53:32.632565 (+ 138us) write_op.cc:301] APPLY: starting
0503 19:53:32.632583 (+ 18us) tablet.cc:1367] starting BulkCheckPresence
0503 19:53:32.632629 (+ 46us) tablet.cc:1370] finished BulkCheckPresence
0503 19:53:32.632668 (+ 39us) tablet.cc:1372] starting ApplyRowOperation cycle
0503 19:53:32.632771 (+ 103us) tablet.cc:1383] finished ApplyRowOperation cycle
0503 19:53:32.632804 (+ 33us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0503 19:53:32.632811 (+ 7us) write_op.cc:312] APPLY: finished
0503 19:53:32.632883 (+ 72us) log.cc:844] Serialized 28 byte log entry
0503 19:53:32.632944 (+ 61us) write_op.cc:489] Releasing partition, row and schema locks
0503 19:53:32.632964 (+ 20us) write_op.cc:454] Released schema lock
0503 19:53:32.632970 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0503 19:53:32.633044 (+ 74us) write_op.cc:489] Releasing partition, row and schema locks
0503 19:53:32.633048 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":1602808,"Hive Metastore.run_cpu_time_us":319,"Hive Metastore.run_wall_time_us":184880,"spinlock_wait_cycles":376064,"child_traces":[["op",{"apply.queue_time_us":80,"apply.run_cpu_time_us":471,"apply.run_wall_time_us":470,"num_ops":2,"prepare.queue_time_us":83,"prepare.run_cpu_time_us":727,"prepare.run_wall_time_us":728,"raft.queue_time_us":847,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":270,"replication_time_us":4178}]]}
I20260503 19:53:32.634795 1586 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:32.651031 1050 tablet_service.cc:1511] Processing CreateTablet for tablet f391e07ff0e24bc5b3cf7e81342161fe (DEFAULT_TABLE table=default.table_f91787a2979849d78ba6ba4daf0b4e70 [id=40f19f1e7eaa4640b8c54d5e9e01c320]), partition=RANGE (key) PARTITION UNBOUNDED
I20260503 19:53:32.651521 1050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f391e07ff0e24bc5b3cf7e81342161fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:32.651712 1191 tablet_service.cc:1511] Processing CreateTablet for tablet f391e07ff0e24bc5b3cf7e81342161fe (DEFAULT_TABLE table=default.table_f91787a2979849d78ba6ba4daf0b4e70 [id=40f19f1e7eaa4640b8c54d5e9e01c320]), partition=RANGE (key) PARTITION UNBOUNDED
I20260503 19:53:32.652288 1191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f391e07ff0e24bc5b3cf7e81342161fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:32.657550 1488 tablet_bootstrap.cc:492] T f391e07ff0e24bc5b3cf7e81342161fe P b8594a7337c54dc58d701259120d361b: Bootstrap starting.
I20260503 19:53:32.657562 1329 tablet_service.cc:1511] Processing CreateTablet for tablet f391e07ff0e24bc5b3cf7e81342161fe (DEFAULT_TABLE table=default.table_f91787a2979849d78ba6ba4daf0b4e70 [id=40f19f1e7eaa4640b8c54d5e9e01c320]), partition=RANGE (key) PARTITION UNBOUNDED
I20260503 19:53:32.658145 1329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f391e07ff0e24bc5b3cf7e81342161fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:32.659288 1488 tablet_bootstrap.cc:654] T f391e07ff0e24bc5b3cf7e81342161fe P b8594a7337c54dc58d701259120d361b: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:32.668829 1486 tablet_bootstrap.cc:492] T f391e07ff0e24bc5b3cf7e81342161fe P b68b1efd1cc44e15b8c69e730229dbdd: Bootstrap starting.
I20260503 19:53:32.670730 808 catalog_manager.cc:5671] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b8594a7337c54dc58d701259120d361b reported cstate change: term changed from 0 to 1, leader changed from <none> to b8594a7337c54dc58d701259120d361b (127.0.105.1). New cstate: current_term: 1 leader_uuid: "b8594a7337c54dc58d701259120d361b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } health_report { overall_health: UNKNOWN } } }
I20260503 19:53:32.671308 1486 tablet_bootstrap.cc:654] T f391e07ff0e24bc5b3cf7e81342161fe P b68b1efd1cc44e15b8c69e730229dbdd: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:32.675307 1487 tablet_bootstrap.cc:492] T f391e07ff0e24bc5b3cf7e81342161fe P fc902de5b2274103ab5e76f967503afb: Bootstrap starting.
I20260503 19:53:32.676260 1488 tablet_bootstrap.cc:492] T f391e07ff0e24bc5b3cf7e81342161fe P b8594a7337c54dc58d701259120d361b: No bootstrap required, opened a new log
I20260503 19:53:32.676453 1488 ts_tablet_manager.cc:1403] T f391e07ff0e24bc5b3cf7e81342161fe P b8594a7337c54dc58d701259120d361b: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.004s
I20260503 19:53:32.677333 1487 tablet_bootstrap.cc:654] T f391e07ff0e24bc5b3cf7e81342161fe P fc902de5b2274103ab5e76f967503afb: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:32.677330 1488 raft_consensus.cc:359] T f391e07ff0e24bc5b3cf7e81342161fe P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.677544 1488 raft_consensus.cc:385] T f391e07ff0e24bc5b3cf7e81342161fe P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:32.677639 1488 raft_consensus.cc:740] T f391e07ff0e24bc5b3cf7e81342161fe P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8594a7337c54dc58d701259120d361b, State: Initialized, Role: FOLLOWER
I20260503 19:53:32.677875 1488 consensus_queue.cc:260] T f391e07ff0e24bc5b3cf7e81342161fe P b8594a7337c54dc58d701259120d361b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.678409 1488 ts_tablet_manager.cc:1434] T f391e07ff0e24bc5b3cf7e81342161fe P b8594a7337c54dc58d701259120d361b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260503 19:53:32.691830 1486 tablet_bootstrap.cc:492] T f391e07ff0e24bc5b3cf7e81342161fe P b68b1efd1cc44e15b8c69e730229dbdd: No bootstrap required, opened a new log
I20260503 19:53:32.692060 1486 ts_tablet_manager.cc:1403] T f391e07ff0e24bc5b3cf7e81342161fe P b68b1efd1cc44e15b8c69e730229dbdd: Time spent bootstrapping tablet: real 0.023s user 0.003s sys 0.007s
I20260503 19:53:32.693248 1486 raft_consensus.cc:359] T f391e07ff0e24bc5b3cf7e81342161fe P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.694067 1486 raft_consensus.cc:385] T f391e07ff0e24bc5b3cf7e81342161fe P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:32.694164 1486 raft_consensus.cc:740] T f391e07ff0e24bc5b3cf7e81342161fe P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b68b1efd1cc44e15b8c69e730229dbdd, State: Initialized, Role: FOLLOWER
I20260503 19:53:32.694411 1486 consensus_queue.cc:260] T f391e07ff0e24bc5b3cf7e81342161fe P b68b1efd1cc44e15b8c69e730229dbdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.694868 1394 heartbeater.cc:499] Master 127.0.105.60:42203 was elected leader, sending a full tablet report...
I20260503 19:53:32.698216 1487 tablet_bootstrap.cc:492] T f391e07ff0e24bc5b3cf7e81342161fe P fc902de5b2274103ab5e76f967503afb: No bootstrap required, opened a new log
I20260503 19:53:32.698390 1487 ts_tablet_manager.cc:1403] T f391e07ff0e24bc5b3cf7e81342161fe P fc902de5b2274103ab5e76f967503afb: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.009s
I20260503 19:53:32.699052 1486 ts_tablet_manager.cc:1434] T f391e07ff0e24bc5b3cf7e81342161fe P b68b1efd1cc44e15b8c69e730229dbdd: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260503 19:53:32.699280 1394 heartbeater.cc:499] Master 127.0.105.60:42203 was elected leader, sending a full tablet report...
I20260503 19:53:32.699230 1487 raft_consensus.cc:359] T f391e07ff0e24bc5b3cf7e81342161fe P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.699440 1487 raft_consensus.cc:385] T f391e07ff0e24bc5b3cf7e81342161fe P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:32.699613 1487 raft_consensus.cc:740] T f391e07ff0e24bc5b3cf7e81342161fe P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc902de5b2274103ab5e76f967503afb, State: Initialized, Role: FOLLOWER
I20260503 19:53:32.699904 1487 consensus_queue.cc:260] T f391e07ff0e24bc5b3cf7e81342161fe P fc902de5b2274103ab5e76f967503afb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.702098 1487 ts_tablet_manager.cc:1434] T f391e07ff0e24bc5b3cf7e81342161fe P fc902de5b2274103ab5e76f967503afb: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260503 19:53:32.704257 808 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54616:
name: "default.Table_8a28426e04cf42b8ae2288a7c2950b1b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:32.711681 1597 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:32.712158 1597 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260503 19:53:32.713217 1586 sys_catalog.cc:565] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260503 19:53:32.748515 807 catalog_manager.cc:2174] created HMS catalog entry for table [id=f9d69e676109494f8ee9b41f557f4201]
I20260503 19:53:32.759065 807 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:54616 (request call id 26) took 1941 ms. Trace:
I20260503 19:53:32.759270 807 rpcz_store.cc:276] 0503 19:53:30.817218 (+ 0us) service_pool.cc:167] Inserting onto call queue
0503 19:53:30.817383 (+ 165us) service_pool.cc:224] Handling call
0503 19:53:30.838726 (+ 21343us) catalog_manager.cc:2101] Acquired catalog manager lock
0503 19:53:30.839498 (+ 772us) catalog_manager.cc:2147] Created new table and tablet info
0503 19:53:32.748436 (+1908938us) catalog_manager.cc:2173] Created new table in HMS catalog
0503 19:53:32.758667 (+ 10231us) catalog_manager.cc:2207] Wrote table and tablets to system table
0503 19:53:32.758859 (+ 192us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0503 19:53:32.759011 (+ 152us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0503 19:53:32.749708 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0503 19:53:32.749840 (+ 132us) write_op.cc:432] Acquiring schema lock in shared mode
0503 19:53:32.749846 (+ 6us) write_op.cc:435] Acquired schema lock
0503 19:53:32.749849 (+ 3us) tablet.cc:662] Decoding operations
0503 19:53:32.749954 (+ 105us) write_op.cc:620] Acquiring the partition lock for write op
0503 19:53:32.749971 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0503 19:53:32.749978 (+ 7us) tablet.cc:685] Acquiring locks for 2 operations
0503 19:53:32.750081 (+ 103us) tablet.cc:701] Row locks acquired
0503 19:53:32.750085 (+ 4us) write_op.cc:260] PREPARE: finished
0503 19:53:32.750136 (+ 51us) write_op.cc:270] Start()
0503 19:53:32.750174 (+ 38us) write_op.cc:276] Timestamp: P: 1777838012750047 usec, L: 0
0503 19:53:32.750177 (+ 3us) op_driver.cc:348] REPLICATION: starting
0503 19:53:32.750382 (+ 205us) log.cc:844] Serialized 432 byte log entry
0503 19:53:32.754668 (+ 4286us) op_driver.cc:464] REPLICATION: finished
0503 19:53:32.754859 (+ 191us) write_op.cc:301] APPLY: starting
0503 19:53:32.754884 (+ 25us) tablet.cc:1367] starting BulkCheckPresence
0503 19:53:32.754948 (+ 64us) tablet.cc:1370] finished BulkCheckPresence
0503 19:53:32.754997 (+ 49us) tablet.cc:1372] starting ApplyRowOperation cycle
0503 19:53:32.755151 (+ 154us) tablet.cc:1383] finished ApplyRowOperation cycle
0503 19:53:32.755203 (+ 52us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0503 19:53:32.755215 (+ 12us) write_op.cc:312] APPLY: finished
0503 19:53:32.755328 (+ 113us) log.cc:844] Serialized 28 byte log entry
0503 19:53:32.755422 (+ 94us) write_op.cc:489] Releasing partition, row and schema locks
0503 19:53:32.755455 (+ 33us) write_op.cc:454] Released schema lock
0503 19:53:32.755464 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0503 19:53:32.755558 (+ 94us) write_op.cc:489] Releasing partition, row and schema locks
0503 19:53:32.755576 (+ 18us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":1787196,"Hive Metastore.run_cpu_time_us":337,"Hive Metastore.run_wall_time_us":121529,"child_traces":[["op",{"apply.queue_time_us":124,"apply.run_cpu_time_us":685,"apply.run_wall_time_us":684,"num_ops":2,"prepare.queue_time_us":194,"prepare.run_cpu_time_us":803,"prepare.run_wall_time_us":802,"raft.queue_time_us":158,"raft.run_cpu_time_us":318,"raft.run_wall_time_us":317,"replication_time_us":4444}]]}
I20260503 19:53:32.793514 1050 tablet_service.cc:1511] Processing CreateTablet for tablet a6cc88aae841477d990ed433c705a803 (DEFAULT_TABLE table=default.table_11964e32037841689422d2a453fff70f [id=f9d69e676109494f8ee9b41f557f4201]), partition=RANGE (key) PARTITION UNBOUNDED
I20260503 19:53:32.793946 1615 catalog_manager.cc:1269] Loaded cluster ID: 636a1bc8fd6746489a9e892cd3f8ca43
I20260503 19:53:32.794018 1050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6cc88aae841477d990ed433c705a803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:32.794044 1615 catalog_manager.cc:1562] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5: loading cluster ID for follower catalog manager: success
I20260503 19:53:32.795316 1191 tablet_service.cc:1511] Processing CreateTablet for tablet a6cc88aae841477d990ed433c705a803 (DEFAULT_TABLE table=default.table_11964e32037841689422d2a453fff70f [id=f9d69e676109494f8ee9b41f557f4201]), partition=RANGE (key) PARTITION UNBOUNDED
I20260503 19:53:32.795773 1191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6cc88aae841477d990ed433c705a803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:32.801486 1329 tablet_service.cc:1511] Processing CreateTablet for tablet a6cc88aae841477d990ed433c705a803 (DEFAULT_TABLE table=default.table_11964e32037841689422d2a453fff70f [id=f9d69e676109494f8ee9b41f557f4201]), partition=RANGE (key) PARTITION UNBOUNDED
I20260503 19:53:32.801560 1493 raft_consensus.cc:493] T df35135905b04fea927000d21e9994e6 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:53:32.801972 1329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6cc88aae841477d990ed433c705a803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:32.802016 1493 raft_consensus.cc:515] T df35135905b04fea927000d21e9994e6 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.802768 1493 leader_election.cc:290] T df35135905b04fea927000d21e9994e6 P b68b1efd1cc44e15b8c69e730229dbdd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8594a7337c54dc58d701259120d361b (127.0.105.1:40289), fc902de5b2274103ab5e76f967503afb (127.0.105.2:46339)
I20260503 19:53:32.803615 1069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df35135905b04fea927000d21e9994e6" candidate_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8594a7337c54dc58d701259120d361b" is_pre_election: true
I20260503 19:53:32.803838 1069 raft_consensus.cc:2468] T df35135905b04fea927000d21e9994e6 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b68b1efd1cc44e15b8c69e730229dbdd in term 0.
I20260503 19:53:32.804570 1210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df35135905b04fea927000d21e9994e6" candidate_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc902de5b2274103ab5e76f967503afb" is_pre_election: true
I20260503 19:53:32.804783 1210 raft_consensus.cc:2468] T df35135905b04fea927000d21e9994e6 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b68b1efd1cc44e15b8c69e730229dbdd in term 0.
I20260503 19:53:32.805117 1284 leader_election.cc:304] T df35135905b04fea927000d21e9994e6 P b68b1efd1cc44e15b8c69e730229dbdd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b68b1efd1cc44e15b8c69e730229dbdd, b8594a7337c54dc58d701259120d361b; no voters:
I20260503 19:53:32.806064 1493 raft_consensus.cc:2804] T df35135905b04fea927000d21e9994e6 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260503 19:53:32.806159 1493 raft_consensus.cc:493] T df35135905b04fea927000d21e9994e6 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260503 19:53:32.806252 1493 raft_consensus.cc:3060] T df35135905b04fea927000d21e9994e6 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Advancing to term 1
I20260503 19:53:32.809541 1493 raft_consensus.cc:515] T df35135905b04fea927000d21e9994e6 P b68b1efd1cc44e15b8c69e730229dbdd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.810559 1069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df35135905b04fea927000d21e9994e6" candidate_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8594a7337c54dc58d701259120d361b"
I20260503 19:53:32.810773 1069 raft_consensus.cc:3060] T df35135905b04fea927000d21e9994e6 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Advancing to term 1
I20260503 19:53:32.810868 1615 catalog_manager.cc:1584] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5: acquiring CA information for follower catalog manager: success
I20260503 19:53:32.813599 1615 catalog_manager.cc:1612] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260503 19:53:32.815622 807 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54616:
name: "default.Table_1eff9f94b0c84e7180c54349d66db8e4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:32.816502 1069 raft_consensus.cc:2468] T df35135905b04fea927000d21e9994e6 P b8594a7337c54dc58d701259120d361b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b68b1efd1cc44e15b8c69e730229dbdd in term 1.
I20260503 19:53:32.816995 1284 leader_election.cc:304] T df35135905b04fea927000d21e9994e6 P b68b1efd1cc44e15b8c69e730229dbdd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b68b1efd1cc44e15b8c69e730229dbdd, b8594a7337c54dc58d701259120d361b; no voters:
I20260503 19:53:32.817392 1563 raft_consensus.cc:2804] T df35135905b04fea927000d21e9994e6 P b68b1efd1cc44e15b8c69e730229dbdd [term 1 FOLLOWER]: Leader election won for term 1
I20260503 19:53:32.818185 1210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df35135905b04fea927000d21e9994e6" candidate_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc902de5b2274103ab5e76f967503afb"
I20260503 19:53:32.818418 1210 raft_consensus.cc:3060] T df35135905b04fea927000d21e9994e6 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Advancing to term 1
I20260503 19:53:32.823520 1486 tablet_bootstrap.cc:492] T a6cc88aae841477d990ed433c705a803 P b68b1efd1cc44e15b8c69e730229dbdd: Bootstrap starting.
I20260503 19:53:32.825497 1486 tablet_bootstrap.cc:654] T a6cc88aae841477d990ed433c705a803 P b68b1efd1cc44e15b8c69e730229dbdd: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:32.826098 1487 tablet_bootstrap.cc:492] T a6cc88aae841477d990ed433c705a803 P fc902de5b2274103ab5e76f967503afb: Bootstrap starting.
I20260503 19:53:32.827104 1488 tablet_bootstrap.cc:492] T a6cc88aae841477d990ed433c705a803 P b8594a7337c54dc58d701259120d361b: Bootstrap starting.
I20260503 19:53:32.828256 1487 tablet_bootstrap.cc:654] T a6cc88aae841477d990ed433c705a803 P fc902de5b2274103ab5e76f967503afb: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:32.828807 1488 tablet_bootstrap.cc:654] T a6cc88aae841477d990ed433c705a803 P b8594a7337c54dc58d701259120d361b: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:32.829353 1563 raft_consensus.cc:697] T df35135905b04fea927000d21e9994e6 P b68b1efd1cc44e15b8c69e730229dbdd [term 1 LEADER]: Becoming Leader. State: Replica: b68b1efd1cc44e15b8c69e730229dbdd, State: Running, Role: LEADER
I20260503 19:53:32.829845 1394 heartbeater.cc:344] Connected to a master server at 127.0.105.60:42203
I20260503 19:53:32.830047 1394 heartbeater.cc:499] Master 127.0.105.60:42203 was elected leader, sending a full tablet report...
I20260503 19:53:32.823221 1210 raft_consensus.cc:2468] T df35135905b04fea927000d21e9994e6 P fc902de5b2274103ab5e76f967503afb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b68b1efd1cc44e15b8c69e730229dbdd in term 1.
I20260503 19:53:32.829757 1563 consensus_queue.cc:237] T df35135905b04fea927000d21e9994e6 P b68b1efd1cc44e15b8c69e730229dbdd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.833514 805 catalog_manager.cc:2174] created HMS catalog entry for table [id=04d62d016fc24931ae6f439b1c6152c5]
I20260503 19:53:32.836094 1493 leader_election.cc:290] T df35135905b04fea927000d21e9994e6 P b68b1efd1cc44e15b8c69e730229dbdd [CANDIDATE]: Term 1 election: Requested vote from peers b8594a7337c54dc58d701259120d361b (127.0.105.1:40289), fc902de5b2274103ab5e76f967503afb (127.0.105.2:46339)
I20260503 19:53:32.840324 805 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:54616 (request call id 28) took 2022 ms. Trace:
I20260503 19:53:32.840454 805 rpcz_store.cc:276] 0503 19:53:30.817618 (+ 0us) service_pool.cc:167] Inserting onto call queue
0503 19:53:30.817812 (+ 194us) service_pool.cc:224] Handling call
0503 19:53:30.839690 (+ 21878us) catalog_manager.cc:2101] Acquired catalog manager lock
0503 19:53:30.840107 (+ 417us) catalog_manager.cc:2147] Created new table and tablet info
0503 19:53:32.833453 (+1993346us) catalog_manager.cc:2173] Created new table in HMS catalog
0503 19:53:32.840074 (+ 6621us) catalog_manager.cc:2207] Wrote table and tablets to system table
0503 19:53:32.840189 (+ 115us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0503 19:53:32.840287 (+ 98us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0503 19:53:32.834555 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0503 19:53:32.834691 (+ 136us) write_op.cc:432] Acquiring schema lock in shared mode
0503 19:53:32.834697 (+ 6us) write_op.cc:435] Acquired schema lock
0503 19:53:32.834700 (+ 3us) tablet.cc:662] Decoding operations
0503 19:53:32.834791 (+ 91us) write_op.cc:620] Acquiring the partition lock for write op
0503 19:53:32.834805 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0503 19:53:32.834810 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0503 19:53:32.834899 (+ 89us) tablet.cc:701] Row locks acquired
0503 19:53:32.834901 (+ 2us) write_op.cc:260] PREPARE: finished
0503 19:53:32.834942 (+ 41us) write_op.cc:270] Start()
0503 19:53:32.834981 (+ 39us) write_op.cc:276] Timestamp: P: 1777838012834839 usec, L: 0
0503 19:53:32.834985 (+ 4us) op_driver.cc:348] REPLICATION: starting
0503 19:53:32.835178 (+ 193us) log.cc:844] Serialized 432 byte log entry
0503 19:53:32.839169 (+ 3991us) op_driver.cc:464] REPLICATION: finished
0503 19:53:32.839306 (+ 137us) write_op.cc:301] APPLY: starting
0503 19:53:32.839323 (+ 17us) tablet.cc:1367] starting BulkCheckPresence
0503 19:53:32.839370 (+ 47us) tablet.cc:1370] finished BulkCheckPresence
0503 19:53:32.839403 (+ 33us) tablet.cc:1372] starting ApplyRowOperation cycle
0503 19:53:32.839520 (+ 117us) tablet.cc:1383] finished ApplyRowOperation cycle
0503 19:53:32.839556 (+ 36us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0503 19:53:32.839564 (+ 8us) write_op.cc:312] APPLY: finished
0503 19:53:32.839650 (+ 86us) log.cc:844] Serialized 28 byte log entry
0503 19:53:32.839710 (+ 60us) write_op.cc:489] Releasing partition, row and schema locks
0503 19:53:32.839734 (+ 24us) write_op.cc:454] Released schema lock
0503 19:53:32.839740 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0503 19:53:32.839808 (+ 68us) write_op.cc:489] Releasing partition, row and schema locks
0503 19:53:32.839812 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":1908134,"Hive Metastore.run_cpu_time_us":289,"Hive Metastore.run_wall_time_us":84992,"child_traces":[["op",{"apply.queue_time_us":93,"apply.run_cpu_time_us":492,"apply.run_wall_time_us":491,"num_ops":2,"prepare.queue_time_us":91,"prepare.run_cpu_time_us":731,"prepare.run_wall_time_us":731,"raft.queue_time_us":923,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":248,"replication_time_us":4155}]]}
I20260503 19:53:32.841583 805 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54616:
name: "default.Table_49084d26aae440ac958d415c2b291948"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:32.855276 1488 tablet_bootstrap.cc:492] T a6cc88aae841477d990ed433c705a803 P b8594a7337c54dc58d701259120d361b: No bootstrap required, opened a new log
I20260503 19:53:32.855479 1488 ts_tablet_manager.cc:1403] T a6cc88aae841477d990ed433c705a803 P b8594a7337c54dc58d701259120d361b: Time spent bootstrapping tablet: real 0.028s user 0.000s sys 0.009s
I20260503 19:53:32.856554 1488 raft_consensus.cc:359] T a6cc88aae841477d990ed433c705a803 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.856889 1488 raft_consensus.cc:385] T a6cc88aae841477d990ed433c705a803 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:32.856989 1488 raft_consensus.cc:740] T a6cc88aae841477d990ed433c705a803 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8594a7337c54dc58d701259120d361b, State: Initialized, Role: FOLLOWER
I20260503 19:53:32.857308 1488 consensus_queue.cc:260] T a6cc88aae841477d990ed433c705a803 P b8594a7337c54dc58d701259120d361b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.859073 1115 heartbeater.cc:344] Connected to a master server at 127.0.105.60:42203
I20260503 19:53:32.859760 1488 ts_tablet_manager.cc:1434] T a6cc88aae841477d990ed433c705a803 P b8594a7337c54dc58d701259120d361b: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260503 19:53:32.861665 1329 tablet_service.cc:1511] Processing CreateTablet for tablet 6726412cbfd947d2849ac9774d4e86f4 (DEFAULT_TABLE table=default.table_15b3325c8812438786ade0fa5c759e21 [id=04d62d016fc24931ae6f439b1c6152c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260503 19:53:32.862046 1535 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "b8594a7337c54dc58d701259120d361b" instance_seqno: 1777838008819588) as {username='slave'} at 127.0.105.1:47881; Asking this server to re-register.
I20260503 19:53:32.862313 1329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6726412cbfd947d2849ac9774d4e86f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:32.862967 1115 heartbeater.cc:461] Registering TS with master...
I20260503 19:53:32.865136 1535 ts_manager.cc:194] Registered new tserver with Master: b8594a7337c54dc58d701259120d361b (127.0.105.1:40289)
I20260503 19:53:32.865711 1191 tablet_service.cc:1511] Processing CreateTablet for tablet 6726412cbfd947d2849ac9774d4e86f4 (DEFAULT_TABLE table=default.table_15b3325c8812438786ade0fa5c759e21 [id=04d62d016fc24931ae6f439b1c6152c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260503 19:53:32.866303 1191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6726412cbfd947d2849ac9774d4e86f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:32.867161 1486 tablet_bootstrap.cc:492] T a6cc88aae841477d990ed433c705a803 P b68b1efd1cc44e15b8c69e730229dbdd: No bootstrap required, opened a new log
I20260503 19:53:32.867357 1486 ts_tablet_manager.cc:1403] T a6cc88aae841477d990ed433c705a803 P b68b1efd1cc44e15b8c69e730229dbdd: Time spent bootstrapping tablet: real 0.044s user 0.008s sys 0.001s
I20260503 19:53:32.867694 1050 tablet_service.cc:1511] Processing CreateTablet for tablet 6726412cbfd947d2849ac9774d4e86f4 (DEFAULT_TABLE table=default.table_15b3325c8812438786ade0fa5c759e21 [id=04d62d016fc24931ae6f439b1c6152c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260503 19:53:32.868168 1050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6726412cbfd947d2849ac9774d4e86f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:32.868196 1486 raft_consensus.cc:359] T a6cc88aae841477d990ed433c705a803 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.868435 1486 raft_consensus.cc:385] T a6cc88aae841477d990ed433c705a803 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:32.868680 1486 raft_consensus.cc:740] T a6cc88aae841477d990ed433c705a803 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b68b1efd1cc44e15b8c69e730229dbdd, State: Initialized, Role: FOLLOWER
I20260503 19:53:32.868614 1589 consensus_queue.cc:1048] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b8594a7337c54dc58d701259120d361b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260503 19:53:32.869550 1486 consensus_queue.cc:260] T a6cc88aae841477d990ed433c705a803 P b68b1efd1cc44e15b8c69e730229dbdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.870260 1486 ts_tablet_manager.cc:1434] T a6cc88aae841477d990ed433c705a803 P b68b1efd1cc44e15b8c69e730229dbdd: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260503 19:53:32.872818 1501 consensus_queue.cc:1048] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b8594a7337c54dc58d701259120d361b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260503 19:53:32.874107 1256 heartbeater.cc:344] Connected to a master server at 127.0.105.60:42203
I20260503 19:53:32.890296 804 catalog_manager.cc:2174] created HMS catalog entry for table [id=bd45994325af4ed9ba34633dea7e140e]
I20260503 19:53:32.893880 1487 tablet_bootstrap.cc:492] T a6cc88aae841477d990ed433c705a803 P fc902de5b2274103ab5e76f967503afb: No bootstrap required, opened a new log
I20260503 19:53:32.894102 1487 ts_tablet_manager.cc:1403] T a6cc88aae841477d990ed433c705a803 P fc902de5b2274103ab5e76f967503afb: Time spent bootstrapping tablet: real 0.068s user 0.004s sys 0.004s
I20260503 19:53:32.895004 1487 raft_consensus.cc:359] T a6cc88aae841477d990ed433c705a803 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.895222 1487 raft_consensus.cc:385] T a6cc88aae841477d990ed433c705a803 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:32.895301 1487 raft_consensus.cc:740] T a6cc88aae841477d990ed433c705a803 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc902de5b2274103ab5e76f967503afb, State: Initialized, Role: FOLLOWER
I20260503 19:53:32.899263 804 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:54616 (request call id 29) took 2081 ms. Trace:
I20260503 19:53:32.899408 804 rpcz_store.cc:276] 0503 19:53:30.817740 (+ 0us) service_pool.cc:167] Inserting onto call queue
0503 19:53:30.818024 (+ 284us) service_pool.cc:224] Handling call
0503 19:53:30.836491 (+ 18467us) catalog_manager.cc:2101] Acquired catalog manager lock
0503 19:53:30.840371 (+ 3880us) catalog_manager.cc:2147] Created new table and tablet info
0503 19:53:32.890242 (+2049871us) catalog_manager.cc:2173] Created new table in HMS catalog
0503 19:53:32.898990 (+ 8748us) catalog_manager.cc:2207] Wrote table and tablets to system table
0503 19:53:32.899113 (+ 123us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0503 19:53:32.899222 (+ 109us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0503 19:53:32.891140 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0503 19:53:32.891314 (+ 174us) write_op.cc:432] Acquiring schema lock in shared mode
0503 19:53:32.891323 (+ 9us) write_op.cc:435] Acquired schema lock
0503 19:53:32.891327 (+ 4us) tablet.cc:662] Decoding operations
0503 19:53:32.891454 (+ 127us) write_op.cc:620] Acquiring the partition lock for write op
0503 19:53:32.891474 (+ 20us) write_op.cc:641] Partition lock acquired for write op
0503 19:53:32.891482 (+ 8us) tablet.cc:685] Acquiring locks for 2 operations
0503 19:53:32.891594 (+ 112us) tablet.cc:701] Row locks acquired
0503 19:53:32.891599 (+ 5us) write_op.cc:260] PREPARE: finished
0503 19:53:32.891645 (+ 46us) write_op.cc:270] Start()
0503 19:53:32.891690 (+ 45us) write_op.cc:276] Timestamp: P: 1777838012891530 usec, L: 0
0503 19:53:32.891696 (+ 6us) op_driver.cc:348] REPLICATION: starting
0503 19:53:32.891972 (+ 276us) log.cc:844] Serialized 432 byte log entry
0503 19:53:32.898184 (+ 6212us) op_driver.cc:464] REPLICATION: finished
0503 19:53:32.898344 (+ 160us) write_op.cc:301] APPLY: starting
0503 19:53:32.898366 (+ 22us) tablet.cc:1367] starting BulkCheckPresence
0503 19:53:32.898437 (+ 71us) tablet.cc:1370] finished BulkCheckPresence
0503 19:53:32.898469 (+ 32us) tablet.cc:1372] starting ApplyRowOperation cycle
0503 19:53:32.898571 (+ 102us) tablet.cc:1383] finished ApplyRowOperation cycle
0503 19:53:32.898605 (+ 34us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0503 19:53:32.898612 (+ 7us) write_op.cc:312] APPLY: finished
0503 19:53:32.898694 (+ 82us) log.cc:844] Serialized 28 byte log entry
0503 19:53:32.898774 (+ 80us) write_op.cc:489] Releasing partition, row and schema locks
0503 19:53:32.898797 (+ 23us) write_op.cc:454] Released schema lock
0503 19:53:32.898804 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0503 19:53:32.898875 (+ 71us) write_op.cc:489] Releasing partition, row and schema locks
0503 19:53:32.898879 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":1992942,"Hive Metastore.run_cpu_time_us":282,"Hive Metastore.run_wall_time_us":56598,"child_traces":[["op",{"apply.queue_time_us":100,"apply.run_cpu_time_us":526,"apply.run_wall_time_us":525,"num_ops":2,"prepare.queue_time_us":58,"prepare.run_cpu_time_us":981,"prepare.run_wall_time_us":981,"raft.queue_time_us":935,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":267,"replication_time_us":6447}]]}
I20260503 19:53:32.906490 804 master_service.cc:954] ReplaceTablet: received request to replace tablet 65f0f8cedacf4024ae7bbb741a25c2f8 from {username='slave'} at 127.0.0.1:54616
I20260503 19:53:32.910117 1487 consensus_queue.cc:260] T a6cc88aae841477d990ed433c705a803 P fc902de5b2274103ab5e76f967503afb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.911500 1487 ts_tablet_manager.cc:1434] T a6cc88aae841477d990ed433c705a803 P fc902de5b2274103ab5e76f967503afb: Time spent starting tablet: real 0.017s user 0.002s sys 0.001s
I20260503 19:53:32.914112 804 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: Sending DeleteTablet for 3 replicas of tablet 65f0f8cedacf4024ae7bbb741a25c2f8
I20260503 19:53:32.915335 1535 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" instance_seqno: 1777838010067456) as {username='slave'} at 127.0.105.3:41249; Asking this server to re-register.
I20260503 19:53:32.918033 1534 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "fc902de5b2274103ab5e76f967503afb" instance_seqno: 1777838009464378) as {username='slave'} at 127.0.105.2:44255; Asking this server to re-register.
I20260503 19:53:32.919114 1049 tablet_service.cc:1558] Processing DeleteTablet for tablet 65f0f8cedacf4024ae7bbb741a25c2f8 with delete_type TABLET_DATA_DELETED (replaced by tablet c4ef894c0fdb4414a6799627641a8857) from {username='slave'} at 127.0.0.1:47844
I20260503 19:53:32.922930 1190 tablet_service.cc:1511] Processing CreateTablet for tablet 75dd4e31d81c4d5da2224e29e4f51e8d (DEFAULT_TABLE table=default.table_db1c67410d564682adca12ef0e942ca2 [id=bd45994325af4ed9ba34633dea7e140e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260503 19:53:32.923885 1190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75dd4e31d81c4d5da2224e29e4f51e8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:32.924031 1256 heartbeater.cc:461] Registering TS with master...
I20260503 19:53:32.925693 1394 heartbeater.cc:461] Registering TS with master...
F20260503 19:53:32.927441 971 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 10 received events: NotificationEvent(eventId=1, eventTime=1777838011, eventType=CREATE_TABLE, dbName=default, tableName=table_cb2fc46da15e4d3cbc25e42aac22a26c, message={"server":"","servicePrincipal":"","db":"default","table":"table_cb2fc46da15e4d3cbc25e42aac22a26c","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_cb2fc46da15e4d3cbc25e42aac22a26c\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1777838010},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/warehouse/table_cb2fc46da15e4d3cbc25e42aac22a26c\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_cb2fc46da15e4d3cbc25e42aac22a26c\",\"kudu.table_id\":\"f40076244f874a50b8ee159c9c1b0786\",\"transient_lastDdlTime\":\"1777838010\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"636a1bc8fd6746489a9e892cd3f8ca43\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1777838011,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1777838011, eventType=CREATE_TABLE, dbName=default, tableName=table_f423a73d5c224651bb5eb676ee98c5e3, message={"server":"","servicePrincipal":"","db":"default","table":"table_f423a73d5c224651bb5eb676ee98c5e3","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_f423a73d5c224651bb5eb676ee98c5e3\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1777838010},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/warehouse/table_f423a73d5c224651bb5eb676ee98c5e3\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_f423a73d5c224651bb5eb676ee98c5e3\",\"kudu.table_id\":\"f8aedcc7a5c34071a86b38b46361455f\",\"transient_lastDdlTime\":\"1777838010\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"636a1bc8fd6746489a9e892cd3f8ca43\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1777838011,"files":[]}, messageFormat=json-0.2, catName=hive);
I20260503 19:53:32.927469 1534 ts_manager.cc:194] Registered new tserver with Master: b68b1efd1cc44e15b8c69e730229dbdd (127.0.105.3:41575)
*** Check failure stack trace: ***
I20260503 19:53:32.929973 1535 ts_manager.cc:194] Registered new tserver with Master: fc902de5b2274103ab5e76f967503afb (127.0.105.2:46339)
I20260503 19:53:32.931071 1501 raft_consensus.cc:493] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:53:32.931200 1328 tablet_service.cc:1511] Processing CreateTablet for tablet 75dd4e31d81c4d5da2224e29e4f51e8d (DEFAULT_TABLE table=default.table_db1c67410d564682adca12ef0e942ca2 [id=bd45994325af4ed9ba34633dea7e140e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260503 19:53:32.931273 1501 raft_consensus.cc:515] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.931602 1328 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75dd4e31d81c4d5da2224e29e4f51e8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:32.931556 1048 tablet_service.cc:1511] Processing CreateTablet for tablet 75dd4e31d81c4d5da2224e29e4f51e8d (DEFAULT_TABLE table=default.table_db1c67410d564682adca12ef0e942ca2 [id=bd45994325af4ed9ba34633dea7e140e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260503 19:53:32.931979 1501 leader_election.cc:290] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b8594a7337c54dc58d701259120d361b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc902de5b2274103ab5e76f967503afb (127.0.105.2:46339), b68b1efd1cc44e15b8c69e730229dbdd (127.0.105.3:41575)
I20260503 19:53:32.931984 1048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75dd4e31d81c4d5da2224e29e4f51e8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:32.932549 1211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65f0f8cedacf4024ae7bbb741a25c2f8" candidate_uuid: "b8594a7337c54dc58d701259120d361b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc902de5b2274103ab5e76f967503afb" is_pre_election: true
I20260503 19:53:32.932746 1211 raft_consensus.cc:2468] T 65f0f8cedacf4024ae7bbb741a25c2f8 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8594a7337c54dc58d701259120d361b in term 0.
I20260503 19:53:32.935580 1349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65f0f8cedacf4024ae7bbb741a25c2f8" candidate_uuid: "b8594a7337c54dc58d701259120d361b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" is_pre_election: true
I20260503 19:53:32.935791 1349 raft_consensus.cc:2468] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8594a7337c54dc58d701259120d361b in term 0.
I20260503 19:53:32.936986 1005 leader_election.cc:304] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b8594a7337c54dc58d701259120d361b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8594a7337c54dc58d701259120d361b, fc902de5b2274103ab5e76f967503afb; no voters:
I20260503 19:53:32.937283 1589 raft_consensus.cc:2804] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260503 19:53:32.937388 1589 raft_consensus.cc:493] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260503 19:53:32.937466 1589 raft_consensus.cc:3060] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Advancing to term 1
I20260503 19:53:32.939566 1589 raft_consensus.cc:515] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b8594a7337c54dc58d701259120d361b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.940164 1589 leader_election.cc:290] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b8594a7337c54dc58d701259120d361b [CANDIDATE]: Term 1 election: Requested vote from peers fc902de5b2274103ab5e76f967503afb (127.0.105.2:46339), b68b1efd1cc44e15b8c69e730229dbdd (127.0.105.3:41575)
I20260503 19:53:32.940209 1189 tablet_service.cc:1558] Processing DeleteTablet for tablet 65f0f8cedacf4024ae7bbb741a25c2f8 with delete_type TABLET_DATA_DELETED (replaced by tablet c4ef894c0fdb4414a6799627641a8857) from {username='slave'} at 127.0.0.1:58604
I20260503 19:53:32.940498 1549 tablet_replica.cc:333] T 65f0f8cedacf4024ae7bbb741a25c2f8 P fc902de5b2274103ab5e76f967503afb: stopping tablet replica
I20260503 19:53:32.940526 1486 tablet_bootstrap.cc:492] T 75dd4e31d81c4d5da2224e29e4f51e8d P b68b1efd1cc44e15b8c69e730229dbdd: Bootstrap starting.
I20260503 19:53:32.940688 1549 raft_consensus.cc:2243] T 65f0f8cedacf4024ae7bbb741a25c2f8 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260503 19:53:32.940845 1549 raft_consensus.cc:2272] T 65f0f8cedacf4024ae7bbb741a25c2f8 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260503 19:53:32.941370 1549 ts_tablet_manager.cc:1916] T 65f0f8cedacf4024ae7bbb741a25c2f8 P fc902de5b2274103ab5e76f967503afb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260503 19:53:32.942833 1486 tablet_bootstrap.cc:654] T 75dd4e31d81c4d5da2224e29e4f51e8d P b68b1efd1cc44e15b8c69e730229dbdd: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:32.945616 1349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65f0f8cedacf4024ae7bbb741a25c2f8" candidate_uuid: "b8594a7337c54dc58d701259120d361b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b68b1efd1cc44e15b8c69e730229dbdd"
I20260503 19:53:32.945824 1349 raft_consensus.cc:3060] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Advancing to term 1
I20260503 19:53:32.948320 1328 tablet_service.cc:1558] Processing DeleteTablet for tablet 65f0f8cedacf4024ae7bbb741a25c2f8 with delete_type TABLET_DATA_DELETED (replaced by tablet c4ef894c0fdb4414a6799627641a8857) from {username='slave'} at 127.0.0.1:36426
I20260503 19:53:32.950059 804 catalog_manager.cc:6741] ReplaceTablet: tablet 65f0f8cedacf4024ae7bbb741a25c2f8 deleted and replaced by tablet c4ef894c0fdb4414a6799627641a8857
I20260503 19:53:32.951987 1349 raft_consensus.cc:2468] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b68b1efd1cc44e15b8c69e730229dbdd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8594a7337c54dc58d701259120d361b in term 1.
I20260503 19:53:32.952443 1005 leader_election.cc:304] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b8594a7337c54dc58d701259120d361b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b68b1efd1cc44e15b8c69e730229dbdd, b8594a7337c54dc58d701259120d361b; no voters:
I20260503 19:53:32.952785 1589 raft_consensus.cc:2804] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b8594a7337c54dc58d701259120d361b [term 1 FOLLOWER]: Leader election won for term 1
I20260503 19:53:32.953039 1589 raft_consensus.cc:697] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b8594a7337c54dc58d701259120d361b [term 1 LEADER]: Becoming Leader. State: Replica: b8594a7337c54dc58d701259120d361b, State: Running, Role: LEADER
I20260503 19:53:32.953338 1589 consensus_queue.cc:237] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b8594a7337c54dc58d701259120d361b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:32.958627 1210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65f0f8cedacf4024ae7bbb741a25c2f8" candidate_uuid: "b8594a7337c54dc58d701259120d361b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc902de5b2274103ab5e76f967503afb"
W20260503 19:53:32.959239 1005 leader_election.cc:343] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b8594a7337c54dc58d701259120d361b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fc902de5b2274103ab5e76f967503afb (127.0.105.2:46339): Illegal state: Tablet not RUNNING: STOPPED
I20260503 19:53:32.964428 1549 ts_tablet_manager.cc:1929] T 65f0f8cedacf4024ae7bbb741a25c2f8 P fc902de5b2274103ab5e76f967503afb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260503 19:53:32.964672 1549 log.cc:1199] T 65f0f8cedacf4024ae7bbb741a25c2f8 P fc902de5b2274103ab5e76f967503afb: Deleting WAL directory at /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-1/wal/wals/65f0f8cedacf4024ae7bbb741a25c2f8
I20260503 19:53:32.966969 804 catalog_manager.cc:5671] T df35135905b04fea927000d21e9994e6 P b68b1efd1cc44e15b8c69e730229dbdd reported cstate change: term changed from 0 to 1, leader changed from <none> to b68b1efd1cc44e15b8c69e730229dbdd (127.0.105.3). New cstate: current_term: 1 leader_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } health_report { overall_health: HEALTHY } } }
@ 0x7f591075edcd google::LogMessage::Fail() at ??:0
@ 0x7f5910762b93 google::LogMessage::SendToLog() at ??:0
@ 0x7f591075e7cc google::LogMessage::Flush() at ??:0
@ 0x7f591075ea11 google::LogMessage::~LogMessage() at ??:0
I20260503 19:53:32.982196 1487 tablet_bootstrap.cc:492] T 75dd4e31d81c4d5da2224e29e4f51e8d P fc902de5b2274103ab5e76f967503afb: Bootstrap starting.
I20260503 19:53:32.984123 1487 tablet_bootstrap.cc:654] T 75dd4e31d81c4d5da2224e29e4f51e8d P fc902de5b2274103ab5e76f967503afb: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:32.985863 1549 ts_tablet_manager.cc:1950] T 65f0f8cedacf4024ae7bbb741a25c2f8 P fc902de5b2274103ab5e76f967503afb: Deleting consensus metadata
I20260503 19:53:32.987041 797 catalog_manager.cc:5002] TS fc902de5b2274103ab5e76f967503afb (127.0.105.2:46339): tablet 65f0f8cedacf4024ae7bbb741a25c2f8 (table default.table_9c30ca5293344be3964460b578f9d6df [id=8d8f5993b50b4a59aaa004b29cf7c6dd]) successfully deleted
@ 0x7f5927237574 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260503 19:53:32.999146 1047 tablet_service.cc:1511] Processing CreateTablet for tablet c4ef894c0fdb4414a6799627641a8857 (DEFAULT_TABLE table=default.table_9c30ca5293344be3964460b578f9d6df [id=8d8f5993b50b4a59aaa004b29cf7c6dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260503 19:53:32.999728 1047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4ef894c0fdb4414a6799627641a8857. 1 dirs total, 0 dirs full, 0 dirs failed
I20260503 19:53:33.001332 1545 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 1 FOLLOWER]: Advancing to term 2
I20260503 19:53:33.008790 1327 tablet_service.cc:1511] Processing CreateTablet for tablet c4ef894c0fdb4414a6799627641a8857 (DEFAULT_TABLE table=default.table_9c30ca5293344be3964460b578f9d6df [id=8d8f5993b50b4a59aaa004b29cf7c6dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260503 19:53:33.009541 1327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4ef894c0fdb4414a6799627641a8857. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7f5927235e21 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260503 19:53:33.016842 1190 tablet_service.cc:1511] Processing CreateTablet for tablet c4ef894c0fdb4414a6799627641a8857 (DEFAULT_TABLE table=default.table_9c30ca5293344be3964460b578f9d6df [id=8d8f5993b50b4a59aaa004b29cf7c6dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260503 19:53:33.017520 1190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4ef894c0fdb4414a6799627641a8857. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7f592723da9c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f592723d919 std::_Function_handler<>::_M_invoke() at ??:0
@ 0xaa4cad std::function<>::operator()() at ??:?
I20260503 19:53:33.040060 1487 tablet_bootstrap.cc:492] T 75dd4e31d81c4d5da2224e29e4f51e8d P fc902de5b2274103ab5e76f967503afb: No bootstrap required, opened a new log
I20260503 19:53:33.040247 1487 ts_tablet_manager.cc:1403] T 75dd4e31d81c4d5da2224e29e4f51e8d P fc902de5b2274103ab5e76f967503afb: Time spent bootstrapping tablet: real 0.058s user 0.002s sys 0.001s
I20260503 19:53:33.040849 1629 tablet_replica.cc:333] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b8594a7337c54dc58d701259120d361b: stopping tablet replica
I20260503 19:53:33.041046 1629 raft_consensus.cc:2243] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b8594a7337c54dc58d701259120d361b [term 1 LEADER]: Raft consensus shutting down.
I20260503 19:53:33.041172 1487 raft_consensus.cc:359] T 75dd4e31d81c4d5da2224e29e4f51e8d P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } } peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } }
I20260503 19:53:33.041348 1487 raft_consensus.cc:385] T 75dd4e31d81c4d5da2224e29e4f51e8d P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:33.041429 1487 raft_consensus.cc:740] T 75dd4e31d81c4d5da2224e29e4f51e8d P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc902de5b2274103ab5e76f967503afb, State: Initialized, Role: FOLLOWER
I20260503 19:53:33.041579 1629 pending_rounds.cc:70] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b8594a7337c54dc58d701259120d361b: Trying to abort 1 pending ops.
I20260503 19:53:33.041715 1487 consensus_queue.cc:260] T 75dd4e31d81c4d5da2224e29e4f51e8d P fc902de5b2274103ab5e76f967503afb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } } peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } }
I20260503 19:53:33.041762 1629 pending_rounds.cc:77] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b8594a7337c54dc58d701259120d361b: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7282024501059211264 op_type: NO_OP noop_request { }
I20260503 19:53:33.042711 1487 ts_tablet_manager.cc:1434] T 75dd4e31d81c4d5da2224e29e4f51e8d P fc902de5b2274103ab5e76f967503afb: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260503 19:53:33.043010 1487 tablet_bootstrap.cc:492] T 6726412cbfd947d2849ac9774d4e86f4 P fc902de5b2274103ab5e76f967503afb: Bootstrap starting.
I20260503 19:53:33.043572 1629 raft_consensus.cc:2889] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b8594a7337c54dc58d701259120d361b [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260503 19:53:33.043707 1629 raft_consensus.cc:2272] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b8594a7337c54dc58d701259120d361b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260503 19:53:33.044273 1632 tablet_replica.cc:333] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b68b1efd1cc44e15b8c69e730229dbdd: stopping tablet replica
I20260503 19:53:33.044466 1632 raft_consensus.cc:2243] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b68b1efd1cc44e15b8c69e730229dbdd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260503 19:53:33.044661 1632 raft_consensus.cc:2272] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b68b1efd1cc44e15b8c69e730229dbdd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260503 19:53:33.044844 1629 ts_tablet_manager.cc:1916] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b8594a7337c54dc58d701259120d361b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260503 19:53:33.045195 1632 ts_tablet_manager.cc:1916] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b68b1efd1cc44e15b8c69e730229dbdd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260503 19:53:33.047672 1488 tablet_bootstrap.cc:492] T 6726412cbfd947d2849ac9774d4e86f4 P b8594a7337c54dc58d701259120d361b: Bootstrap starting.
I20260503 19:53:33.049413 1487 tablet_bootstrap.cc:654] T 6726412cbfd947d2849ac9774d4e86f4 P fc902de5b2274103ab5e76f967503afb: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:33.049861 1488 tablet_bootstrap.cc:654] T 6726412cbfd947d2849ac9774d4e86f4 P b8594a7337c54dc58d701259120d361b: Neither blocks nor log segments found. Creating new log.
@ 0x7f59134f7529 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f5916ae56db start_thread at ??:0
@ 0x7f590f60f61f clone at ??:0
I20260503 19:53:33.057696 1488 tablet_bootstrap.cc:492] T 6726412cbfd947d2849ac9774d4e86f4 P b8594a7337c54dc58d701259120d361b: No bootstrap required, opened a new log
I20260503 19:53:33.057860 1488 ts_tablet_manager.cc:1403] T 6726412cbfd947d2849ac9774d4e86f4 P b8594a7337c54dc58d701259120d361b: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260503 19:53:33.058640 1488 raft_consensus.cc:359] T 6726412cbfd947d2849ac9774d4e86f4 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:33.058866 1488 raft_consensus.cc:385] T 6726412cbfd947d2849ac9774d4e86f4 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:33.058961 1488 raft_consensus.cc:740] T 6726412cbfd947d2849ac9774d4e86f4 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8594a7337c54dc58d701259120d361b, State: Initialized, Role: FOLLOWER
I20260503 19:53:33.059247 1488 consensus_queue.cc:260] T 6726412cbfd947d2849ac9774d4e86f4 P b8594a7337c54dc58d701259120d361b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:33.063791 1629 ts_tablet_manager.cc:1929] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b8594a7337c54dc58d701259120d361b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260503 19:53:33.063939 1629 log.cc:1199] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b8594a7337c54dc58d701259120d361b: Deleting WAL directory at /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-0/wal/wals/65f0f8cedacf4024ae7bbb741a25c2f8
I20260503 19:53:33.064028 1488 ts_tablet_manager.cc:1434] T 6726412cbfd947d2849ac9774d4e86f4 P b8594a7337c54dc58d701259120d361b: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260503 19:53:33.064414 1488 tablet_bootstrap.cc:492] T 75dd4e31d81c4d5da2224e29e4f51e8d P b8594a7337c54dc58d701259120d361b: Bootstrap starting.
I20260503 19:53:33.064462 1629 ts_tablet_manager.cc:1950] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b8594a7337c54dc58d701259120d361b: Deleting consensus metadata
I20260503 19:53:33.068783 1488 tablet_bootstrap.cc:654] T 75dd4e31d81c4d5da2224e29e4f51e8d P b8594a7337c54dc58d701259120d361b: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:33.080061 1564 raft_consensus.cc:493] T a6cc88aae841477d990ed433c705a803 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:53:33.080217 1564 raft_consensus.cc:515] T a6cc88aae841477d990ed433c705a803 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:33.080961 1564 leader_election.cc:290] T a6cc88aae841477d990ed433c705a803 P fc902de5b2274103ab5e76f967503afb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8594a7337c54dc58d701259120d361b (127.0.105.1:40289), b68b1efd1cc44e15b8c69e730229dbdd (127.0.105.3:41575)
I20260503 19:53:33.081749 1349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6cc88aae841477d990ed433c705a803" candidate_uuid: "fc902de5b2274103ab5e76f967503afb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" is_pre_election: true
I20260503 19:53:33.082043 1349 raft_consensus.cc:2468] T a6cc88aae841477d990ed433c705a803 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc902de5b2274103ab5e76f967503afb in term 0.
I20260503 19:53:33.083228 1069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6cc88aae841477d990ed433c705a803" candidate_uuid: "fc902de5b2274103ab5e76f967503afb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8594a7337c54dc58d701259120d361b" is_pre_election: true
I20260503 19:53:33.083323 1147 leader_election.cc:304] T a6cc88aae841477d990ed433c705a803 P fc902de5b2274103ab5e76f967503afb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b68b1efd1cc44e15b8c69e730229dbdd, fc902de5b2274103ab5e76f967503afb; no voters:
I20260503 19:53:33.083501 1069 raft_consensus.cc:2468] T a6cc88aae841477d990ed433c705a803 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc902de5b2274103ab5e76f967503afb in term 0.
I20260503 19:53:33.083590 1564 raft_consensus.cc:2804] T a6cc88aae841477d990ed433c705a803 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260503 19:53:33.083686 1564 raft_consensus.cc:493] T a6cc88aae841477d990ed433c705a803 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260503 19:53:33.083760 1564 raft_consensus.cc:3060] T a6cc88aae841477d990ed433c705a803 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Advancing to term 1
I20260503 19:53:33.086616 1564 raft_consensus.cc:515] T a6cc88aae841477d990ed433c705a803 P fc902de5b2274103ab5e76f967503afb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:33.088352 1069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6cc88aae841477d990ed433c705a803" candidate_uuid: "fc902de5b2274103ab5e76f967503afb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8594a7337c54dc58d701259120d361b"
I20260503 19:53:33.088546 1069 raft_consensus.cc:3060] T a6cc88aae841477d990ed433c705a803 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Advancing to term 1
I20260503 19:53:33.091606 1069 raft_consensus.cc:2468] T a6cc88aae841477d990ed433c705a803 P b8594a7337c54dc58d701259120d361b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc902de5b2274103ab5e76f967503afb in term 1.
I20260503 19:53:33.092042 1146 leader_election.cc:304] T a6cc88aae841477d990ed433c705a803 P fc902de5b2274103ab5e76f967503afb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8594a7337c54dc58d701259120d361b, fc902de5b2274103ab5e76f967503afb; no voters:
I20260503 19:53:33.101890 1349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6cc88aae841477d990ed433c705a803" candidate_uuid: "fc902de5b2274103ab5e76f967503afb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b68b1efd1cc44e15b8c69e730229dbdd"
I20260503 19:53:33.102648 1349 raft_consensus.cc:3060] T a6cc88aae841477d990ed433c705a803 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Advancing to term 1
I20260503 19:53:33.107306 1349 raft_consensus.cc:2468] T a6cc88aae841477d990ed433c705a803 P b68b1efd1cc44e15b8c69e730229dbdd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc902de5b2274103ab5e76f967503afb in term 1.
I20260503 19:53:33.108521 1564 leader_election.cc:290] T a6cc88aae841477d990ed433c705a803 P fc902de5b2274103ab5e76f967503afb [CANDIDATE]: Term 1 election: Requested vote from peers b8594a7337c54dc58d701259120d361b (127.0.105.1:40289), b68b1efd1cc44e15b8c69e730229dbdd (127.0.105.3:41575)
I20260503 19:53:33.108790 1564 raft_consensus.cc:493] T f391e07ff0e24bc5b3cf7e81342161fe P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:53:33.108969 1564 raft_consensus.cc:515] T f391e07ff0e24bc5b3cf7e81342161fe P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:33.110080 1564 leader_election.cc:290] T f391e07ff0e24bc5b3cf7e81342161fe P fc902de5b2274103ab5e76f967503afb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8594a7337c54dc58d701259120d361b (127.0.105.1:40289), b68b1efd1cc44e15b8c69e730229dbdd (127.0.105.3:41575)
I20260503 19:53:33.110239 1564 raft_consensus.cc:2804] T a6cc88aae841477d990ed433c705a803 P fc902de5b2274103ab5e76f967503afb [term 1 FOLLOWER]: Leader election won for term 1
I20260503 19:53:33.110363 1564 raft_consensus.cc:697] T a6cc88aae841477d990ed433c705a803 P fc902de5b2274103ab5e76f967503afb [term 1 LEADER]: Becoming Leader. State: Replica: fc902de5b2274103ab5e76f967503afb, State: Running, Role: LEADER
I20260503 19:53:33.110710 1564 consensus_queue.cc:237] T a6cc88aae841477d990ed433c705a803 P fc902de5b2274103ab5e76f967503afb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:33.111940 1349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f391e07ff0e24bc5b3cf7e81342161fe" candidate_uuid: "fc902de5b2274103ab5e76f967503afb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" is_pre_election: true
I20260503 19:53:33.112143 1349 raft_consensus.cc:2468] T f391e07ff0e24bc5b3cf7e81342161fe P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc902de5b2274103ab5e76f967503afb in term 0.
I20260503 19:53:33.112601 1147 leader_election.cc:304] T f391e07ff0e24bc5b3cf7e81342161fe P fc902de5b2274103ab5e76f967503afb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b68b1efd1cc44e15b8c69e730229dbdd, fc902de5b2274103ab5e76f967503afb; no voters:
I20260503 19:53:33.121996 1564 raft_consensus.cc:2804] T f391e07ff0e24bc5b3cf7e81342161fe P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260503 19:53:33.122138 1564 raft_consensus.cc:493] T f391e07ff0e24bc5b3cf7e81342161fe P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260503 19:53:33.122248 1564 raft_consensus.cc:3060] T f391e07ff0e24bc5b3cf7e81342161fe P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Advancing to term 1
I20260503 19:53:33.127828 1597 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ba485a545f44fdd8217c3f86c79d0ca. Latest consensus state: current_term: 2 leader_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:33.128082 1597 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260503 19:53:33.132175 1487 tablet_bootstrap.cc:492] T 6726412cbfd947d2849ac9774d4e86f4 P fc902de5b2274103ab5e76f967503afb: No bootstrap required, opened a new log
I20260503 19:53:33.132377 1487 ts_tablet_manager.cc:1403] T 6726412cbfd947d2849ac9774d4e86f4 P fc902de5b2274103ab5e76f967503afb: Time spent bootstrapping tablet: real 0.089s user 0.007s sys 0.006s
I20260503 19:53:33.133917 1487 raft_consensus.cc:359] T 6726412cbfd947d2849ac9774d4e86f4 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:33.134123 1487 raft_consensus.cc:385] T 6726412cbfd947d2849ac9774d4e86f4 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:33.134187 1487 raft_consensus.cc:740] T 6726412cbfd947d2849ac9774d4e86f4 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc902de5b2274103ab5e76f967503afb, State: Initialized, Role: FOLLOWER
I20260503 19:53:33.134431 1487 consensus_queue.cc:260] T 6726412cbfd947d2849ac9774d4e86f4 P fc902de5b2274103ab5e76f967503afb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:33.134910 1487 ts_tablet_manager.cc:1434] T 6726412cbfd947d2849ac9774d4e86f4 P fc902de5b2274103ab5e76f967503afb: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260503 19:53:33.135265 1487 tablet_bootstrap.cc:492] T c4ef894c0fdb4414a6799627641a8857 P fc902de5b2274103ab5e76f967503afb: Bootstrap starting.
I20260503 19:53:33.137065 1652 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:33.137377 1652 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260503 19:53:33.138198 1488 tablet_bootstrap.cc:492] T 75dd4e31d81c4d5da2224e29e4f51e8d P b8594a7337c54dc58d701259120d361b: No bootstrap required, opened a new log
I20260503 19:53:33.138365 1488 ts_tablet_manager.cc:1403] T 75dd4e31d81c4d5da2224e29e4f51e8d P b8594a7337c54dc58d701259120d361b: Time spent bootstrapping tablet: real 0.074s user 0.000s sys 0.004s
I20260503 19:53:33.139113 1488 raft_consensus.cc:359] T 75dd4e31d81c4d5da2224e29e4f51e8d P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } } peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } }
I20260503 19:53:33.139333 1488 raft_consensus.cc:385] T 75dd4e31d81c4d5da2224e29e4f51e8d P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:33.139402 1488 raft_consensus.cc:740] T 75dd4e31d81c4d5da2224e29e4f51e8d P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8594a7337c54dc58d701259120d361b, State: Initialized, Role: FOLLOWER
I20260503 19:53:33.141451 1487 tablet_bootstrap.cc:654] T c4ef894c0fdb4414a6799627641a8857 P fc902de5b2274103ab5e76f967503afb: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:33.145970 1486 tablet_bootstrap.cc:492] T 75dd4e31d81c4d5da2224e29e4f51e8d P b68b1efd1cc44e15b8c69e730229dbdd: No bootstrap required, opened a new log
I20260503 19:53:33.146123 1486 ts_tablet_manager.cc:1403] T 75dd4e31d81c4d5da2224e29e4f51e8d P b68b1efd1cc44e15b8c69e730229dbdd: Time spent bootstrapping tablet: real 0.206s user 0.048s sys 0.042s
I20260503 19:53:33.146929 1486 raft_consensus.cc:359] T 75dd4e31d81c4d5da2224e29e4f51e8d P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } } peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } }
I20260503 19:53:33.147116 1486 raft_consensus.cc:385] T 75dd4e31d81c4d5da2224e29e4f51e8d P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:33.147195 1486 raft_consensus.cc:740] T 75dd4e31d81c4d5da2224e29e4f51e8d P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b68b1efd1cc44e15b8c69e730229dbdd, State: Initialized, Role: FOLLOWER
I20260503 19:53:33.147447 1486 consensus_queue.cc:260] T 75dd4e31d81c4d5da2224e29e4f51e8d P b68b1efd1cc44e15b8c69e730229dbdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } } peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } }
I20260503 19:53:33.149278 1488 consensus_queue.cc:260] T 75dd4e31d81c4d5da2224e29e4f51e8d P b8594a7337c54dc58d701259120d361b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } } peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } }
I20260503 19:53:33.152221 1488 ts_tablet_manager.cc:1434] T 75dd4e31d81c4d5da2224e29e4f51e8d P b8594a7337c54dc58d701259120d361b: Time spent starting tablet: real 0.014s user 0.000s sys 0.002s
I20260503 19:53:33.152509 1486 ts_tablet_manager.cc:1434] T 75dd4e31d81c4d5da2224e29e4f51e8d P b68b1efd1cc44e15b8c69e730229dbdd: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260503 19:53:33.152702 1488 tablet_bootstrap.cc:492] T c4ef894c0fdb4414a6799627641a8857 P b8594a7337c54dc58d701259120d361b: Bootstrap starting.
W20260503 19:53:33.154991 1146 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.1:40289, user_credentials={real_user=slave}} blocked reactor thread for 62922.5us
I20260503 19:53:33.155635 1069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f391e07ff0e24bc5b3cf7e81342161fe" candidate_uuid: "fc902de5b2274103ab5e76f967503afb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8594a7337c54dc58d701259120d361b" is_pre_election: true
I20260503 19:53:33.155845 1069 raft_consensus.cc:2468] T f391e07ff0e24bc5b3cf7e81342161fe P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc902de5b2274103ab5e76f967503afb in term 0.
I20260503 19:53:33.157779 1564 raft_consensus.cc:515] T f391e07ff0e24bc5b3cf7e81342161fe P fc902de5b2274103ab5e76f967503afb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:33.157785 1486 tablet_bootstrap.cc:492] T c4ef894c0fdb4414a6799627641a8857 P b68b1efd1cc44e15b8c69e730229dbdd: Bootstrap starting.
I20260503 19:53:33.159174 1564 leader_election.cc:290] T f391e07ff0e24bc5b3cf7e81342161fe P fc902de5b2274103ab5e76f967503afb [CANDIDATE]: Term 1 election: Requested vote from peers b8594a7337c54dc58d701259120d361b (127.0.105.1:40289), b68b1efd1cc44e15b8c69e730229dbdd (127.0.105.3:41575)
I20260503 19:53:33.159487 1488 tablet_bootstrap.cc:654] T c4ef894c0fdb4414a6799627641a8857 P b8594a7337c54dc58d701259120d361b: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:33.159433 1349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f391e07ff0e24bc5b3cf7e81342161fe" candidate_uuid: "fc902de5b2274103ab5e76f967503afb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b68b1efd1cc44e15b8c69e730229dbdd"
I20260503 19:53:33.159660 1349 raft_consensus.cc:3060] T f391e07ff0e24bc5b3cf7e81342161fe P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Advancing to term 1
I20260503 19:53:33.160604 1632 ts_tablet_manager.cc:1929] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b68b1efd1cc44e15b8c69e730229dbdd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260503 19:53:33.160727 1632 log.cc:1199] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b68b1efd1cc44e15b8c69e730229dbdd: Deleting WAL directory at /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-2/wal/wals/65f0f8cedacf4024ae7bbb741a25c2f8
I20260503 19:53:33.161159 1632 ts_tablet_manager.cc:1950] T 65f0f8cedacf4024ae7bbb741a25c2f8 P b68b1efd1cc44e15b8c69e730229dbdd: Deleting consensus metadata
I20260503 19:53:33.162691 1349 raft_consensus.cc:2468] T f391e07ff0e24bc5b3cf7e81342161fe P b68b1efd1cc44e15b8c69e730229dbdd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc902de5b2274103ab5e76f967503afb in term 1.
I20260503 19:53:33.163107 1147 leader_election.cc:304] T f391e07ff0e24bc5b3cf7e81342161fe P fc902de5b2274103ab5e76f967503afb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b68b1efd1cc44e15b8c69e730229dbdd, fc902de5b2274103ab5e76f967503afb; no voters:
I20260503 19:53:33.163671 1069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f391e07ff0e24bc5b3cf7e81342161fe" candidate_uuid: "fc902de5b2274103ab5e76f967503afb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8594a7337c54dc58d701259120d361b"
I20260503 19:53:33.163877 1069 raft_consensus.cc:3060] T f391e07ff0e24bc5b3cf7e81342161fe P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Advancing to term 1
I20260503 19:53:33.164070 1564 raft_consensus.cc:2804] T f391e07ff0e24bc5b3cf7e81342161fe P fc902de5b2274103ab5e76f967503afb [term 1 FOLLOWER]: Leader election won for term 1
I20260503 19:53:33.164181 1488 tablet_bootstrap.cc:492] T c4ef894c0fdb4414a6799627641a8857 P b8594a7337c54dc58d701259120d361b: No bootstrap required, opened a new log
I20260503 19:53:33.164239 1564 raft_consensus.cc:697] T f391e07ff0e24bc5b3cf7e81342161fe P fc902de5b2274103ab5e76f967503afb [term 1 LEADER]: Becoming Leader. State: Replica: fc902de5b2274103ab5e76f967503afb, State: Running, Role: LEADER
I20260503 19:53:33.164317 1488 ts_tablet_manager.cc:1403] T c4ef894c0fdb4414a6799627641a8857 P b8594a7337c54dc58d701259120d361b: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.000s
I20260503 19:53:33.164515 1564 consensus_queue.cc:237] T f391e07ff0e24bc5b3cf7e81342161fe P fc902de5b2274103ab5e76f967503afb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:33.165160 1488 raft_consensus.cc:359] T c4ef894c0fdb4414a6799627641a8857 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:33.165343 1488 raft_consensus.cc:385] T c4ef894c0fdb4414a6799627641a8857 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:33.165432 1488 raft_consensus.cc:740] T c4ef894c0fdb4414a6799627641a8857 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8594a7337c54dc58d701259120d361b, State: Initialized, Role: FOLLOWER
I20260503 19:53:33.165648 1488 consensus_queue.cc:260] T c4ef894c0fdb4414a6799627641a8857 P b8594a7337c54dc58d701259120d361b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:33.166749 1069 raft_consensus.cc:2468] T f391e07ff0e24bc5b3cf7e81342161fe P b8594a7337c54dc58d701259120d361b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc902de5b2274103ab5e76f967503afb in term 1.
I20260503 19:53:33.167455 1486 tablet_bootstrap.cc:654] T c4ef894c0fdb4414a6799627641a8857 P b68b1efd1cc44e15b8c69e730229dbdd: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:33.174088 1487 tablet_bootstrap.cc:492] T c4ef894c0fdb4414a6799627641a8857 P fc902de5b2274103ab5e76f967503afb: No bootstrap required, opened a new log
I20260503 19:53:33.174254 1487 ts_tablet_manager.cc:1403] T c4ef894c0fdb4414a6799627641a8857 P fc902de5b2274103ab5e76f967503afb: Time spent bootstrapping tablet: real 0.039s user 0.005s sys 0.000s
I20260503 19:53:33.175104 1487 raft_consensus.cc:359] T c4ef894c0fdb4414a6799627641a8857 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:33.175323 1487 raft_consensus.cc:385] T c4ef894c0fdb4414a6799627641a8857 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:33.175415 1487 raft_consensus.cc:740] T c4ef894c0fdb4414a6799627641a8857 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc902de5b2274103ab5e76f967503afb, State: Initialized, Role: FOLLOWER
I20260503 19:53:33.175639 1487 consensus_queue.cc:260] T c4ef894c0fdb4414a6799627641a8857 P fc902de5b2274103ab5e76f967503afb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:33.176281 1487 ts_tablet_manager.cc:1434] T c4ef894c0fdb4414a6799627641a8857 P fc902de5b2274103ab5e76f967503afb: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260503 19:53:33.176983 1488 ts_tablet_manager.cc:1434] T c4ef894c0fdb4414a6799627641a8857 P b8594a7337c54dc58d701259120d361b: Time spent starting tablet: real 0.013s user 0.002s sys 0.000s
I20260503 19:53:33.202973 1486 tablet_bootstrap.cc:492] T c4ef894c0fdb4414a6799627641a8857 P b68b1efd1cc44e15b8c69e730229dbdd: No bootstrap required, opened a new log
I20260503 19:53:33.203150 1486 ts_tablet_manager.cc:1403] T c4ef894c0fdb4414a6799627641a8857 P b68b1efd1cc44e15b8c69e730229dbdd: Time spent bootstrapping tablet: real 0.045s user 0.002s sys 0.006s
I20260503 19:53:33.204058 1486 raft_consensus.cc:359] T c4ef894c0fdb4414a6799627641a8857 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:33.204321 1486 raft_consensus.cc:385] T c4ef894c0fdb4414a6799627641a8857 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:33.204455 1486 raft_consensus.cc:740] T c4ef894c0fdb4414a6799627641a8857 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b68b1efd1cc44e15b8c69e730229dbdd, State: Initialized, Role: FOLLOWER
I20260503 19:53:33.204787 1486 consensus_queue.cc:260] T c4ef894c0fdb4414a6799627641a8857 P b68b1efd1cc44e15b8c69e730229dbdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:33.207309 1486 ts_tablet_manager.cc:1434] T c4ef894c0fdb4414a6799627641a8857 P b68b1efd1cc44e15b8c69e730229dbdd: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260503 19:53:33.207651 1486 tablet_bootstrap.cc:492] T 6726412cbfd947d2849ac9774d4e86f4 P b68b1efd1cc44e15b8c69e730229dbdd: Bootstrap starting.
I20260503 19:53:33.209721 1486 tablet_bootstrap.cc:654] T 6726412cbfd947d2849ac9774d4e86f4 P b68b1efd1cc44e15b8c69e730229dbdd: Neither blocks nor log segments found. Creating new log.
I20260503 19:53:33.212502 1640 mvcc.cc:204] Tried to move back new op lower bound from 7282024500908044288 to 7282024492191686656. Current Snapshot: MvccSnapshot[applied={T|T < 7282024500908044288}]
I20260503 19:53:33.230988 1486 tablet_bootstrap.cc:492] T 6726412cbfd947d2849ac9774d4e86f4 P b68b1efd1cc44e15b8c69e730229dbdd: No bootstrap required, opened a new log
I20260503 19:53:33.231158 1486 ts_tablet_manager.cc:1403] T 6726412cbfd947d2849ac9774d4e86f4 P b68b1efd1cc44e15b8c69e730229dbdd: Time spent bootstrapping tablet: real 0.024s user 0.004s sys 0.004s
I20260503 19:53:33.231978 1486 raft_consensus.cc:359] T 6726412cbfd947d2849ac9774d4e86f4 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:33.232189 1486 raft_consensus.cc:385] T 6726412cbfd947d2849ac9774d4e86f4 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260503 19:53:33.232276 1486 raft_consensus.cc:740] T 6726412cbfd947d2849ac9774d4e86f4 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b68b1efd1cc44e15b8c69e730229dbdd, State: Initialized, Role: FOLLOWER
I20260503 19:53:33.232511 1486 consensus_queue.cc:260] T 6726412cbfd947d2849ac9774d4e86f4 P b68b1efd1cc44e15b8c69e730229dbdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:33.234170 1486 ts_tablet_manager.cc:1434] T 6726412cbfd947d2849ac9774d4e86f4 P b68b1efd1cc44e15b8c69e730229dbdd: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260503 19:53:33.290712 1563 consensus_queue.cc:1048] T ed5861fcf94c47199eb1a81d4702f19b P b68b1efd1cc44e15b8c69e730229dbdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260503 19:53:33.299080 1620 consensus_queue.cc:1048] T ed5861fcf94c47199eb1a81d4702f19b P b68b1efd1cc44e15b8c69e730229dbdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260503 19:53:33.318333 1589 consensus_queue.cc:1048] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b8594a7337c54dc58d701259120d361b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260503 19:53:33.324152 1501 consensus_queue.cc:1048] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b8594a7337c54dc58d701259120d361b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260503 19:53:33.350672 1564 raft_consensus.cc:493] T c4ef894c0fdb4414a6799627641a8857 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:53:33.350879 1564 raft_consensus.cc:515] T c4ef894c0fdb4414a6799627641a8857 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:33.351619 1564 leader_election.cc:290] T c4ef894c0fdb4414a6799627641a8857 P fc902de5b2274103ab5e76f967503afb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8594a7337c54dc58d701259120d361b (127.0.105.1:40289), b68b1efd1cc44e15b8c69e730229dbdd (127.0.105.3:41575)
I20260503 19:53:33.352188 1069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4ef894c0fdb4414a6799627641a8857" candidate_uuid: "fc902de5b2274103ab5e76f967503afb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8594a7337c54dc58d701259120d361b" is_pre_election: true
I20260503 19:53:33.352407 1349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4ef894c0fdb4414a6799627641a8857" candidate_uuid: "fc902de5b2274103ab5e76f967503afb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" is_pre_election: true
I20260503 19:53:33.352609 1349 raft_consensus.cc:2468] T c4ef894c0fdb4414a6799627641a8857 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc902de5b2274103ab5e76f967503afb in term 0.
I20260503 19:53:33.352864 1069 raft_consensus.cc:2468] T c4ef894c0fdb4414a6799627641a8857 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc902de5b2274103ab5e76f967503afb in term 0.
I20260503 19:53:33.353139 1147 leader_election.cc:304] T c4ef894c0fdb4414a6799627641a8857 P fc902de5b2274103ab5e76f967503afb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b68b1efd1cc44e15b8c69e730229dbdd, fc902de5b2274103ab5e76f967503afb; no voters:
I20260503 19:53:33.353440 1564 raft_consensus.cc:2804] T c4ef894c0fdb4414a6799627641a8857 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260503 19:53:33.353556 1564 raft_consensus.cc:493] T c4ef894c0fdb4414a6799627641a8857 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260503 19:53:33.353648 1564 raft_consensus.cc:3060] T c4ef894c0fdb4414a6799627641a8857 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Advancing to term 1
I20260503 19:53:33.356532 1564 raft_consensus.cc:515] T c4ef894c0fdb4414a6799627641a8857 P fc902de5b2274103ab5e76f967503afb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:33.357640 1564 leader_election.cc:290] T c4ef894c0fdb4414a6799627641a8857 P fc902de5b2274103ab5e76f967503afb [CANDIDATE]: Term 1 election: Requested vote from peers b8594a7337c54dc58d701259120d361b (127.0.105.1:40289), b68b1efd1cc44e15b8c69e730229dbdd (127.0.105.3:41575)
I20260503 19:53:33.357759 1069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4ef894c0fdb4414a6799627641a8857" candidate_uuid: "fc902de5b2274103ab5e76f967503afb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8594a7337c54dc58d701259120d361b"
I20260503 19:53:33.357949 1069 raft_consensus.cc:3060] T c4ef894c0fdb4414a6799627641a8857 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Advancing to term 1
I20260503 19:53:33.357954 1349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4ef894c0fdb4414a6799627641a8857" candidate_uuid: "fc902de5b2274103ab5e76f967503afb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b68b1efd1cc44e15b8c69e730229dbdd"
I20260503 19:53:33.358156 1349 raft_consensus.cc:3060] T c4ef894c0fdb4414a6799627641a8857 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Advancing to term 1
I20260503 19:53:33.361028 1349 raft_consensus.cc:2468] T c4ef894c0fdb4414a6799627641a8857 P b68b1efd1cc44e15b8c69e730229dbdd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc902de5b2274103ab5e76f967503afb in term 1.
I20260503 19:53:33.361315 1069 raft_consensus.cc:2468] T c4ef894c0fdb4414a6799627641a8857 P b8594a7337c54dc58d701259120d361b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc902de5b2274103ab5e76f967503afb in term 1.
I20260503 19:53:33.361626 1147 leader_election.cc:304] T c4ef894c0fdb4414a6799627641a8857 P fc902de5b2274103ab5e76f967503afb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b68b1efd1cc44e15b8c69e730229dbdd, fc902de5b2274103ab5e76f967503afb; no voters:
I20260503 19:53:33.362102 1564 raft_consensus.cc:2804] T c4ef894c0fdb4414a6799627641a8857 P fc902de5b2274103ab5e76f967503afb [term 1 FOLLOWER]: Leader election won for term 1
I20260503 19:53:33.362253 1564 raft_consensus.cc:697] T c4ef894c0fdb4414a6799627641a8857 P fc902de5b2274103ab5e76f967503afb [term 1 LEADER]: Becoming Leader. State: Replica: fc902de5b2274103ab5e76f967503afb, State: Running, Role: LEADER
I20260503 19:53:33.362552 1564 consensus_queue.cc:237] T c4ef894c0fdb4414a6799627641a8857 P fc902de5b2274103ab5e76f967503afb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:33.422411 1564 raft_consensus.cc:493] T 6726412cbfd947d2849ac9774d4e86f4 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:53:33.422663 1564 raft_consensus.cc:515] T 6726412cbfd947d2849ac9774d4e86f4 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:33.423467 1564 leader_election.cc:290] T 6726412cbfd947d2849ac9774d4e86f4 P fc902de5b2274103ab5e76f967503afb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8594a7337c54dc58d701259120d361b (127.0.105.1:40289), b68b1efd1cc44e15b8c69e730229dbdd (127.0.105.3:41575)
I20260503 19:53:33.424108 1069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6726412cbfd947d2849ac9774d4e86f4" candidate_uuid: "fc902de5b2274103ab5e76f967503afb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8594a7337c54dc58d701259120d361b" is_pre_election: true
I20260503 19:53:33.424213 1349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6726412cbfd947d2849ac9774d4e86f4" candidate_uuid: "fc902de5b2274103ab5e76f967503afb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" is_pre_election: true
I20260503 19:53:33.424361 1069 raft_consensus.cc:2468] T 6726412cbfd947d2849ac9774d4e86f4 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc902de5b2274103ab5e76f967503afb in term 0.
I20260503 19:53:33.424443 1349 raft_consensus.cc:2468] T 6726412cbfd947d2849ac9774d4e86f4 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc902de5b2274103ab5e76f967503afb in term 0.
I20260503 19:53:33.424944 1147 leader_election.cc:304] T 6726412cbfd947d2849ac9774d4e86f4 P fc902de5b2274103ab5e76f967503afb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b68b1efd1cc44e15b8c69e730229dbdd, fc902de5b2274103ab5e76f967503afb; no voters:
I20260503 19:53:33.425407 1564 raft_consensus.cc:2804] T 6726412cbfd947d2849ac9774d4e86f4 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260503 19:53:33.425506 1564 raft_consensus.cc:493] T 6726412cbfd947d2849ac9774d4e86f4 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260503 19:53:33.425616 1564 raft_consensus.cc:3060] T 6726412cbfd947d2849ac9774d4e86f4 P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Advancing to term 1
I20260503 19:53:33.428517 1564 raft_consensus.cc:515] T 6726412cbfd947d2849ac9774d4e86f4 P fc902de5b2274103ab5e76f967503afb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
I20260503 19:53:33.429811 1069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6726412cbfd947d2849ac9774d4e86f4" candidate_uuid: "fc902de5b2274103ab5e76f967503afb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8594a7337c54dc58d701259120d361b"
I20260503 19:53:33.429786 1349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6726412cbfd947d2849ac9774d4e86f4" candidate_uuid: "fc902de5b2274103ab5e76f967503afb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b68b1efd1cc44e15b8c69e730229dbdd"
I20260503 19:53:33.430037 1349 raft_consensus.cc:3060] T 6726412cbfd947d2849ac9774d4e86f4 P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Advancing to term 1
I20260503 19:53:33.430042 1069 raft_consensus.cc:3060] T 6726412cbfd947d2849ac9774d4e86f4 P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Advancing to term 1
W20260503 19:53:33.430234 1519 connection.cc:570] server connection from 127.0.0.1:45758 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260503 19:53:33.433377 1404 connection.cc:570] client connection to 127.0.105.61:35345 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260503 19:53:33.434511 1404 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.105.61:35345)
W20260503 19:53:33.434823 1404 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.105.61:35345)
I20260503 19:53:33.434774 1349 raft_consensus.cc:2468] T 6726412cbfd947d2849ac9774d4e86f4 P b68b1efd1cc44e15b8c69e730229dbdd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc902de5b2274103ab5e76f967503afb in term 1.
I20260503 19:53:33.435261 1147 leader_election.cc:304] T 6726412cbfd947d2849ac9774d4e86f4 P fc902de5b2274103ab5e76f967503afb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b68b1efd1cc44e15b8c69e730229dbdd, fc902de5b2274103ab5e76f967503afb; no voters:
W20260503 19:53:33.435416 1404 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.105.61:35345)
I20260503 19:53:33.435629 1648 raft_consensus.cc:2804] T 6726412cbfd947d2849ac9774d4e86f4 P fc902de5b2274103ab5e76f967503afb [term 1 FOLLOWER]: Leader election won for term 1
I20260503 19:53:33.435770 1648 raft_consensus.cc:697] T 6726412cbfd947d2849ac9774d4e86f4 P fc902de5b2274103ab5e76f967503afb [term 1 LEADER]: Becoming Leader. State: Replica: fc902de5b2274103ab5e76f967503afb, State: Running, Role: LEADER
I20260503 19:53:33.436103 1648 consensus_queue.cc:237] T 6726412cbfd947d2849ac9774d4e86f4 P fc902de5b2274103ab5e76f967503afb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } } peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } }
W20260503 19:53:33.436863 1404 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.105.61:35345)
W20260503 19:53:33.437290 1284 connection.cc:570] server connection from 127.0.0.1:36426 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260503 19:53:33.438128 1145 connection.cc:570] server connection from 127.0.0.1:58604 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260503 19:53:33.438880 724 connection.cc:570] client connection to 127.0.105.61:35345 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260503 19:53:33.439119 709 master_runner.cc:186] Error getting master registration for 127.0.105.61:35345: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107),
I20260503 19:53:33.439287 709 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260503 19:53:33.441970 1004 connection.cc:570] server connection from 127.0.0.1:47844 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260503 19:53:33.437522 1460 master_runner.cc:186] Error getting master registration for 127.0.105.61:35345: Network error: recv got EOF from 127.0.105.61:35345 (error 108),
I20260503 19:53:33.445533 1564 leader_election.cc:290] T 6726412cbfd947d2849ac9774d4e86f4 P fc902de5b2274103ab5e76f967503afb [CANDIDATE]: Term 1 election: Requested vote from peers b8594a7337c54dc58d701259120d361b (127.0.105.1:40289), b68b1efd1cc44e15b8c69e730229dbdd (127.0.105.3:41575)
I20260503 19:53:33.447376 1069 raft_consensus.cc:2468] T 6726412cbfd947d2849ac9774d4e86f4 P b8594a7337c54dc58d701259120d361b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc902de5b2274103ab5e76f967503afb in term 1.
I20260503 19:53:33.449133 1460 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260503 19:53:33.458083 1670 negotiation.cc:336] Failed RPC negotiation. Trace:
0503 19:53:33.431515 (+ 0us) reactor.cc:678] Submitting negotiation task for client connection to 127.0.105.61:35345 (local address 127.0.105.2:39691)
0503 19:53:33.456852 (+ 25337us) negotiation.cc:107] Waiting for socket to connect
0503 19:53:33.457015 (+ 163us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.105.61:35345: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":25210,"threads_started":1}
W20260503 19:53:33.458431 1671 negotiation.cc:336] Failed RPC negotiation. Trace:
0503 19:53:33.434297 (+ 0us) reactor.cc:678] Submitting negotiation task for client connection to 127.0.105.61:35345 (local address 127.0.105.3:34021)
0503 19:53:33.457451 (+ 23154us) negotiation.cc:107] Waiting for socket to connect
0503 19:53:33.457593 (+ 142us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.105.61:35345: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":23022,"threads_started":1}
I20260503 19:53:33.497494 1675 raft_consensus.cc:493] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2ba485a545f44fdd8217c3f86c79d0ca)
I20260503 19:53:33.497741 1675 raft_consensus.cc:515] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:33.498433 1675 leader_election.cc:290] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:53:33.499429 724 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.61:35345: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20260503 19:53:33.501761 724 leader_election.cc:336] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345): Network error: Client connection negotiation failed: client connection to 127.0.105.61:35345: connect: Connection refused (error 111)
I20260503 19:53:33.507323 1545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f3a96e537cec46df958468878509b9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 33 } } ignore_live_leader: false dest_uuid: "b0cad11e600c48a3af9a3c56115696d5" is_pre_election: true
I20260503 19:53:33.507881 1545 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3a96e537cec46df958468878509b9f9 in term 2.
I20260503 19:53:33.508152 1652 raft_consensus.cc:493] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2ba485a545f44fdd8217c3f86c79d0ca)
I20260503 19:53:33.508359 1652 raft_consensus.cc:515] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:33.508656 724 leader_election.cc:304] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9; no voters: 2ba485a545f44fdd8217c3f86c79d0ca
I20260503 19:53:33.509053 1675 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260503 19:53:33.509208 1675 raft_consensus.cc:493] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2ba485a545f44fdd8217c3f86c79d0ca)
I20260503 19:53:33.509299 1675 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260503 19:53:33.509796 1589 raft_consensus.cc:493] T 75dd4e31d81c4d5da2224e29e4f51e8d P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:53:33.509961 1589 raft_consensus.cc:515] T 75dd4e31d81c4d5da2224e29e4f51e8d P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } } peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } }
I20260503 19:53:33.510676 1589 leader_election.cc:290] T 75dd4e31d81c4d5da2224e29e4f51e8d P b8594a7337c54dc58d701259120d361b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc902de5b2274103ab5e76f967503afb (127.0.105.2:46339), b68b1efd1cc44e15b8c69e730229dbdd (127.0.105.3:41575)
I20260503 19:53:33.510756 1652 leader_election.cc:290] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345)
I20260503 19:53:33.511570 1349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75dd4e31d81c4d5da2224e29e4f51e8d" candidate_uuid: "b8594a7337c54dc58d701259120d361b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" is_pre_election: true
I20260503 19:53:33.511819 1349 raft_consensus.cc:2468] T 75dd4e31d81c4d5da2224e29e4f51e8d P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8594a7337c54dc58d701259120d361b in term 0.
I20260503 19:53:33.511811 1210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75dd4e31d81c4d5da2224e29e4f51e8d" candidate_uuid: "b8594a7337c54dc58d701259120d361b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc902de5b2274103ab5e76f967503afb" is_pre_election: true
I20260503 19:53:33.512032 1210 raft_consensus.cc:2468] T 75dd4e31d81c4d5da2224e29e4f51e8d P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8594a7337c54dc58d701259120d361b in term 0.
I20260503 19:53:33.512364 1675 raft_consensus.cc:515] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:33.512627 1005 leader_election.cc:304] T 75dd4e31d81c4d5da2224e29e4f51e8d P b8594a7337c54dc58d701259120d361b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b68b1efd1cc44e15b8c69e730229dbdd, b8594a7337c54dc58d701259120d361b; no voters:
I20260503 19:53:33.513149 1675 leader_election.cc:290] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 3 election: Requested vote from peers 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
I20260503 19:53:33.513875 749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b0cad11e600c48a3af9a3c56115696d5" candidate_term: 3 candidate_status { last_received { term: 2 index: 31 } } ignore_live_leader: false dest_uuid: "f3a96e537cec46df958468878509b9f9" is_pre_election: true
I20260503 19:53:33.513954 1589 raft_consensus.cc:2804] T 75dd4e31d81c4d5da2224e29e4f51e8d P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260503 19:53:33.514068 1589 raft_consensus.cc:493] T 75dd4e31d81c4d5da2224e29e4f51e8d P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260503 19:53:33.514160 1589 raft_consensus.cc:3060] T 75dd4e31d81c4d5da2224e29e4f51e8d P b8594a7337c54dc58d701259120d361b [term 0 FOLLOWER]: Advancing to term 1
I20260503 19:53:33.514179 749 raft_consensus.cc:2393] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b0cad11e600c48a3af9a3c56115696d5 in current term 3: Already voted for candidate f3a96e537cec46df958468878509b9f9 in this term.
I20260503 19:53:33.518026 1589 raft_consensus.cc:515] T 75dd4e31d81c4d5da2224e29e4f51e8d P b8594a7337c54dc58d701259120d361b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } } peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } }
W20260503 19:53:33.518424 724 leader_election.cc:336] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345): Network error: Client connection negotiation failed: client connection to 127.0.105.61:35345: connect: Connection refused (error 111)
I20260503 19:53:33.519342 1210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75dd4e31d81c4d5da2224e29e4f51e8d" candidate_uuid: "b8594a7337c54dc58d701259120d361b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc902de5b2274103ab5e76f967503afb"
I20260503 19:53:33.519585 1210 raft_consensus.cc:3060] T 75dd4e31d81c4d5da2224e29e4f51e8d P fc902de5b2274103ab5e76f967503afb [term 0 FOLLOWER]: Advancing to term 1
I20260503 19:53:33.521365 1589 leader_election.cc:290] T 75dd4e31d81c4d5da2224e29e4f51e8d P b8594a7337c54dc58d701259120d361b [CANDIDATE]: Term 1 election: Requested vote from peers fc902de5b2274103ab5e76f967503afb (127.0.105.2:46339), b68b1efd1cc44e15b8c69e730229dbdd (127.0.105.3:41575)
I20260503 19:53:33.522111 1349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75dd4e31d81c4d5da2224e29e4f51e8d" candidate_uuid: "b8594a7337c54dc58d701259120d361b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b68b1efd1cc44e15b8c69e730229dbdd"
I20260503 19:53:33.522341 1349 raft_consensus.cc:3060] T 75dd4e31d81c4d5da2224e29e4f51e8d P b68b1efd1cc44e15b8c69e730229dbdd [term 0 FOLLOWER]: Advancing to term 1
I20260503 19:53:33.522559 1210 raft_consensus.cc:2468] T 75dd4e31d81c4d5da2224e29e4f51e8d P fc902de5b2274103ab5e76f967503afb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8594a7337c54dc58d701259120d361b in term 1.
I20260503 19:53:33.522611 1545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f3a96e537cec46df958468878509b9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 33 } } ignore_live_leader: false dest_uuid: "b0cad11e600c48a3af9a3c56115696d5"
I20260503 19:53:33.522864 1545 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 2 FOLLOWER]: Advancing to term 3
I20260503 19:53:33.523051 1005 leader_election.cc:304] T 75dd4e31d81c4d5da2224e29e4f51e8d P b8594a7337c54dc58d701259120d361b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8594a7337c54dc58d701259120d361b, fc902de5b2274103ab5e76f967503afb; no voters:
I20260503 19:53:33.523480 1589 raft_consensus.cc:2804] T 75dd4e31d81c4d5da2224e29e4f51e8d P b8594a7337c54dc58d701259120d361b [term 1 FOLLOWER]: Leader election won for term 1
I20260503 19:53:33.523758 1589 raft_consensus.cc:697] T 75dd4e31d81c4d5da2224e29e4f51e8d P b8594a7337c54dc58d701259120d361b [term 1 LEADER]: Becoming Leader. State: Replica: b8594a7337c54dc58d701259120d361b, State: Running, Role: LEADER
I20260503 19:53:33.524641 1589 consensus_queue.cc:237] T 75dd4e31d81c4d5da2224e29e4f51e8d P b8594a7337c54dc58d701259120d361b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc902de5b2274103ab5e76f967503afb" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 46339 } } peers { permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 } } peers { permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 } }
I20260503 19:53:33.525231 1349 raft_consensus.cc:2468] T 75dd4e31d81c4d5da2224e29e4f51e8d P b68b1efd1cc44e15b8c69e730229dbdd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8594a7337c54dc58d701259120d361b in term 1.
I20260503 19:53:33.527137 1545 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3a96e537cec46df958468878509b9f9 in term 3.
I20260503 19:53:33.527750 724 leader_election.cc:304] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9; no voters: 2ba485a545f44fdd8217c3f86c79d0ca
I20260503 19:53:33.528137 1675 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 3 FOLLOWER]: Leader election won for term 3
I20260503 19:53:33.528827 1675 raft_consensus.cc:697] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 3 LEADER]: Becoming Leader. State: Replica: f3a96e537cec46df958468878509b9f9, State: Running, Role: LEADER
I20260503 19:53:33.529690 1675 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 2.33, Last appended by leader: 33, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
W20260503 19:53:33.536698 1517 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.61:35345: connect: Connection refused (error 111)
I20260503 19:53:33.537514 1681 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3a96e537cec46df958468878509b9f9. Latest consensus state: current_term: 3 leader_uuid: "f3a96e537cec46df958468878509b9f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:33.537889 1681 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [sys.catalog]: This master's current role is: LEADER
I20260503 19:53:33.538767 1683 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
W20260503 19:53:33.539153 1517 leader_election.cc:336] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345): Network error: Client connection negotiation failed: client connection to 127.0.105.61:35345: connect: Connection refused (error 111)
I20260503 19:53:33.539619 1517 leader_election.cc:304] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0cad11e600c48a3af9a3c56115696d5; no voters: 2ba485a545f44fdd8217c3f86c79d0ca, f3a96e537cec46df958468878509b9f9
I20260503 19:53:33.540084 1652 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260503 19:53:33.542025 1683 catalog_manager.cc:679] Loaded metadata for table default.table_15b3325c8812438786ade0fa5c759e21 [id=04d62d016fc24931ae6f439b1c6152c5]
I20260503 19:53:33.542555 1683 catalog_manager.cc:679] Loaded metadata for table default.table_f91787a2979849d78ba6ba4daf0b4e70 [id=40f19f1e7eaa4640b8c54d5e9e01c320]
I20260503 19:53:33.542968 1683 catalog_manager.cc:679] Loaded metadata for table default.table_0fc3df0b61324d7eac247468c05ce87f [id=54840346b61f4174a481e91d1f745c71]
I20260503 19:53:33.543426 1683 catalog_manager.cc:679] Loaded metadata for table default.table_9c30ca5293344be3964460b578f9d6df [id=8d8f5993b50b4a59aaa004b29cf7c6dd]
I20260503 19:53:33.543869 1683 catalog_manager.cc:679] Loaded metadata for table default.table_db1c67410d564682adca12ef0e942ca2 [id=bd45994325af4ed9ba34633dea7e140e]
I20260503 19:53:33.544261 1683 catalog_manager.cc:679] Loaded metadata for table default.table_dafc7a171f954df18eee7c2911f793f1 [id=da8af64ba75b4650858bb82728b61cd5]
I20260503 19:53:33.544627 1683 catalog_manager.cc:679] Loaded metadata for table default.table_1148c991089047a88d36862e99224cac [id=dce4d9d47ba04a3187de97305003ba5d]
I20260503 19:53:33.545017 1683 catalog_manager.cc:679] Loaded metadata for table default.table_f423a73d5c224651bb5eb676ee98c5e3 [id=f8aedcc7a5c34071a86b38b46361455f]
I20260503 19:53:33.545424 1683 catalog_manager.cc:679] Loaded metadata for table default.table_11964e32037841689422d2a453fff70f [id=f9d69e676109494f8ee9b41f557f4201]
I20260503 19:53:33.547505 1683 tablet_loader.cc:96] loaded metadata for tablet 1e0cc6e8b2734484a67fbb9572eb68e7 (table default.table_f423a73d5c224651bb5eb676ee98c5e3 [id=f8aedcc7a5c34071a86b38b46361455f])
I20260503 19:53:33.548058 1683 tablet_loader.cc:96] loaded metadata for tablet 47b6e48ca8c44c7e8bda3c62201b9dd1 (table default.table_0fc3df0b61324d7eac247468c05ce87f [id=54840346b61f4174a481e91d1f745c71])
I20260503 19:53:33.548852 1683 tablet_loader.cc:96] loaded metadata for tablet 6726412cbfd947d2849ac9774d4e86f4 (table default.table_15b3325c8812438786ade0fa5c759e21 [id=04d62d016fc24931ae6f439b1c6152c5])
I20260503 19:53:33.549549 1683 tablet_loader.cc:96] loaded metadata for tablet 75dd4e31d81c4d5da2224e29e4f51e8d (table default.table_db1c67410d564682adca12ef0e942ca2 [id=bd45994325af4ed9ba34633dea7e140e])
I20260503 19:53:33.550297 1683 tablet_loader.cc:96] loaded metadata for tablet a6cc88aae841477d990ed433c705a803 (table default.table_11964e32037841689422d2a453fff70f [id=f9d69e676109494f8ee9b41f557f4201])
I20260503 19:53:33.550765 1683 tablet_loader.cc:96] loaded metadata for tablet c4ef894c0fdb4414a6799627641a8857 (table default.table_9c30ca5293344be3964460b578f9d6df [id=8d8f5993b50b4a59aaa004b29cf7c6dd])
I20260503 19:53:33.551209 1683 tablet_loader.cc:96] loaded metadata for tablet df35135905b04fea927000d21e9994e6 (table default.table_1148c991089047a88d36862e99224cac [id=dce4d9d47ba04a3187de97305003ba5d])
I20260503 19:53:33.551578 1683 tablet_loader.cc:96] loaded metadata for tablet ed5861fcf94c47199eb1a81d4702f19b (table default.table_dafc7a171f954df18eee7c2911f793f1 [id=da8af64ba75b4650858bb82728b61cd5])
I20260503 19:53:33.552172 1683 tablet_loader.cc:96] loaded metadata for tablet f391e07ff0e24bc5b3cf7e81342161fe (table default.table_f91787a2979849d78ba6ba4daf0b4e70 [id=40f19f1e7eaa4640b8c54d5e9e01c320])
I20260503 19:53:33.552511 1683 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260503 19:53:33.553774 1683 catalog_manager.cc:1269] Loaded cluster ID: 636a1bc8fd6746489a9e892cd3f8ca43
I20260503 19:53:33.553886 1683 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260503 19:53:33.555132 1683 catalog_manager.cc:1514] Loading token signing keys...
I20260503 19:53:33.556038 1683 catalog_manager.cc:6055] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9: Loaded TSK: 0
I20260503 19:53:33.556941 1683 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260503 19:53:33.557950 1683 catalog_manager.cc:1537] Loading latest processed Hive Metastore notification log event ID...
I20260503 19:53:33.558796 1683 catalog_manager.cc:5893] Last processed Hive Metastore notification event ID: -1
I20260503 19:53:33.572261 739 master_service.cc:954] ReplaceTablet: received request to replace tablet 75dd4e31d81c4d5da2224e29e4f51e8d from {username='slave'} at 127.0.0.1:60652
I20260503 19:53:33.573063 740 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60652:
name: "default.Table_ba1a53aa8c174babb4bc2b60b558a8c1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:33.573680 738 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60652:
name: "default.Table_3f2252b7f4da4d9a9752ecd7ce54019f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:33.573167 737 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60652:
name: "default.Table_ad6e238ff40f4be99d7cc2d556a40a6b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:33.573231 736 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60652:
table { table_name: "default.Table_db1c67410d564682adca12ef0e942ca2" } modify_external_catalogs: true
I20260503 19:53:33.574887 736 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60652:
table { table_name: "default.Table_db1c67410d564682adca12ef0e942ca2" } modify_external_catalogs: true
I20260503 19:53:33.575318 734 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60652:
name: "default.Table_fa0eb6f31c344d07a5532aced2019227"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:33.576088 731 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60652:
table { table_name: "default.Table_f91787a2979849d78ba6ba4daf0b4e70" } modify_external_catalogs: true
I20260503 19:53:33.575996 732 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60652:
name: "default.Table_f28cafcdb9394779ac1e1d270bab59fb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:33.576321 731 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60652:
table { table_name: "default.Table_f91787a2979849d78ba6ba4daf0b4e70" } modify_external_catalogs: true
I20260503 19:53:33.580071 735 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60652:
table { table_name: "default.Table_15b3325c8812438786ade0fa5c759e21" } modify_external_catalogs: true
I20260503 19:53:33.580343 735 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60652:
table { table_name: "default.Table_15b3325c8812438786ade0fa5c759e21" } modify_external_catalogs: true
I20260503 19:53:33.580721 733 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60652:
table { table_name: "default.Table_11964e32037841689422d2a453fff70f" } modify_external_catalogs: true
I20260503 19:53:33.580967 733 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60652:
table { table_name: "default.Table_11964e32037841689422d2a453fff70f" } modify_external_catalogs: true
I20260503 19:53:33.583940 1545 raft_consensus.cc:1275] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 3 FOLLOWER]: Refusing update from remote peer f3a96e537cec46df958468878509b9f9: Log matching property violated. Preceding OpId in replica: term: 2 index: 31. Preceding OpId from leader: term: 3 index: 35. (index mismatch)
W20260503 19:53:33.585048 724 consensus_peers.cc:597] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 -> Peer 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345): Couldn't send request to peer 2ba485a545f44fdd8217c3f86c79d0ca. Status: Network error: Client connection negotiation failed: client connection to 127.0.105.61:35345: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260503 19:53:33.585037 1675 consensus_queue.cc:1048] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 31, Time since last communication: 0.000s
F20260503 19:53:33.585253 954 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 10 received events: NotificationEvent(eventId=1, eventTime=1777838011, eventType=CREATE_TABLE, dbName=default, tableName=table_cb2fc46da15e4d3cbc25e42aac22a26c, message={"server":"","servicePrincipal":"","db":"default","table":"table_cb2fc46da15e4d3cbc25e42aac22a26c","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_cb2fc46da15e4d3cbc25e42aac22a26c\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1777838010},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/warehouse/table_cb2fc46da15e4d3cbc25e42aac22a26c\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_cb2fc46da15e4d3cbc25e42aac22a26c\",\"kudu.table_id\":\"f40076244f874a50b8ee159c9c1b0786\",\"transient_lastDdlTime\":\"1777838010\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"636a1bc8fd6746489a9e892cd3f8ca43\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1777838011,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1777838011, eventType=CREATE_TABLE, dbName=default, tableName=table_f423a73d5c224651bb5eb676ee98c5e3, message={"server":"","servicePrincipal":"","db":"default","table":"table_f423a73d5c224651bb5eb676ee98c5e3","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_f423a73d5c224651bb5eb676ee98c5e3\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1777838010},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/warehouse/table_f423a73d5c224651bb5eb676ee98c5e3\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_f423a73d5c224651bb5eb676ee98c5e3\",\"kudu.table_id\":\"f8aedcc7a5c34071a86b38b46361455f\",\"transient_lastDdlTime\":\"1777838010\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"636a1bc8fd6746489a9e892cd3f8ca43\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1777838011,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
@ 0x7f8ff92fadcd google::LogMessage::Fail() at ??:0
@ 0x7f8ff92feb93 google::LogMessage::SendToLog() at ??:0
@ 0x7f8ff92fa7cc google::LogMessage::Flush() at ??:0
@ 0x7f8ff92faa11 google::LogMessage::~LogMessage() at ??:0
I20260503 19:53:33.594462 1652 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3a96e537cec46df958468878509b9f9. Latest consensus state: current_term: 3 leader_uuid: "f3a96e537cec46df958468878509b9f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:33.594672 1652 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260503 19:53:33.598105 1652 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "f3a96e537cec46df958468878509b9f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:33.598294 1652 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: This master's current role is: FOLLOWER
@ 0x7f900fdd3574 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260503 19:53:33.605455 739 catalog_manager.cc:5958] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9: Sending DeleteTablet for 3 replicas of tablet 75dd4e31d81c4d5da2224e29e4f51e8d
I20260503 19:53:33.607404 739 catalog_manager.cc:6741] ReplaceTablet: tablet 75dd4e31d81c4d5da2224e29e4f51e8d deleted and replaced by tablet 61730fd814344d6f85f8411bda525df0
I20260503 19:53:33.607777 739 master_service.cc:954] ReplaceTablet: received request to replace tablet 47b6e48ca8c44c7e8bda3c62201b9dd1 from {username='slave'} at 127.0.0.1:60652
@ 0x7f900fdd1e21 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260503 19:53:33.631575 1190 tablet_service.cc:1558] Processing DeleteTablet for tablet 75dd4e31d81c4d5da2224e29e4f51e8d with delete_type TABLET_DATA_DELETED (replaced by tablet 61730fd814344d6f85f8411bda525df0) from {username='slave'} at 127.0.0.1:32786
I20260503 19:53:33.632298 1696 tablet_replica.cc:333] T 75dd4e31d81c4d5da2224e29e4f51e8d P fc902de5b2274103ab5e76f967503afb: stopping tablet replica
I20260503 19:53:33.632511 1696 raft_consensus.cc:2243] T 75dd4e31d81c4d5da2224e29e4f51e8d P fc902de5b2274103ab5e76f967503afb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260503 19:53:33.632722 1696 raft_consensus.cc:2272] T 75dd4e31d81c4d5da2224e29e4f51e8d P fc902de5b2274103ab5e76f967503afb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260503 19:53:33.633383 1328 tablet_service.cc:1558] Processing DeleteTablet for tablet 75dd4e31d81c4d5da2224e29e4f51e8d with delete_type TABLET_DATA_DELETED (replaced by tablet 61730fd814344d6f85f8411bda525df0) from {username='slave'} at 127.0.0.1:39724
I20260503 19:53:33.633651 1632 tablet_replica.cc:333] T 75dd4e31d81c4d5da2224e29e4f51e8d P b68b1efd1cc44e15b8c69e730229dbdd: stopping tablet replica
I20260503 19:53:33.633845 1632 raft_consensus.cc:2243] T 75dd4e31d81c4d5da2224e29e4f51e8d P b68b1efd1cc44e15b8c69e730229dbdd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260503 19:53:33.633996 1632 raft_consensus.cc:2272] T 75dd4e31d81c4d5da2224e29e4f51e8d P b68b1efd1cc44e15b8c69e730229dbdd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260503 19:53:33.635223 1632 ts_tablet_manager.cc:1916] T 75dd4e31d81c4d5da2224e29e4f51e8d P b68b1efd1cc44e15b8c69e730229dbdd: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7f900fdd9a9c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260503 19:53:33.642061 1632 ts_tablet_manager.cc:1929] T 75dd4e31d81c4d5da2224e29e4f51e8d P b68b1efd1cc44e15b8c69e730229dbdd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260503 19:53:33.642213 1632 log.cc:1199] T 75dd4e31d81c4d5da2224e29e4f51e8d P b68b1efd1cc44e15b8c69e730229dbdd: Deleting WAL directory at /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-2/wal/wals/75dd4e31d81c4d5da2224e29e4f51e8d
I20260503 19:53:33.643030 1675 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "f3a96e537cec46df958468878509b9f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:33.643401 1675 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [sys.catalog]: This master's current role is: LEADER
I20260503 19:53:33.649494 1050 tablet_service.cc:1558] Processing DeleteTablet for tablet 75dd4e31d81c4d5da2224e29e4f51e8d with delete_type TABLET_DATA_DELETED (replaced by tablet 61730fd814344d6f85f8411bda525df0) from {username='slave'} at 127.0.0.1:59562
I20260503 19:53:33.650156 1632 ts_tablet_manager.cc:1950] T 75dd4e31d81c4d5da2224e29e4f51e8d P b68b1efd1cc44e15b8c69e730229dbdd: Deleting consensus metadata
I20260503 19:53:33.651688 1696 ts_tablet_manager.cc:1916] T 75dd4e31d81c4d5da2224e29e4f51e8d P fc902de5b2274103ab5e76f967503afb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260503 19:53:33.652989 739 catalog_manager.cc:5958] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9: Sending DeleteTablet for 3 replicas of tablet 47b6e48ca8c44c7e8bda3c62201b9dd1
I20260503 19:53:33.653472 739 catalog_manager.cc:6741] ReplaceTablet: tablet 47b6e48ca8c44c7e8bda3c62201b9dd1 deleted and replaced by tablet 542ff627b557426ab448a2339ef1c665
I20260503 19:53:33.653715 739 master_service.cc:954] ReplaceTablet: received request to replace tablet df35135905b04fea927000d21e9994e6 from {username='slave'} at 127.0.0.1:60652
I20260503 19:53:33.655203 1047 tablet_service.cc:1558] Processing DeleteTablet for tablet 47b6e48ca8c44c7e8bda3c62201b9dd1 with delete_type TABLET_DATA_DELETED (replaced by tablet 542ff627b557426ab448a2339ef1c665) from {username='slave'} at 127.0.0.1:59562
I20260503 19:53:33.658329 1698 tablet_replica.cc:333] T 75dd4e31d81c4d5da2224e29e4f51e8d P b8594a7337c54dc58d701259120d361b: stopping tablet replica
I20260503 19:53:33.658550 1698 raft_consensus.cc:2243] T 75dd4e31d81c4d5da2224e29e4f51e8d P b8594a7337c54dc58d701259120d361b [term 1 LEADER]: Raft consensus shutting down.
I20260503 19:53:33.658844 1698 pending_rounds.cc:70] T 75dd4e31d81c4d5da2224e29e4f51e8d P b8594a7337c54dc58d701259120d361b: Trying to abort 1 pending ops.
I20260503 19:53:33.658906 1698 pending_rounds.cc:77] T 75dd4e31d81c4d5da2224e29e4f51e8d P b8594a7337c54dc58d701259120d361b: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7282024503399890944 op_type: NO_OP noop_request { }
I20260503 19:53:33.659034 1698 raft_consensus.cc:2889] T 75dd4e31d81c4d5da2224e29e4f51e8d P b8594a7337c54dc58d701259120d361b [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260503 19:53:33.659129 1698 raft_consensus.cc:2272] T 75dd4e31d81c4d5da2224e29e4f51e8d P b8594a7337c54dc58d701259120d361b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260503 19:53:33.659840 739 catalog_manager.cc:5958] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9: Sending DeleteTablet for 3 replicas of tablet df35135905b04fea927000d21e9994e6
I20260503 19:53:33.660312 739 catalog_manager.cc:6741] ReplaceTablet: tablet df35135905b04fea927000d21e9994e6 deleted and replaced by tablet 6600b15c33ca480292d1c6eb3002eb86
I20260503 19:53:33.660544 739 master_service.cc:954] ReplaceTablet: received request to replace tablet 1e0cc6e8b2734484a67fbb9572eb68e7 from {username='slave'} at 127.0.0.1:60652
I20260503 19:53:33.661216 1050 tablet_service.cc:1558] Processing DeleteTablet for tablet df35135905b04fea927000d21e9994e6 with delete_type TABLET_DATA_DELETED (replaced by tablet 6600b15c33ca480292d1c6eb3002eb86) from {username='slave'} at 127.0.0.1:59562
I20260503 19:53:33.661631 1698 ts_tablet_manager.cc:1916] T 75dd4e31d81c4d5da2224e29e4f51e8d P b8594a7337c54dc58d701259120d361b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260503 19:53:33.666142 1696 ts_tablet_manager.cc:1929] T 75dd4e31d81c4d5da2224e29e4f51e8d P fc902de5b2274103ab5e76f967503afb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260503 19:53:33.666277 1696 log.cc:1199] T 75dd4e31d81c4d5da2224e29e4f51e8d P fc902de5b2274103ab5e76f967503afb: Deleting WAL directory at /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-1/wal/wals/75dd4e31d81c4d5da2224e29e4f51e8d
I20260503 19:53:33.666371 739 catalog_manager.cc:5958] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9: Sending DeleteTablet for 3 replicas of tablet 1e0cc6e8b2734484a67fbb9572eb68e7
I20260503 19:53:33.666698 1696 ts_tablet_manager.cc:1950] T 75dd4e31d81c4d5da2224e29e4f51e8d P fc902de5b2274103ab5e76f967503afb: Deleting consensus metadata
I20260503 19:53:33.666857 739 catalog_manager.cc:6741] ReplaceTablet: tablet 1e0cc6e8b2734484a67fbb9572eb68e7 deleted and replaced by tablet ac7d7fffebf5463abd8484bffa42bdc1
I20260503 19:53:33.667137 739 master_service.cc:954] ReplaceTablet: received request to replace tablet 47b6e48ca8c44c7e8bda3c62201b9dd1 from {username='slave'} at 127.0.0.1:60652
I20260503 19:53:33.667673 1050 tablet_service.cc:1558] Processing DeleteTablet for tablet 1e0cc6e8b2734484a67fbb9572eb68e7 with delete_type TABLET_DATA_DELETED (replaced by tablet ac7d7fffebf5463abd8484bffa42bdc1) from {username='slave'} at 127.0.0.1:59562
I20260503 19:53:33.669480 1698 ts_tablet_manager.cc:1929] T 75dd4e31d81c4d5da2224e29e4f51e8d P b8594a7337c54dc58d701259120d361b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260503 19:53:33.669582 1698 log.cc:1199] T 75dd4e31d81c4d5da2224e29e4f51e8d P b8594a7337c54dc58d701259120d361b: Deleting WAL directory at /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-0/wal/wals/75dd4e31d81c4d5da2224e29e4f51e8d
I20260503 19:53:33.669895 1698 ts_tablet_manager.cc:1950] T 75dd4e31d81c4d5da2224e29e4f51e8d P b8594a7337c54dc58d701259120d361b: Deleting consensus metadata
I20260503 19:53:33.671115 1698 tablet_replica.cc:333] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b8594a7337c54dc58d701259120d361b: stopping tablet replica
I20260503 19:53:33.671288 1698 raft_consensus.cc:2243] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b8594a7337c54dc58d701259120d361b [term 1 LEADER]: Raft consensus shutting down.
I20260503 19:53:33.671638 1698 raft_consensus.cc:2272] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b8594a7337c54dc58d701259120d361b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260503 19:53:33.672284 1698 ts_tablet_manager.cc:1916] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b8594a7337c54dc58d701259120d361b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260503 19:53:33.673349 726 catalog_manager.cc:5002] TS b8594a7337c54dc58d701259120d361b (127.0.105.1:40289): tablet 75dd4e31d81c4d5da2224e29e4f51e8d (table default.table_db1c67410d564682adca12ef0e942ca2 [id=bd45994325af4ed9ba34633dea7e140e]) successfully deleted
@ 0x7f900fdd9919 std::_Function_handler<>::_M_invoke() at ??:0
@ 0xaa4cad std::function<>::operator()() at ??:?
I20260503 19:53:33.677232 727 catalog_manager.cc:5002] TS b68b1efd1cc44e15b8c69e730229dbdd (127.0.105.3:41575): tablet 75dd4e31d81c4d5da2224e29e4f51e8d (table default.table_db1c67410d564682adca12ef0e942ca2 [id=bd45994325af4ed9ba34633dea7e140e]) successfully deleted
I20260503 19:53:33.678108 727 catalog_manager.cc:5002] TS fc902de5b2274103ab5e76f967503afb (127.0.105.2:46339): tablet 75dd4e31d81c4d5da2224e29e4f51e8d (table default.table_db1c67410d564682adca12ef0e942ca2 [id=bd45994325af4ed9ba34633dea7e140e]) successfully deleted
I20260503 19:53:33.678238 1190 tablet_service.cc:1558] Processing DeleteTablet for tablet 47b6e48ca8c44c7e8bda3c62201b9dd1 with delete_type TABLET_DATA_DELETED (replaced by tablet 542ff627b557426ab448a2339ef1c665) from {username='slave'} at 127.0.0.1:32786
I20260503 19:53:33.678504 1191 tablet_service.cc:1558] Processing DeleteTablet for tablet df35135905b04fea927000d21e9994e6 with delete_type TABLET_DATA_DELETED (replaced by tablet 6600b15c33ca480292d1c6eb3002eb86) from {username='slave'} at 127.0.0.1:32786
I20260503 19:53:33.678697 1189 tablet_service.cc:1558] Processing DeleteTablet for tablet 1e0cc6e8b2734484a67fbb9572eb68e7 with delete_type TABLET_DATA_DELETED (replaced by tablet ac7d7fffebf5463abd8484bffa42bdc1) from {username='slave'} at 127.0.0.1:32786
I20260503 19:53:33.678853 739 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60652:
table { table_name: "default.Table_1148c991089047a88d36862e99224cac" } new_table_name: "default.Table_a0d72b6ac2cf408296e54ae1978d1ff2" modify_external_catalogs: true
I20260503 19:53:33.679564 1328 tablet_service.cc:1558] Processing DeleteTablet for tablet 47b6e48ca8c44c7e8bda3c62201b9dd1 with delete_type TABLET_DATA_DELETED (replaced by tablet 542ff627b557426ab448a2339ef1c665) from {username='slave'} at 127.0.0.1:39724
I20260503 19:53:33.679814 1329 tablet_service.cc:1558] Processing DeleteTablet for tablet df35135905b04fea927000d21e9994e6 with delete_type TABLET_DATA_DELETED (replaced by tablet 6600b15c33ca480292d1c6eb3002eb86) from {username='slave'} at 127.0.0.1:39724
I20260503 19:53:33.680029 1327 tablet_service.cc:1558] Processing DeleteTablet for tablet 1e0cc6e8b2734484a67fbb9572eb68e7 with delete_type TABLET_DATA_DELETED (replaced by tablet ac7d7fffebf5463abd8484bffa42bdc1) from {username='slave'} at 127.0.0.1:39724
I20260503 19:53:33.680290 1698 ts_tablet_manager.cc:1929] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b8594a7337c54dc58d701259120d361b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260503 19:53:33.680436 1698 log.cc:1199] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b8594a7337c54dc58d701259120d361b: Deleting WAL directory at /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-0/wal/wals/47b6e48ca8c44c7e8bda3c62201b9dd1
I20260503 19:53:33.680948 1698 ts_tablet_manager.cc:1950] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b8594a7337c54dc58d701259120d361b: Deleting consensus metadata
I20260503 19:53:33.682271 1698 tablet_replica.cc:333] T df35135905b04fea927000d21e9994e6 P b8594a7337c54dc58d701259120d361b: stopping tablet replica
I20260503 19:53:33.682396 726 catalog_manager.cc:5002] TS b8594a7337c54dc58d701259120d361b (127.0.105.1:40289): tablet 47b6e48ca8c44c7e8bda3c62201b9dd1 (table default.table_0fc3df0b61324d7eac247468c05ce87f [id=54840346b61f4174a481e91d1f745c71]) successfully deleted
I20260503 19:53:33.682487 1698 raft_consensus.cc:2243] T df35135905b04fea927000d21e9994e6 P b8594a7337c54dc58d701259120d361b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260503 19:53:33.682646 1696 tablet_replica.cc:333] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P fc902de5b2274103ab5e76f967503afb: stopping tablet replica
I20260503 19:53:33.682672 1698 raft_consensus.cc:2272] T df35135905b04fea927000d21e9994e6 P b8594a7337c54dc58d701259120d361b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260503 19:53:33.682859 1696 raft_consensus.cc:2243] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P fc902de5b2274103ab5e76f967503afb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260503 19:53:33.683043 1696 raft_consensus.cc:2272] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P fc902de5b2274103ab5e76f967503afb [term 1 FOLLOWER]: Raft consensus is shut down!
@ 0x7f8ffc093529 kudu::Thread::SuperviseThread() at ??:0
I20260503 19:53:33.683344 1698 ts_tablet_manager.cc:1916] T df35135905b04fea927000d21e9994e6 P b8594a7337c54dc58d701259120d361b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260503 19:53:33.683883 1696 ts_tablet_manager.cc:1916] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P fc902de5b2274103ab5e76f967503afb: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7f8fff6816db start_thread at ??:0
@ 0x7f8ff81ab61f clone at ??:0
I20260503 19:53:33.688208 1632 tablet_replica.cc:333] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b68b1efd1cc44e15b8c69e730229dbdd: stopping tablet replica
I20260503 19:53:33.688443 1632 raft_consensus.cc:2243] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b68b1efd1cc44e15b8c69e730229dbdd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260503 19:53:33.688678 1632 raft_consensus.cc:2272] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b68b1efd1cc44e15b8c69e730229dbdd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260503 19:53:33.689620 1632 ts_tablet_manager.cc:1916] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b68b1efd1cc44e15b8c69e730229dbdd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260503 19:53:33.690584 1698 ts_tablet_manager.cc:1929] T df35135905b04fea927000d21e9994e6 P b8594a7337c54dc58d701259120d361b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260503 19:53:33.690704 1698 log.cc:1199] T df35135905b04fea927000d21e9994e6 P b8594a7337c54dc58d701259120d361b: Deleting WAL directory at /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-0/wal/wals/df35135905b04fea927000d21e9994e6
I20260503 19:53:33.690960 1698 ts_tablet_manager.cc:1950] T df35135905b04fea927000d21e9994e6 P b8594a7337c54dc58d701259120d361b: Deleting consensus metadata
I20260503 19:53:33.692154 1698 tablet_replica.cc:333] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b8594a7337c54dc58d701259120d361b: stopping tablet replica
I20260503 19:53:33.692333 1698 raft_consensus.cc:2243] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b8594a7337c54dc58d701259120d361b [term 1 LEADER]: Raft consensus shutting down.
I20260503 19:53:33.692670 1698 raft_consensus.cc:2272] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b8594a7337c54dc58d701259120d361b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260503 19:53:33.693701 1696 ts_tablet_manager.cc:1929] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P fc902de5b2274103ab5e76f967503afb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260503 19:53:33.693827 1696 log.cc:1199] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P fc902de5b2274103ab5e76f967503afb: Deleting WAL directory at /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-1/wal/wals/47b6e48ca8c44c7e8bda3c62201b9dd1
I20260503 19:53:33.693815 1698 ts_tablet_manager.cc:1916] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b8594a7337c54dc58d701259120d361b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260503 19:53:33.694106 1696 ts_tablet_manager.cc:1950] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P fc902de5b2274103ab5e76f967503afb: Deleting consensus metadata
I20260503 19:53:33.694943 1696 tablet_replica.cc:333] T df35135905b04fea927000d21e9994e6 P fc902de5b2274103ab5e76f967503afb: stopping tablet replica
I20260503 19:53:33.695129 1696 raft_consensus.cc:2243] T df35135905b04fea927000d21e9994e6 P fc902de5b2274103ab5e76f967503afb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260503 19:53:33.695294 1696 raft_consensus.cc:2272] T df35135905b04fea927000d21e9994e6 P fc902de5b2274103ab5e76f967503afb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260503 19:53:33.695843 1696 ts_tablet_manager.cc:1916] T df35135905b04fea927000d21e9994e6 P fc902de5b2274103ab5e76f967503afb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260503 19:53:33.696576 1632 ts_tablet_manager.cc:1929] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b68b1efd1cc44e15b8c69e730229dbdd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260503 19:53:33.696694 1632 log.cc:1199] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b68b1efd1cc44e15b8c69e730229dbdd: Deleting WAL directory at /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-2/wal/wals/47b6e48ca8c44c7e8bda3c62201b9dd1
I20260503 19:53:33.697114 1632 ts_tablet_manager.cc:1950] T 47b6e48ca8c44c7e8bda3c62201b9dd1 P b68b1efd1cc44e15b8c69e730229dbdd: Deleting consensus metadata
I20260503 19:53:33.698184 1632 tablet_replica.cc:333] T df35135905b04fea927000d21e9994e6 P b68b1efd1cc44e15b8c69e730229dbdd: stopping tablet replica
I20260503 19:53:33.698374 1632 raft_consensus.cc:2243] T df35135905b04fea927000d21e9994e6 P b68b1efd1cc44e15b8c69e730229dbdd [term 1 LEADER]: Raft consensus shutting down.
I20260503 19:53:33.698745 1632 pending_rounds.cc:70] T df35135905b04fea927000d21e9994e6 P b68b1efd1cc44e15b8c69e730229dbdd: Trying to abort 1 pending ops.
I20260503 19:53:33.698904 1632 pending_rounds.cc:77] T df35135905b04fea927000d21e9994e6 P b68b1efd1cc44e15b8c69e730229dbdd: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7282024500577767424 op_type: NO_OP noop_request { }
I20260503 19:53:33.699047 1632 raft_consensus.cc:2889] T df35135905b04fea927000d21e9994e6 P b68b1efd1cc44e15b8c69e730229dbdd [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260503 19:53:33.699157 1632 raft_consensus.cc:2272] T df35135905b04fea927000d21e9994e6 P b68b1efd1cc44e15b8c69e730229dbdd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260503 19:53:33.699708 1698 ts_tablet_manager.cc:1929] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b8594a7337c54dc58d701259120d361b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260503 19:53:33.699812 1698 log.cc:1199] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b8594a7337c54dc58d701259120d361b: Deleting WAL directory at /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-0/wal/wals/1e0cc6e8b2734484a67fbb9572eb68e7
I20260503 19:53:33.699831 1632 ts_tablet_manager.cc:1916] T df35135905b04fea927000d21e9994e6 P b68b1efd1cc44e15b8c69e730229dbdd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260503 19:53:33.700218 1698 ts_tablet_manager.cc:1950] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b8594a7337c54dc58d701259120d361b: Deleting consensus metadata
I20260503 19:53:33.702440 1696 ts_tablet_manager.cc:1929] T df35135905b04fea927000d21e9994e6 P fc902de5b2274103ab5e76f967503afb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260503 19:53:33.702577 1696 log.cc:1199] T df35135905b04fea927000d21e9994e6 P fc902de5b2274103ab5e76f967503afb: Deleting WAL directory at /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-1/wal/wals/df35135905b04fea927000d21e9994e6
I20260503 19:53:33.702955 1696 ts_tablet_manager.cc:1950] T df35135905b04fea927000d21e9994e6 P fc902de5b2274103ab5e76f967503afb: Deleting consensus metadata
I20260503 19:53:33.703882 1696 tablet_replica.cc:333] T 1e0cc6e8b2734484a67fbb9572eb68e7 P fc902de5b2274103ab5e76f967503afb: stopping tablet replica
I20260503 19:53:33.704074 1696 raft_consensus.cc:2243] T 1e0cc6e8b2734484a67fbb9572eb68e7 P fc902de5b2274103ab5e76f967503afb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260503 19:53:33.704244 1696 raft_consensus.cc:2272] T 1e0cc6e8b2734484a67fbb9572eb68e7 P fc902de5b2274103ab5e76f967503afb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260503 19:53:33.705721 1696 ts_tablet_manager.cc:1916] T 1e0cc6e8b2734484a67fbb9572eb68e7 P fc902de5b2274103ab5e76f967503afb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260503 19:53:33.706642 1632 ts_tablet_manager.cc:1929] T df35135905b04fea927000d21e9994e6 P b68b1efd1cc44e15b8c69e730229dbdd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260503 19:53:33.706736 1632 log.cc:1199] T df35135905b04fea927000d21e9994e6 P b68b1efd1cc44e15b8c69e730229dbdd: Deleting WAL directory at /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-2/wal/wals/df35135905b04fea927000d21e9994e6
I20260503 19:53:33.707031 1632 ts_tablet_manager.cc:1950] T df35135905b04fea927000d21e9994e6 P b68b1efd1cc44e15b8c69e730229dbdd: Deleting consensus metadata
I20260503 19:53:33.708159 1632 tablet_replica.cc:333] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b68b1efd1cc44e15b8c69e730229dbdd: stopping tablet replica
I20260503 19:53:33.708374 1632 raft_consensus.cc:2243] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b68b1efd1cc44e15b8c69e730229dbdd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260503 19:53:33.708561 1632 raft_consensus.cc:2272] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b68b1efd1cc44e15b8c69e730229dbdd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260503 19:53:33.709378 1632 ts_tablet_manager.cc:1916] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b68b1efd1cc44e15b8c69e730229dbdd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260503 19:53:33.712605 1696 ts_tablet_manager.cc:1929] T 1e0cc6e8b2734484a67fbb9572eb68e7 P fc902de5b2274103ab5e76f967503afb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260503 19:53:33.712772 1696 log.cc:1199] T 1e0cc6e8b2734484a67fbb9572eb68e7 P fc902de5b2274103ab5e76f967503afb: Deleting WAL directory at /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-1/wal/wals/1e0cc6e8b2734484a67fbb9572eb68e7
I20260503 19:53:33.713255 1696 ts_tablet_manager.cc:1950] T 1e0cc6e8b2734484a67fbb9572eb68e7 P fc902de5b2274103ab5e76f967503afb: Deleting consensus metadata
I20260503 19:53:33.716130 1632 ts_tablet_manager.cc:1929] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b68b1efd1cc44e15b8c69e730229dbdd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260503 19:53:33.716272 1632 log.cc:1199] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b68b1efd1cc44e15b8c69e730229dbdd: Deleting WAL directory at /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/ts-2/wal/wals/1e0cc6e8b2734484a67fbb9572eb68e7
I20260503 19:53:33.716691 1632 ts_tablet_manager.cc:1950] T 1e0cc6e8b2734484a67fbb9572eb68e7 P b68b1efd1cc44e15b8c69e730229dbdd: Deleting consensus metadata
W20260503 19:53:33.879889 1283 connection.cc:570] server connection from 127.0.0.1:39724 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260503 19:53:33.879909 1146 connection.cc:570] server connection from 127.0.0.1:32786 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260503 19:53:33.879917 1002 connection.cc:570] server connection from 127.0.0.1:59562 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260503 19:53:33.880214 1404 connection.cc:570] client connection to 127.0.105.62:33821 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260503 19:53:33.881347 1282 connection.cc:570] client connection to 127.0.105.62:33821 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260503 19:53:33.940665 1648 consensus_queue.cc:1048] T f391e07ff0e24bc5b3cf7e81342161fe P fc902de5b2274103ab5e76f967503afb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260503 19:53:33.947413 1564 consensus_queue.cc:1048] T f391e07ff0e24bc5b3cf7e81342161fe P fc902de5b2274103ab5e76f967503afb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260503 19:53:34.185289 1714 raft_consensus.cc:493] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3a96e537cec46df958468878509b9f9)
I20260503 19:53:34.185541 1714 raft_consensus.cc:515] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:34.186343 1714 leader_election.cc:290] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345)
W20260503 19:53:34.188777 1517 leader_election.cc:336] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:53:34.189018 1517 leader_election.cc:336] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345): Network error: Client connection negotiation failed: client connection to 127.0.105.61:35345: connect: Connection refused (error 111)
I20260503 19:53:34.189189 1517 leader_election.cc:304] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0cad11e600c48a3af9a3c56115696d5; no voters: 2ba485a545f44fdd8217c3f86c79d0ca, f3a96e537cec46df958468878509b9f9
I20260503 19:53:34.189513 1714 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260503 19:53:34.216434 1564 consensus_queue.cc:1048] T a6cc88aae841477d990ed433c705a803 P fc902de5b2274103ab5e76f967503afb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260503 19:53:34.219305 1564 consensus_queue.cc:1048] T c4ef894c0fdb4414a6799627641a8857 P fc902de5b2274103ab5e76f967503afb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260503 19:53:34.225569 1719 consensus_queue.cc:1048] T a6cc88aae841477d990ed433c705a803 P fc902de5b2274103ab5e76f967503afb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260503 19:53:34.229626 1723 consensus_queue.cc:1048] T c4ef894c0fdb4414a6799627641a8857 P fc902de5b2274103ab5e76f967503afb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260503 19:53:34.431574 1722 consensus_queue.cc:1048] T 6726412cbfd947d2849ac9774d4e86f4 P fc902de5b2274103ab5e76f967503afb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 41575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260503 19:53:34.436223 1648 consensus_queue.cc:1048] T 6726412cbfd947d2849ac9774d4e86f4 P fc902de5b2274103ab5e76f967503afb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8594a7337c54dc58d701259120d361b" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 40289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260503 19:53:34.736788 1730 raft_consensus.cc:493] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3a96e537cec46df958468878509b9f9)
I20260503 19:53:34.737020 1730 raft_consensus.cc:515] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:34.737704 1730 leader_election.cc:290] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345)
W20260503 19:53:34.739876 1517 leader_election.cc:336] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:53:34.740121 1517 leader_election.cc:336] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345): Network error: Client connection negotiation failed: client connection to 127.0.105.61:35345: connect: Connection refused (error 111)
I20260503 19:53:34.740247 1517 leader_election.cc:304] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0cad11e600c48a3af9a3c56115696d5; no voters: 2ba485a545f44fdd8217c3f86c79d0ca, f3a96e537cec46df958468878509b9f9
I20260503 19:53:34.740535 1730 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260503 19:53:35.308141 1738 raft_consensus.cc:493] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3a96e537cec46df958468878509b9f9)
I20260503 19:53:35.308393 1738 raft_consensus.cc:515] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:35.309072 1738 leader_election.cc:290] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345)
W20260503 19:53:35.314407 1517 leader_election.cc:336] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:53:35.314675 1517 leader_election.cc:336] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345): Network error: Client connection negotiation failed: client connection to 127.0.105.61:35345: connect: Connection refused (error 111)
I20260503 19:53:35.314800 1517 leader_election.cc:304] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0cad11e600c48a3af9a3c56115696d5; no voters: 2ba485a545f44fdd8217c3f86c79d0ca, f3a96e537cec46df958468878509b9f9
I20260503 19:53:35.315119 1738 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260503 19:53:35.902742 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-task12vCql/build/asan/bin/kudu
/tmp/dist-test-task12vCql/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.61:35345
--webserver_interface=127.0.105.61
--webserver_port=39263
--builtin_ntp_servers=127.0.105.20:36687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203
--hive_metastore_uris=thrift://127.0.0.1:38395
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260503 19:53:36.247848 1744 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260503 19:53:36.248211 1744 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260503 19:53:36.248337 1744 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260503 19:53:36.257773 1744 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260503 19:53:36.257876 1744 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260503 19:53:36.257987 1744 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260503 19:53:36.258073 1744 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260503 19:53:36.268750 1744 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.105.20:36687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:38395
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.61:35345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.105.61
--webserver_port=39263
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 0ba7f54e15c9f673bf3e350f30b8d5f1a4037b79
build type FASTDEBUG
built by None at 03 May 2026 19:43:25 UTC on bdcb31816ec0
build id 11730
ASAN enabled
I20260503 19:53:36.270634 1744 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260503 19:53:36.272490 1744 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260503 19:53:36.282370 1752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260503 19:53:36.282403 1753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260503 19:53:36.282749 1755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260503 19:53:36.283897 1744 server_base.cc:1061] running on GCE node
I20260503 19:53:36.284672 1744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260503 19:53:36.286289 1744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260503 19:53:36.287560 1744 hybrid_clock.cc:648] HybridClock initialized: now 1777838016287518 us; error 58 us; skew 500 ppm
I20260503 19:53:36.288064 1744 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260503 19:53:36.290858 1744 webserver.cc:492] Webserver started at http://127.0.105.61:39263/ using document root <none> and password file <none>
I20260503 19:53:36.291663 1744 fs_manager.cc:362] Metadata directory not provided
I20260503 19:53:36.291818 1744 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260503 19:53:36.298151 1744 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260503 19:53:36.301416 1761 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260503 19:53:36.303064 1744 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260503 19:53:36.303272 1744 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data,/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/wal
uuid: "2ba485a545f44fdd8217c3f86c79d0ca"
format_stamp: "Formatted at 2026-05-03 19:53:27 on dist-test-slave-gl77"
I20260503 19:53:36.304127 1744 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260503 19:53:36.321733 1744 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260503 19:53:36.322631 1744 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260503 19:53:36.322938 1744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260503 19:53:36.344547 1744 rpc_server.cc:307] RPC server started. Bound to: 127.0.105.61:35345
I20260503 19:53:36.344590 1812 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.105.61:35345 every 8 connection(s)
I20260503 19:53:36.346269 1744 server_base.cc:1193] Dumped server information to /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data/info.pb
I20260503 19:53:36.351742 420 external_mini_cluster.cc:1428] Started /tmp/dist-test-task12vCql/build/asan/bin/kudu as pid 1744
I20260503 19:53:36.353776 1813 sys_catalog.cc:263] Verifying existing consensus state
I20260503 19:53:36.358768 1813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: Bootstrap starting.
I20260503 19:53:36.371677 1813 log.cc:826] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: Log is configured to *not* fsync() on all Append() calls
I20260503 19:53:36.405817 1813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=33 overwritten=0 applied=33 ignored=0} inserts{seen=24 ignored=0} mutations{seen=19 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260503 19:53:36.406420 1813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: Bootstrap complete.
I20260503 19:53:36.412160 1813 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:36.412887 1813 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba485a545f44fdd8217c3f86c79d0ca, State: Initialized, Role: FOLLOWER
I20260503 19:53:36.413647 1813 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 33, Last appended: 2.33, Last appended by leader: 33, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:36.414534 1817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:36.415052 1817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: This master's current role is: FOLLOWER
I20260503 19:53:36.415951 1813 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: configured and running, proceeding with master startup.
I20260503 19:53:36.436398 1744 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:53:36.444921 1744 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:53:36.460968 1261 heartbeater.cc:344] Connected to a master server at 127.0.105.61:35345
I20260503 19:53:36.465188 1778 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "fc902de5b2274103ab5e76f967503afb" instance_seqno: 1777838009464378) as {username='slave'} at 127.0.105.2:33817; Asking this server to re-register.
I20260503 19:53:36.466392 1261 heartbeater.cc:461] Registering TS with master...
I20260503 19:53:36.468616 1778 ts_manager.cc:194] Registered new tserver with Master: fc902de5b2274103ab5e76f967503afb (127.0.105.2:46339)
I20260503 19:53:36.482623 1831 catalog_manager.cc:1269] Loaded cluster ID: 636a1bc8fd6746489a9e892cd3f8ca43
I20260503 19:53:36.482887 1831 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: loading cluster ID for follower catalog manager: success
I20260503 19:53:36.483009 1396 heartbeater.cc:344] Connected to a master server at 127.0.105.61:35345
I20260503 19:53:36.485067 1778 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" instance_seqno: 1777838010067456) as {username='slave'} at 127.0.105.3:44977; Asking this server to re-register.
I20260503 19:53:36.486827 1117 heartbeater.cc:344] Connected to a master server at 127.0.105.61:35345
I20260503 19:53:36.487366 1831 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: acquiring CA information for follower catalog manager: success
I20260503 19:53:36.488273 1396 heartbeater.cc:461] Registering TS with master...
I20260503 19:53:36.489492 1778 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "b8594a7337c54dc58d701259120d361b" instance_seqno: 1777838008819588) as {username='slave'} at 127.0.105.1:42521; Asking this server to re-register.
I20260503 19:53:36.490275 1117 heartbeater.cc:461] Registering TS with master...
I20260503 19:53:36.490284 1777 ts_manager.cc:194] Registered new tserver with Master: b68b1efd1cc44e15b8c69e730229dbdd (127.0.105.3:41575)
I20260503 19:53:36.490381 1831 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260503 19:53:36.491786 1777 ts_manager.cc:194] Registered new tserver with Master: b8594a7337c54dc58d701259120d361b (127.0.105.1:40289)
I20260503 19:53:36.534058 1842 raft_consensus.cc:493] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3a96e537cec46df958468878509b9f9)
I20260503 19:53:36.534282 1842 raft_consensus.cc:515] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:36.534950 1842 leader_election.cc:290] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345)
W20260503 19:53:36.537904 1517 leader_election.cc:336] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
I20260503 19:53:36.543751 1788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b0cad11e600c48a3af9a3c56115696d5" candidate_term: 4 candidate_status { last_received { term: 3 index: 39 } } ignore_live_leader: false dest_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" is_pre_election: true
I20260503 19:53:36.544586 1788 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0cad11e600c48a3af9a3c56115696d5 in term 2.
I20260503 19:53:36.545254 1517 leader_election.cc:304] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca, b0cad11e600c48a3af9a3c56115696d5; no voters: f3a96e537cec46df958468878509b9f9
I20260503 19:53:36.545590 1842 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260503 19:53:36.545773 1842 raft_consensus.cc:493] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f3a96e537cec46df958468878509b9f9)
I20260503 19:53:36.545862 1842 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 3 FOLLOWER]: Advancing to term 4
I20260503 19:53:36.547978 1842 raft_consensus.cc:515] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:36.548748 1842 leader_election.cc:290] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 4 election: Requested vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345)
I20260503 19:53:36.550076 1788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b0cad11e600c48a3af9a3c56115696d5" candidate_term: 4 candidate_status { last_received { term: 3 index: 39 } } ignore_live_leader: false dest_uuid: "2ba485a545f44fdd8217c3f86c79d0ca"
I20260503 19:53:36.550305 1788 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 2 FOLLOWER]: Advancing to term 4
W20260503 19:53:36.550855 1517 leader_election.cc:336] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
I20260503 19:53:36.556043 1788 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0cad11e600c48a3af9a3c56115696d5 in term 4.
I20260503 19:53:36.556628 1517 leader_election.cc:304] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca, b0cad11e600c48a3af9a3c56115696d5; no voters: f3a96e537cec46df958468878509b9f9
I20260503 19:53:36.556998 1842 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 4 FOLLOWER]: Leader election won for term 4
I20260503 19:53:36.561367 1842 raft_consensus.cc:697] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 4 LEADER]: Becoming Leader. State: Replica: b0cad11e600c48a3af9a3c56115696d5, State: Running, Role: LEADER
I20260503 19:53:36.561969 1842 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 39, Committed index: 39, Last appended: 3.39, Last appended by leader: 39, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:36.563891 1846 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0cad11e600c48a3af9a3c56115696d5. Latest consensus state: current_term: 4 leader_uuid: "b0cad11e600c48a3af9a3c56115696d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:36.564178 1846 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: This master's current role is: LEADER
I20260503 19:53:36.565433 1848 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260503 19:53:36.568722 1848 catalog_manager.cc:679] Loaded metadata for table default.table_15b3325c8812438786ade0fa5c759e21 [id=04d62d016fc24931ae6f439b1c6152c5]
I20260503 19:53:36.569388 1848 catalog_manager.cc:679] Loaded metadata for table default.table_f91787a2979849d78ba6ba4daf0b4e70 [id=40f19f1e7eaa4640b8c54d5e9e01c320]
I20260503 19:53:36.569787 1848 catalog_manager.cc:679] Loaded metadata for table default.table_0fc3df0b61324d7eac247468c05ce87f [id=54840346b61f4174a481e91d1f745c71]
I20260503 19:53:36.570219 1848 catalog_manager.cc:679] Loaded metadata for table default.table_9c30ca5293344be3964460b578f9d6df [id=8d8f5993b50b4a59aaa004b29cf7c6dd]
I20260503 19:53:36.570571 1848 catalog_manager.cc:679] Loaded metadata for table default.table_db1c67410d564682adca12ef0e942ca2 [id=bd45994325af4ed9ba34633dea7e140e]
I20260503 19:53:36.570945 1848 catalog_manager.cc:679] Loaded metadata for table default.table_dafc7a171f954df18eee7c2911f793f1 [id=da8af64ba75b4650858bb82728b61cd5]
I20260503 19:53:36.571287 1848 catalog_manager.cc:679] Loaded metadata for table default.table_1148c991089047a88d36862e99224cac [id=dce4d9d47ba04a3187de97305003ba5d]
I20260503 19:53:36.571635 1848 catalog_manager.cc:679] Loaded metadata for table default.table_f423a73d5c224651bb5eb676ee98c5e3 [id=f8aedcc7a5c34071a86b38b46361455f]
I20260503 19:53:36.572031 1848 catalog_manager.cc:679] Loaded metadata for table default.table_11964e32037841689422d2a453fff70f [id=f9d69e676109494f8ee9b41f557f4201]
I20260503 19:53:36.574218 1848 tablet_loader.cc:96] loaded metadata for tablet 542ff627b557426ab448a2339ef1c665 (table default.table_0fc3df0b61324d7eac247468c05ce87f [id=54840346b61f4174a481e91d1f745c71])
I20260503 19:53:36.574599 1848 tablet_loader.cc:96] loaded metadata for tablet 61730fd814344d6f85f8411bda525df0 (table default.table_db1c67410d564682adca12ef0e942ca2 [id=bd45994325af4ed9ba34633dea7e140e])
I20260503 19:53:36.575114 1848 tablet_loader.cc:96] loaded metadata for tablet 6600b15c33ca480292d1c6eb3002eb86 (table default.table_1148c991089047a88d36862e99224cac [id=dce4d9d47ba04a3187de97305003ba5d])
I20260503 19:53:36.575493 1848 tablet_loader.cc:96] loaded metadata for tablet 6726412cbfd947d2849ac9774d4e86f4 (table default.table_15b3325c8812438786ade0fa5c759e21 [id=04d62d016fc24931ae6f439b1c6152c5])
I20260503 19:53:36.576542 1848 tablet_loader.cc:96] loaded metadata for tablet a6cc88aae841477d990ed433c705a803 (table default.table_11964e32037841689422d2a453fff70f [id=f9d69e676109494f8ee9b41f557f4201])
I20260503 19:53:36.576813 1848 tablet_loader.cc:96] loaded metadata for tablet ac7d7fffebf5463abd8484bffa42bdc1 (table default.table_f423a73d5c224651bb5eb676ee98c5e3 [id=f8aedcc7a5c34071a86b38b46361455f])
I20260503 19:53:36.577185 1848 tablet_loader.cc:96] loaded metadata for tablet c4ef894c0fdb4414a6799627641a8857 (table default.table_9c30ca5293344be3964460b578f9d6df [id=8d8f5993b50b4a59aaa004b29cf7c6dd])
I20260503 19:53:36.577761 1848 tablet_loader.cc:96] loaded metadata for tablet ed5861fcf94c47199eb1a81d4702f19b (table default.table_dafc7a171f954df18eee7c2911f793f1 [id=da8af64ba75b4650858bb82728b61cd5])
I20260503 19:53:36.578118 1848 tablet_loader.cc:96] loaded metadata for tablet f391e07ff0e24bc5b3cf7e81342161fe (table default.table_f91787a2979849d78ba6ba4daf0b4e70 [id=40f19f1e7eaa4640b8c54d5e9e01c320])
I20260503 19:53:36.578390 1848 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260503 19:53:36.579362 1848 catalog_manager.cc:1269] Loaded cluster ID: 636a1bc8fd6746489a9e892cd3f8ca43
I20260503 19:53:36.579468 1848 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260503 19:53:36.580655 1848 catalog_manager.cc:1514] Loading token signing keys...
I20260503 19:53:36.581601 1848 catalog_manager.cc:6055] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5: Loaded TSK: 0
I20260503 19:53:36.582580 1848 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260503 19:53:36.583436 1848 catalog_manager.cc:1537] Loading latest processed Hive Metastore notification log event ID...
I20260503 19:53:36.584237 1848 catalog_manager.cc:5893] Last processed Hive Metastore notification event ID: -1
I20260503 19:53:36.624373 1535 master_service.cc:954] ReplaceTablet: received request to replace tablet a6cc88aae841477d990ed433c705a803 from {username='slave'} at 127.0.0.1:45898
I20260503 19:53:36.626331 1526 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45898:
table { table_name: "default.Table_0fc3df0b61324d7eac247468c05ce87f" } new_table_name: "default.Table_dd74e45a321f4e7f92e8b514d7378c86" modify_external_catalogs: true
I20260503 19:53:36.626978 1534 master_service.cc:954] ReplaceTablet: received request to replace tablet f391e07ff0e24bc5b3cf7e81342161fe from {username='slave'} at 127.0.0.1:45898
I20260503 19:53:36.627622 1527 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45898:
table { table_name: "default.Table_f423a73d5c224651bb5eb676ee98c5e3" } new_table_name: "default.Table_90069e6755454566a407bbb659d59d9e" modify_external_catalogs: true
I20260503 19:53:36.628036 1533 master_service.cc:954] ReplaceTablet: received request to replace tablet c4ef894c0fdb4414a6799627641a8857 from {username='slave'} at 127.0.0.1:45898
I20260503 19:53:36.629549 1528 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45898:
table { table_name: "default.Table_9c30ca5293344be3964460b578f9d6df" } new_table_name: "default.Table_90313d18e9a542119fe8e0e296d4396c" modify_external_catalogs: true
I20260503 19:53:36.631179 1530 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45898:
table { table_name: "default.Table_dafc7a171f954df18eee7c2911f793f1" } new_table_name: "default.Table_d2f4bbb38f1d4d0191643de99eeb35b6" modify_external_catalogs: true
I20260503 19:53:36.631327 1529 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45898:
name: "default.Table_b578e2eeb11a4b698a7b51e206e1d015"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:36.631736 1531 master_service.cc:954] ReplaceTablet: received request to replace tablet f391e07ff0e24bc5b3cf7e81342161fe from {username='slave'} at 127.0.0.1:45898
I20260503 19:53:36.632086 1532 master_service.cc:954] ReplaceTablet: received request to replace tablet f391e07ff0e24bc5b3cf7e81342161fe from {username='slave'} at 127.0.0.1:45898
I20260503 19:53:36.633991 1788 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 4 FOLLOWER]: Refusing update from remote peer b0cad11e600c48a3af9a3c56115696d5: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 4 index: 41. (index mismatch)
I20260503 19:53:36.635461 1842 consensus_queue.cc:1048] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 40, Last known committed idx: 33, Time since last communication: 0.000s
W20260503 19:53:36.635860 1517 consensus_peers.cc:597] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 -> Peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Couldn't send request to peer f3a96e537cec46df958468878509b9f9. Status: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
F20260503 19:53:36.640928 1613 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 10 received events: NotificationEvent(eventId=1, eventTime=1777838011, eventType=CREATE_TABLE, dbName=default, tableName=table_cb2fc46da15e4d3cbc25e42aac22a26c, message={"server":"","servicePrincipal":"","db":"default","table":"table_cb2fc46da15e4d3cbc25e42aac22a26c","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_cb2fc46da15e4d3cbc25e42aac22a26c\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1777838010},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/warehouse/table_cb2fc46da15e4d3cbc25e42aac22a26c\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_cb2fc46da15e4d3cbc25e42aac22a26c\",\"kudu.table_id\":\"f40076244f874a50b8ee159c9c1b0786\",\"transient_lastDdlTime\":\"1777838010\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"636a1bc8fd6746489a9e892cd3f8ca43\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1777838011,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1777838011, eventType=CREATE_TABLE, dbName=default, tableName=table_f423a73d5c224651bb5eb676ee98c5e3, message={"server":"","servicePrincipal":"","db":"default","table":"table_f423a73d5c224651bb5eb676ee98c5e3","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_f423a73d5c224651bb5eb676ee98c5e3\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1777838010},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/warehouse/table_f423a73d5c224651bb5eb676ee98c5e3\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_f423a73d5c224651bb5eb676ee98c5e3\",\"kudu.table_id\":\"f8aedcc7a5c34071a86b38b46361455f\",\"transient_lastDdlTime\":\"1777838010\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"636a1bc8fd6746489a9e892cd3f8ca43\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1777838011,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
@ 0x7f3c9926bdcd google::LogMessage::Fail() at ??:0
@ 0x7f3c9926fb93 google::LogMessage::SendToLog() at ??:0
@ 0x7f3c9926b7cc google::LogMessage::Flush() at ??:0
@ 0x7f3c9926ba11 google::LogMessage::~LogMessage() at ??:0
@ 0x7f3cafd44574 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f3cafd42e21 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f3cafd4aa9c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f3cafd4a919 std::_Function_handler<>::_M_invoke() at ??:0
I20260503 19:53:36.680769 1850 mvcc.cc:204] Tried to move back new op lower bound from 7282024503891705856 to 7282024503423799296. Current Snapshot: MvccSnapshot[applied={T|T < 7282024503891705856}]
@ 0xaa4cad std::function<>::operator()() at ??:?
I20260503 19:53:36.687405 1817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0cad11e600c48a3af9a3c56115696d5. Latest consensus state: current_term: 4 leader_uuid: "b0cad11e600c48a3af9a3c56115696d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:36.687355 1853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "b0cad11e600c48a3af9a3c56115696d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:36.687685 1817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: This master's current role is: FOLLOWER
I20260503 19:53:36.687716 1853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: This master's current role is: FOLLOWER
@ 0x7f3c9c004529 kudu::Thread::SuperviseThread() at ??:0
I20260503 19:53:36.690201 1853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "b0cad11e600c48a3af9a3c56115696d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:36.690456 1853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: This master's current role is: FOLLOWER
@ 0x7f3c9f5f26db start_thread at ??:0
@ 0x7f3c9811c61f clone at ??:0
W20260503 19:53:36.889463 1404 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.105.60:42203)
W20260503 19:53:36.889770 1404 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.105.60:42203)
W20260503 19:53:36.889958 1404 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.105.60:42203)
W20260503 19:53:36.890663 1404 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.105.60:42203)
I20260503 19:53:37.189425 1853 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b0cad11e600c48a3af9a3c56115696d5)
I20260503 19:53:37.189651 1853 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:37.191460 1853 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:53:37.192369 1762 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:53:37.193876 1762 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:53:37.194149 1762 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:37.194303 1762 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:53:37.194721 1853 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260503 19:53:37.876394 1880 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b0cad11e600c48a3af9a3c56115696d5)
I20260503 19:53:37.876572 1880 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:37.877305 1880 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:53:37.879369 1762 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:53:37.879585 1762 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:37.879679 1762 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:53:37.879899 1880 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260503 19:53:38.664276 1891 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b0cad11e600c48a3af9a3c56115696d5)
I20260503 19:53:38.664427 1891 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:38.664932 1891 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:53:38.666898 1762 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:53:38.667135 1762 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:38.667258 1762 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:53:38.667552 1891 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260503 19:53:39.351732 1898 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b0cad11e600c48a3af9a3c56115696d5)
I20260503 19:53:39.351970 1898 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:39.352768 1898 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:53:39.356113 1762 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:53:39.356304 1762 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:39.356405 1762 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:53:39.356643 1898 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260503 19:53:39.696813 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-task12vCql/build/asan/bin/kudu
/tmp/dist-test-task12vCql/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.62:33821
--webserver_interface=127.0.105.62
--webserver_port=42361
--builtin_ntp_servers=127.0.105.20:36687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203
--hive_metastore_uris=thrift://127.0.0.1:38395
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260503 19:53:40.057927 1907 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260503 19:53:40.058324 1907 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260503 19:53:40.058463 1907 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260503 19:53:40.067508 1907 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260503 19:53:40.067642 1907 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260503 19:53:40.067763 1907 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260503 19:53:40.067844 1907 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260503 19:53:40.078397 1907 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.105.20:36687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:38395
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.62:33821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.105.62
--webserver_port=42361
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 0ba7f54e15c9f673bf3e350f30b8d5f1a4037b79
build type FASTDEBUG
built by None at 03 May 2026 19:43:25 UTC on bdcb31816ec0
build id 11730
ASAN enabled
I20260503 19:53:40.080302 1907 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260503 19:53:40.082386 1907 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260503 19:53:40.092554 1917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260503 19:53:40.092772 1918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260503 19:53:40.094825 1920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260503 19:53:40.094883 1907 server_base.cc:1061] running on GCE node
I20260503 19:53:40.095824 1907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260503 19:53:40.097411 1907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260503 19:53:40.098743 1907 hybrid_clock.cc:648] HybridClock initialized: now 1777838020098667 us; error 63 us; skew 500 ppm
I20260503 19:53:40.099133 1907 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260503 19:53:40.101514 1907 webserver.cc:492] Webserver started at http://127.0.105.62:42361/ using document root <none> and password file <none>
I20260503 19:53:40.102375 1907 fs_manager.cc:362] Metadata directory not provided
I20260503 19:53:40.102528 1907 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260503 19:53:40.109045 1907 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.004s
I20260503 19:53:40.112355 1926 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260503 19:53:40.113948 1907 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260503 19:53:40.114151 1907 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-0/data,/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-0/wal
uuid: "f3a96e537cec46df958468878509b9f9"
format_stamp: "Formatted at 2026-05-03 19:53:26 on dist-test-slave-gl77"
I20260503 19:53:40.114919 1907 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260503 19:53:40.137254 1907 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260503 19:53:40.138159 1907 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260503 19:53:40.138486 1907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260503 19:53:40.160953 1907 rpc_server.cc:307] RPC server started. Bound to: 127.0.105.62:33821
I20260503 19:53:40.160992 1977 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.105.62:33821 every 8 connection(s)
I20260503 19:53:40.162788 1907 server_base.cc:1193] Dumped server information to /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-0/data/info.pb
I20260503 19:53:40.166654 420 external_mini_cluster.cc:1428] Started /tmp/dist-test-task12vCql/build/asan/bin/kudu as pid 1907
I20260503 19:53:40.170413 1978 sys_catalog.cc:263] Verifying existing consensus state
I20260503 19:53:40.174396 1978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9: Bootstrap starting.
I20260503 19:53:40.187436 1978 log.cc:826] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9: Log is configured to *not* fsync() on all Append() calls
I20260503 19:53:40.219623 1978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=39 overwritten=0 applied=39 ignored=0} inserts{seen=28 ignored=0} mutations{seen=24 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260503 19:53:40.220329 1978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9: Bootstrap complete.
I20260503 19:53:40.226317 1978 raft_consensus.cc:359] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:40.227082 1978 raft_consensus.cc:740] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3a96e537cec46df958468878509b9f9, State: Initialized, Role: FOLLOWER
I20260503 19:53:40.227730 1978 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 39, Last appended: 3.39, Last appended by leader: 39, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:40.228576 1982 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:40.228901 1982 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260503 19:53:40.229956 1978 sys_catalog.cc:565] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260503 19:53:40.246418 1996 catalog_manager.cc:1269] Loaded cluster ID: 636a1bc8fd6746489a9e892cd3f8ca43
I20260503 19:53:40.246690 1996 catalog_manager.cc:1562] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9: loading cluster ID for follower catalog manager: success
I20260503 19:53:40.251912 1996 catalog_manager.cc:1584] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9: acquiring CA information for follower catalog manager: success
I20260503 19:53:40.255308 1996 catalog_manager.cc:1612] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260503 19:53:40.275699 1907 master_runner.cc:186] Error getting master registration for 127.0.105.60:42203: Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111),
I20260503 19:53:40.279469 1907 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:53:40.471925 1257 heartbeater.cc:344] Connected to a master server at 127.0.105.62:33821
I20260503 19:53:40.475853 1943 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "fc902de5b2274103ab5e76f967503afb" instance_seqno: 1777838009464378) as {username='slave'} at 127.0.105.2:33873; Asking this server to re-register.
I20260503 19:53:40.476729 1257 heartbeater.cc:461] Registering TS with master...
I20260503 19:53:40.478700 1943 ts_manager.cc:194] Registered new tserver with Master: fc902de5b2274103ab5e76f967503afb (127.0.105.2:46339)
I20260503 19:53:40.506424 1395 heartbeater.cc:344] Connected to a master server at 127.0.105.62:33821
I20260503 19:53:40.508351 1943 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" instance_seqno: 1777838010067456) as {username='slave'} at 127.0.105.3:51603; Asking this server to re-register.
I20260503 19:53:40.509032 1395 heartbeater.cc:461] Registering TS with master...
I20260503 19:53:40.510565 1943 ts_manager.cc:194] Registered new tserver with Master: b68b1efd1cc44e15b8c69e730229dbdd (127.0.105.3:41575)
I20260503 19:53:40.517313 1116 heartbeater.cc:344] Connected to a master server at 127.0.105.62:33821
I20260503 19:53:40.519246 1943 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "b8594a7337c54dc58d701259120d361b" instance_seqno: 1777838008819588) as {username='slave'} at 127.0.105.1:44769; Asking this server to re-register.
I20260503 19:53:40.520061 1116 heartbeater.cc:461] Registering TS with master...
I20260503 19:53:40.521677 1943 ts_manager.cc:194] Registered new tserver with Master: b8594a7337c54dc58d701259120d361b (127.0.105.1:40289)
I20260503 19:53:40.545966 2009 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b0cad11e600c48a3af9a3c56115696d5)
I20260503 19:53:40.546402 2009 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:40.547197 2009 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:53:40.550803 1762 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:40.556250 1953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" candidate_term: 5 candidate_status { last_received { term: 4 index: 43 } } ignore_live_leader: false dest_uuid: "f3a96e537cec46df958468878509b9f9" is_pre_election: true
I20260503 19:53:40.556911 1953 raft_consensus.cc:2468] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ba485a545f44fdd8217c3f86c79d0ca in term 3.
I20260503 19:53:40.557741 1762 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca, f3a96e537cec46df958468878509b9f9; no voters: b0cad11e600c48a3af9a3c56115696d5
I20260503 19:53:40.558104 2009 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260503 19:53:40.558243 2009 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 4 FOLLOWER]: Starting leader election (detected failure of leader b0cad11e600c48a3af9a3c56115696d5)
I20260503 19:53:40.558337 2009 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 4 FOLLOWER]: Advancing to term 5
I20260503 19:53:40.561388 2009 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:40.562147 2009 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 5 election: Requested vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
I20260503 19:53:40.562680 1953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" candidate_term: 5 candidate_status { last_received { term: 4 index: 43 } } ignore_live_leader: false dest_uuid: "f3a96e537cec46df958468878509b9f9"
I20260503 19:53:40.562901 1953 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 3 FOLLOWER]: Advancing to term 5
W20260503 19:53:40.563652 1762 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:40.568691 1953 raft_consensus.cc:2468] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba485a545f44fdd8217c3f86c79d0ca in term 5.
I20260503 19:53:40.569276 1762 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca, f3a96e537cec46df958468878509b9f9; no voters: b0cad11e600c48a3af9a3c56115696d5
I20260503 19:53:40.569665 2009 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 5 FOLLOWER]: Leader election won for term 5
I20260503 19:53:40.570255 2009 raft_consensus.cc:697] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 5 LEADER]: Becoming Leader. State: Replica: 2ba485a545f44fdd8217c3f86c79d0ca, State: Running, Role: LEADER
I20260503 19:53:40.570842 2009 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 43, Committed index: 43, Last appended: 4.43, Last appended by leader: 43, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:40.572842 2013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ba485a545f44fdd8217c3f86c79d0ca. Latest consensus state: current_term: 5 leader_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:40.573235 2013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: This master's current role is: LEADER
I20260503 19:53:40.574141 2015 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260503 19:53:40.576948 2015 catalog_manager.cc:679] Loaded metadata for table default.table_15b3325c8812438786ade0fa5c759e21 [id=04d62d016fc24931ae6f439b1c6152c5]
I20260503 19:53:40.577535 2015 catalog_manager.cc:679] Loaded metadata for table default.table_f91787a2979849d78ba6ba4daf0b4e70 [id=40f19f1e7eaa4640b8c54d5e9e01c320]
I20260503 19:53:40.577967 2015 catalog_manager.cc:679] Loaded metadata for table default.table_0fc3df0b61324d7eac247468c05ce87f [id=54840346b61f4174a481e91d1f745c71]
I20260503 19:53:40.578359 2015 catalog_manager.cc:679] Loaded metadata for table default.table_9c30ca5293344be3964460b578f9d6df [id=8d8f5993b50b4a59aaa004b29cf7c6dd]
I20260503 19:53:40.578744 2015 catalog_manager.cc:679] Loaded metadata for table default.table_db1c67410d564682adca12ef0e942ca2 [id=bd45994325af4ed9ba34633dea7e140e]
I20260503 19:53:40.579128 2015 catalog_manager.cc:679] Loaded metadata for table default.table_dafc7a171f954df18eee7c2911f793f1 [id=da8af64ba75b4650858bb82728b61cd5]
I20260503 19:53:40.579493 2015 catalog_manager.cc:679] Loaded metadata for table default.table_1148c991089047a88d36862e99224cac [id=dce4d9d47ba04a3187de97305003ba5d]
I20260503 19:53:40.579878 2015 catalog_manager.cc:679] Loaded metadata for table default.table_f423a73d5c224651bb5eb676ee98c5e3 [id=f8aedcc7a5c34071a86b38b46361455f]
I20260503 19:53:40.580247 2015 catalog_manager.cc:679] Loaded metadata for table default.table_11964e32037841689422d2a453fff70f [id=f9d69e676109494f8ee9b41f557f4201]
I20260503 19:53:40.582571 2015 tablet_loader.cc:96] loaded metadata for tablet 3ef45a9e31414d209ed6021a0c3f2a04 (table default.table_9c30ca5293344be3964460b578f9d6df [id=8d8f5993b50b4a59aaa004b29cf7c6dd])
I20260503 19:53:40.583112 2015 tablet_loader.cc:96] loaded metadata for tablet 542ff627b557426ab448a2339ef1c665 (table default.table_0fc3df0b61324d7eac247468c05ce87f [id=54840346b61f4174a481e91d1f745c71])
I20260503 19:53:40.583473 2015 tablet_loader.cc:96] loaded metadata for tablet 61730fd814344d6f85f8411bda525df0 (table default.table_db1c67410d564682adca12ef0e942ca2 [id=bd45994325af4ed9ba34633dea7e140e])
I20260503 19:53:40.583961 2015 tablet_loader.cc:96] loaded metadata for tablet 6600b15c33ca480292d1c6eb3002eb86 (table default.table_1148c991089047a88d36862e99224cac [id=dce4d9d47ba04a3187de97305003ba5d])
I20260503 19:53:40.584300 2015 tablet_loader.cc:96] loaded metadata for tablet 6726412cbfd947d2849ac9774d4e86f4 (table default.table_15b3325c8812438786ade0fa5c759e21 [id=04d62d016fc24931ae6f439b1c6152c5])
I20260503 19:53:40.584933 2015 tablet_loader.cc:96] loaded metadata for tablet 7f7923ecc9354b73bb202b969d51e8fa (table default.table_f91787a2979849d78ba6ba4daf0b4e70 [id=40f19f1e7eaa4640b8c54d5e9e01c320])
I20260503 19:53:40.585639 2015 tablet_loader.cc:96] loaded metadata for tablet ac7d7fffebf5463abd8484bffa42bdc1 (table default.table_f423a73d5c224651bb5eb676ee98c5e3 [id=f8aedcc7a5c34071a86b38b46361455f])
I20260503 19:53:40.586301 2015 tablet_loader.cc:96] loaded metadata for tablet ed5861fcf94c47199eb1a81d4702f19b (table default.table_dafc7a171f954df18eee7c2911f793f1 [id=da8af64ba75b4650858bb82728b61cd5])
I20260503 19:53:40.586791 2015 tablet_loader.cc:96] loaded metadata for tablet f8fb50ad87bf46dcbefe765ae02b017f (table default.table_11964e32037841689422d2a453fff70f [id=f9d69e676109494f8ee9b41f557f4201])
I20260503 19:53:40.587085 2015 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260503 19:53:40.588224 2015 catalog_manager.cc:1269] Loaded cluster ID: 636a1bc8fd6746489a9e892cd3f8ca43
I20260503 19:53:40.588343 2015 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260503 19:53:40.589761 2015 catalog_manager.cc:1514] Loading token signing keys...
I20260503 19:53:40.590735 2015 catalog_manager.cc:6055] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: Loaded TSK: 0
I20260503 19:53:40.591653 2015 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260503 19:53:40.592599 2015 catalog_manager.cc:1537] Loading latest processed Hive Metastore notification log event ID...
I20260503 19:53:40.593575 2015 catalog_manager.cc:5893] Last processed Hive Metastore notification event ID: -1
I20260503 19:53:40.664177 1778 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:54712:
table { table_name: "default.Table_1148c991089047a88d36862e99224cac" } new_table_name: "default.Table_a0d72b6ac2cf408296e54ae1978d1ff2" modify_external_catalogs: true
I20260503 19:53:40.664088 1777 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54712:
name: "default.Table_49084d26aae440ac958d415c2b291948"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:40.664723 1773 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54712:
name: "default.Table_fa0eb6f31c344d07a5532aced2019227"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:40.664953 1770 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:54712:
table { table_name: "default.Table_dafc7a171f954df18eee7c2911f793f1" } new_table_name: "default.Table_d2f4bbb38f1d4d0191643de99eeb35b6" modify_external_catalogs: true
I20260503 19:53:40.665053 1776 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54712:
table { table_name: "default.Table_f91787a2979849d78ba6ba4daf0b4e70" } modify_external_catalogs: true
I20260503 19:53:40.665359 1776 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54712:
table { table_name: "default.Table_f91787a2979849d78ba6ba4daf0b4e70" } modify_external_catalogs: true
I20260503 19:53:40.665591 1775 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54712:
name: "default.Table_f28cafcdb9394779ac1e1d270bab59fb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:40.666936 1769 master_service.cc:954] ReplaceTablet: received request to replace tablet f391e07ff0e24bc5b3cf7e81342161fe from {username='slave'} at 127.0.0.1:54712
I20260503 19:53:40.666837 1772 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54712:
name: "default.Table_ba1a53aa8c174babb4bc2b60b558a8c1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:40.667011 1771 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54712:
name: "default.Table_3f2252b7f4da4d9a9752ecd7ce54019f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:40.667361 1774 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54712:
table { table_name: "default.Table_11964e32037841689422d2a453fff70f" } modify_external_catalogs: true
I20260503 19:53:40.667610 1774 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54712:
table { table_name: "default.Table_11964e32037841689422d2a453fff70f" } modify_external_catalogs: true
I20260503 19:53:40.668036 1769 master_service.cc:954] ReplaceTablet: received request to replace tablet f391e07ff0e24bc5b3cf7e81342161fe from {username='slave'} at 127.0.0.1:54712
I20260503 19:53:40.668516 1769 master_service.cc:954] ReplaceTablet: received request to replace tablet c4ef894c0fdb4414a6799627641a8857 from {username='slave'} at 127.0.0.1:54712
I20260503 19:53:40.668938 1769 master_service.cc:954] ReplaceTablet: received request to replace tablet a6cc88aae841477d990ed433c705a803 from {username='slave'} at 127.0.0.1:54712
I20260503 19:53:40.669386 1769 master_service.cc:954] ReplaceTablet: received request to replace tablet f391e07ff0e24bc5b3cf7e81342161fe from {username='slave'} at 127.0.0.1:54712
I20260503 19:53:40.669898 1769 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54712:
name: "default.Table_b578e2eeb11a4b698a7b51e206e1d015"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260503 19:53:40.673265 1830 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 10 received events: NotificationEvent(eventId=1, eventTime=1777838011, eventType=CREATE_TABLE, dbName=default, tableName=table_cb2fc46da15e4d3cbc25e42aac22a26c, message={"server":"","servicePrincipal":"","db":"default","table":"table_cb2fc46da15e4d3cbc25e42aac22a26c","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_cb2fc46da15e4d3cbc25e42aac22a26c\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1777838010},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/warehouse/table_cb2fc46da15e4d3cbc25e42aac22a26c\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_cb2fc46da15e4d3cbc25e42aac22a26c\",\"kudu.table_id\":\"f40076244f874a50b8ee159c9c1b0786\",\"transient_lastDdlTime\":\"1777838010\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"636a1bc8fd6746489a9e892cd3f8ca43\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1777838011,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1777838011, eventType=CREATE_TABLE, dbName=default, tableName=table_f423a73d5c224651bb5eb676ee98c5e3, message={"server":"","servicePrincipal":"","db":"default","table":"table_f423a73d5c224651bb5eb676ee98c5e3","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_f423a73d5c224651bb5eb676ee98c5e3\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1777838010},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/warehouse/table_f423a73d5c224651bb5eb676ee98c5e3\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_f423a73d5c224651bb5eb676ee98c5e3\",\"kudu.table_id\":\"f8aedcc7a5c34071a86b38b46361455f\",\"transient_lastDdlTime\":\"1777838010\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"636a1bc8fd6746489a9e892cd3f8ca43\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1777838011,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
@ 0x7f7f0ddb3dcd google::LogMessage::Fail() at ??:0
@ 0x7f7f0ddb7b93 google::LogMessage::SendToLog() at ??:0
@ 0x7f7f0ddb37cc google::LogMessage::Flush() at ??:0
@ 0x7f7f0ddb3a11 google::LogMessage::~LogMessage() at ??:0
@ 0x7f7f2488c574 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f7f2488ae21 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f7f24892a9c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f7f24892919 std::_Function_handler<>::_M_invoke() at ??:0
@ 0xaa4cad std::function<>::operator()() at ??:?
@ 0x7f7f10b4c529 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f7f1413a6db start_thread at ??:0
@ 0x7f7f0cc6461f clone at ??:0
W20260503 19:53:40.874486 1404 connection.cc:744] client connection to 127.0.105.61:35345 send error: Network error: sendmsg error: Connection reset by peer (error 104)
W20260503 19:53:40.875174 1404 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.105.61:35345)
W20260503 19:53:40.875638 1404 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.105.61:35345)
W20260503 19:53:40.875881 1404 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.105.61:35345)
W20260503 19:53:40.876271 1404 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.105.61:35345)
I20260503 19:53:41.069590 2029 raft_consensus.cc:493] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:53:41.069844 2029 raft_consensus.cc:515] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:41.071898 2029 leader_election.cc:290] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:53:41.072969 1927 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.61:35345: connect: Connection refused (error 111)
W20260503 19:53:41.074651 1927 leader_election.cc:336] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345): Network error: Client connection negotiation failed: client connection to 127.0.105.61:35345: connect: Connection refused (error 111)
W20260503 19:53:41.074960 1927 leader_election.cc:336] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:41.075282 1927 leader_election.cc:304] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3a96e537cec46df958468878509b9f9; no voters: 2ba485a545f44fdd8217c3f86c79d0ca, b0cad11e600c48a3af9a3c56115696d5
I20260503 19:53:41.075706 2029 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260503 19:53:41.692134 2033 raft_consensus.cc:493] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:53:41.692363 2033 raft_consensus.cc:515] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:41.693027 2033 leader_election.cc:290] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:53:41.695611 1927 leader_election.cc:336] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345): Network error: Client connection negotiation failed: client connection to 127.0.105.61:35345: connect: Connection refused (error 111)
W20260503 19:53:41.695830 1927 leader_election.cc:336] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:41.695981 1927 leader_election.cc:304] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3a96e537cec46df958468878509b9f9; no voters: 2ba485a545f44fdd8217c3f86c79d0ca, b0cad11e600c48a3af9a3c56115696d5
I20260503 19:53:41.696281 2033 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260503 19:53:42.352859 2038 raft_consensus.cc:493] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:53:42.353091 2038 raft_consensus.cc:515] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:42.353881 2038 leader_election.cc:290] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:53:42.358318 1927 leader_election.cc:336] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345): Network error: Client connection negotiation failed: client connection to 127.0.105.61:35345: connect: Connection refused (error 111)
W20260503 19:53:42.358582 1927 leader_election.cc:336] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:42.358683 1927 leader_election.cc:304] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3a96e537cec46df958468878509b9f9; no voters: 2ba485a545f44fdd8217c3f86c79d0ca, b0cad11e600c48a3af9a3c56115696d5
I20260503 19:53:42.359010 2038 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260503 19:53:42.935060 2042 raft_consensus.cc:493] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:53:42.935271 2042 raft_consensus.cc:515] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:42.936038 2042 leader_election.cc:290] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:53:42.938359 1927 leader_election.cc:336] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345): Network error: Client connection negotiation failed: client connection to 127.0.105.61:35345: connect: Connection refused (error 111)
W20260503 19:53:42.938530 1927 leader_election.cc:336] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:42.938614 1927 leader_election.cc:304] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3a96e537cec46df958468878509b9f9; no voters: 2ba485a545f44fdd8217c3f86c79d0ca, b0cad11e600c48a3af9a3c56115696d5
I20260503 19:53:42.938856 2042 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260503 19:53:43.502411 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-task12vCql/build/asan/bin/kudu
/tmp/dist-test-task12vCql/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.61:35345
--webserver_interface=127.0.105.61
--webserver_port=39263
--builtin_ntp_servers=127.0.105.20:36687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203
--hive_metastore_uris=thrift://127.0.0.1:38395
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260503 19:53:43.854426 2047 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260503 19:53:43.854790 2047 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260503 19:53:43.854910 2047 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260503 19:53:43.863709 2047 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260503 19:53:43.863814 2047 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260503 19:53:43.863917 2047 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260503 19:53:43.863987 2047 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260503 19:53:43.874560 2047 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.105.20:36687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:38395
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.61:35345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.105.61
--webserver_port=39263
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 0ba7f54e15c9f673bf3e350f30b8d5f1a4037b79
build type FASTDEBUG
built by None at 03 May 2026 19:43:25 UTC on bdcb31816ec0
build id 11730
ASAN enabled
I20260503 19:53:43.876505 2047 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260503 19:53:43.878540 2047 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260503 19:53:43.887019 2052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260503 19:53:43.887112 2053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260503 19:53:43.887785 2055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260503 19:53:43.888411 2047 server_base.cc:1061] running on GCE node
I20260503 19:53:43.889228 2047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260503 19:53:43.890838 2047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260503 19:53:43.892052 2047 hybrid_clock.cc:648] HybridClock initialized: now 1777838023892016 us; error 40 us; skew 500 ppm
I20260503 19:53:43.892437 2047 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260503 19:53:43.895013 2047 webserver.cc:492] Webserver started at http://127.0.105.61:39263/ using document root <none> and password file <none>
I20260503 19:53:43.895994 2047 fs_manager.cc:362] Metadata directory not provided
I20260503 19:53:43.896152 2047 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260503 19:53:43.902258 2047 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260503 19:53:43.905692 2061 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260503 19:53:43.907229 2047 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260503 19:53:43.907428 2047 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data,/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/wal
uuid: "2ba485a545f44fdd8217c3f86c79d0ca"
format_stamp: "Formatted at 2026-05-03 19:53:27 on dist-test-slave-gl77"
I20260503 19:53:43.908269 2047 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260503 19:53:43.925985 2066 raft_consensus.cc:493] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:53:43.926174 2066 raft_consensus.cc:515] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:43.926832 2066 leader_election.cc:290] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:53:43.928898 1927 leader_election.cc:336] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345): Network error: Client connection negotiation failed: client connection to 127.0.105.61:35345: connect: Connection refused (error 111)
W20260503 19:53:43.929167 1927 leader_election.cc:336] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:43.929256 1927 leader_election.cc:304] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3a96e537cec46df958468878509b9f9; no voters: 2ba485a545f44fdd8217c3f86c79d0ca, b0cad11e600c48a3af9a3c56115696d5
I20260503 19:53:43.929525 2066 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260503 19:53:43.935612 2047 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260503 19:53:43.936515 2047 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260503 19:53:43.936803 2047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260503 19:53:43.959255 2047 rpc_server.cc:307] RPC server started. Bound to: 127.0.105.61:35345
I20260503 19:53:43.959278 2115 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.105.61:35345 every 8 connection(s)
I20260503 19:53:43.960942 2047 server_base.cc:1193] Dumped server information to /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data/info.pb
I20260503 19:53:43.962440 420 external_mini_cluster.cc:1428] Started /tmp/dist-test-task12vCql/build/asan/bin/kudu as pid 2047
I20260503 19:53:43.968751 2116 sys_catalog.cc:263] Verifying existing consensus state
I20260503 19:53:43.972405 2116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: Bootstrap starting.
I20260503 19:53:43.987288 2116 log.cc:826] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: Log is configured to *not* fsync() on all Append() calls
I20260503 19:53:44.006736 1261 heartbeater.cc:344] Connected to a master server at 127.0.105.61:35345
I20260503 19:53:44.023350 1396 heartbeater.cc:344] Connected to a master server at 127.0.105.61:35345
I20260503 19:53:44.029812 1117 heartbeater.cc:344] Connected to a master server at 127.0.105.61:35345
I20260503 19:53:44.033357 2116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=44 overwritten=0 applied=43 ignored=0} inserts{seen=31 ignored=0} mutations{seen=27 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260503 19:53:44.034032 2116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: Bootstrap complete.
I20260503 19:53:44.039649 2116 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 5 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:44.040580 2116 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 5 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba485a545f44fdd8217c3f86c79d0ca, State: Initialized, Role: FOLLOWER
I20260503 19:53:44.041312 2116 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 43, Last appended: 5.44, Last appended by leader: 44, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:44.042160 2121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 5 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:44.042418 2121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: This master's current role is: FOLLOWER
I20260503 19:53:44.043262 2116 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: configured and running, proceeding with master startup.
I20260503 19:53:44.068820 2135 catalog_manager.cc:1269] Loaded cluster ID: 636a1bc8fd6746489a9e892cd3f8ca43
I20260503 19:53:44.069046 2135 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: loading cluster ID for follower catalog manager: success
I20260503 19:53:44.073781 2135 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: acquiring CA information for follower catalog manager: success
I20260503 19:53:44.077286 2135 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260503 19:53:44.095403 2047 master_runner.cc:186] Error getting master registration for 127.0.105.60:42203: Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111),
I20260503 19:53:44.099304 2047 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:53:44.420058 2121 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:53:44.420368 2121 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:44.423091 2121 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
I20260503 19:53:44.423686 1953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" candidate_term: 6 candidate_status { last_received { term: 5 index: 44 } } ignore_live_leader: false dest_uuid: "f3a96e537cec46df958468878509b9f9" is_pre_election: true
W20260503 19:53:44.423702 2062 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:44.423940 1953 raft_consensus.cc:2468] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ba485a545f44fdd8217c3f86c79d0ca in term 5.
I20260503 19:53:44.424513 2062 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca, f3a96e537cec46df958468878509b9f9; no voters:
I20260503 19:53:44.424876 2121 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260503 19:53:44.425028 2121 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260503 19:53:44.425169 2121 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 5 FOLLOWER]: Advancing to term 6
W20260503 19:53:44.425426 2062 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:44.428979 2121 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:44.429764 2121 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 6 election: Requested vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
I20260503 19:53:44.430372 1953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" candidate_term: 6 candidate_status { last_received { term: 5 index: 44 } } ignore_live_leader: false dest_uuid: "f3a96e537cec46df958468878509b9f9"
I20260503 19:53:44.430624 1953 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 5 FOLLOWER]: Advancing to term 6
W20260503 19:53:44.431411 2062 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:44.432873 1953 raft_consensus.cc:2468] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba485a545f44fdd8217c3f86c79d0ca in term 6.
I20260503 19:53:44.433359 2062 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca, f3a96e537cec46df958468878509b9f9; no voters: b0cad11e600c48a3af9a3c56115696d5
I20260503 19:53:44.433669 2121 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 6 FOLLOWER]: Leader election won for term 6
I20260503 19:53:44.434209 2121 raft_consensus.cc:697] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 6 LEADER]: Becoming Leader. State: Replica: 2ba485a545f44fdd8217c3f86c79d0ca, State: Running, Role: LEADER
I20260503 19:53:44.434789 2121 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 43, Committed index: 43, Last appended: 5.44, Last appended by leader: 44, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:44.437716 2142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ba485a545f44fdd8217c3f86c79d0ca. Latest consensus state: current_term: 6 leader_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:44.438020 2142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: This master's current role is: LEADER
I20260503 19:53:44.438732 2144 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260503 19:53:44.441334 2144 catalog_manager.cc:679] Loaded metadata for table default.table_15b3325c8812438786ade0fa5c759e21 [id=04d62d016fc24931ae6f439b1c6152c5]
I20260503 19:53:44.441807 2144 catalog_manager.cc:679] Loaded metadata for table default.table_f91787a2979849d78ba6ba4daf0b4e70 [id=40f19f1e7eaa4640b8c54d5e9e01c320]
I20260503 19:53:44.442188 2144 catalog_manager.cc:679] Loaded metadata for table default.table_0fc3df0b61324d7eac247468c05ce87f [id=54840346b61f4174a481e91d1f745c71]
I20260503 19:53:44.442571 2144 catalog_manager.cc:679] Loaded metadata for table default.table_9c30ca5293344be3964460b578f9d6df [id=8d8f5993b50b4a59aaa004b29cf7c6dd]
I20260503 19:53:44.442907 2144 catalog_manager.cc:679] Loaded metadata for table default.table_db1c67410d564682adca12ef0e942ca2 [id=bd45994325af4ed9ba34633dea7e140e]
I20260503 19:53:44.443343 2144 catalog_manager.cc:679] Loaded metadata for table default.table_dafc7a171f954df18eee7c2911f793f1 [id=da8af64ba75b4650858bb82728b61cd5]
I20260503 19:53:44.443749 2144 catalog_manager.cc:679] Loaded metadata for table default.table_1148c991089047a88d36862e99224cac [id=dce4d9d47ba04a3187de97305003ba5d]
I20260503 19:53:44.444209 2144 catalog_manager.cc:679] Loaded metadata for table default.table_f423a73d5c224651bb5eb676ee98c5e3 [id=f8aedcc7a5c34071a86b38b46361455f]
I20260503 19:53:44.444566 2144 catalog_manager.cc:679] Loaded metadata for table default.table_11964e32037841689422d2a453fff70f [id=f9d69e676109494f8ee9b41f557f4201]
I20260503 19:53:44.446797 2144 tablet_loader.cc:96] loaded metadata for tablet 3ef45a9e31414d209ed6021a0c3f2a04 (table default.table_9c30ca5293344be3964460b578f9d6df [id=8d8f5993b50b4a59aaa004b29cf7c6dd])
I20260503 19:53:44.447283 2144 tablet_loader.cc:96] loaded metadata for tablet 542ff627b557426ab448a2339ef1c665 (table default.table_0fc3df0b61324d7eac247468c05ce87f [id=54840346b61f4174a481e91d1f745c71])
I20260503 19:53:44.447592 2144 tablet_loader.cc:96] loaded metadata for tablet 61730fd814344d6f85f8411bda525df0 (table default.table_db1c67410d564682adca12ef0e942ca2 [id=bd45994325af4ed9ba34633dea7e140e])
I20260503 19:53:44.448009 2144 tablet_loader.cc:96] loaded metadata for tablet 6600b15c33ca480292d1c6eb3002eb86 (table default.table_1148c991089047a88d36862e99224cac [id=dce4d9d47ba04a3187de97305003ba5d])
I20260503 19:53:44.448323 2144 tablet_loader.cc:96] loaded metadata for tablet 6726412cbfd947d2849ac9774d4e86f4 (table default.table_15b3325c8812438786ade0fa5c759e21 [id=04d62d016fc24931ae6f439b1c6152c5])
I20260503 19:53:44.448863 2144 tablet_loader.cc:96] loaded metadata for tablet 7f7923ecc9354b73bb202b969d51e8fa (table default.table_f91787a2979849d78ba6ba4daf0b4e70 [id=40f19f1e7eaa4640b8c54d5e9e01c320])
I20260503 19:53:44.449503 2144 tablet_loader.cc:96] loaded metadata for tablet ac7d7fffebf5463abd8484bffa42bdc1 (table default.table_f423a73d5c224651bb5eb676ee98c5e3 [id=f8aedcc7a5c34071a86b38b46361455f])
I20260503 19:53:44.450073 2144 tablet_loader.cc:96] loaded metadata for tablet ed5861fcf94c47199eb1a81d4702f19b (table default.table_dafc7a171f954df18eee7c2911f793f1 [id=da8af64ba75b4650858bb82728b61cd5])
I20260503 19:53:44.450505 2144 tablet_loader.cc:96] loaded metadata for tablet f8fb50ad87bf46dcbefe765ae02b017f (table default.table_11964e32037841689422d2a453fff70f [id=f9d69e676109494f8ee9b41f557f4201])
I20260503 19:53:44.450754 2144 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260503 19:53:44.451707 2144 catalog_manager.cc:1269] Loaded cluster ID: 636a1bc8fd6746489a9e892cd3f8ca43
I20260503 19:53:44.451830 2144 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260503 19:53:44.453140 2144 catalog_manager.cc:1514] Loading token signing keys...
I20260503 19:53:44.454000 2144 catalog_manager.cc:6055] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: Loaded TSK: 0
I20260503 19:53:44.454870 2144 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260503 19:53:44.455947 2144 catalog_manager.cc:1537] Loading latest processed Hive Metastore notification log event ID...
I20260503 19:53:44.457163 2144 catalog_manager.cc:5893] Last processed Hive Metastore notification event ID: -1
I20260503 19:53:44.511292 2081 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "fc902de5b2274103ab5e76f967503afb" instance_seqno: 1777838009464378) as {username='slave'} at 127.0.105.2:43419; Asking this server to re-register.
I20260503 19:53:44.512048 1261 heartbeater.cc:461] Registering TS with master...
I20260503 19:53:44.512375 1261 heartbeater.cc:499] Master 127.0.105.61:35345 was elected leader, sending a full tablet report...
I20260503 19:53:44.513007 2080 master_service.cc:954] ReplaceTablet: received request to replace tablet f391e07ff0e24bc5b3cf7e81342161fe from {username='slave'} at 127.0.0.1:58818
I20260503 19:53:44.514406 2073 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58818:
name: "default.Table_8a28426e04cf42b8ae2288a7c2950b1b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:44.514362 2074 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58818:
name: "default.Table_1eff9f94b0c84e7180c54349d66db8e4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:44.515447 2072 ts_manager.cc:194] Registered new tserver with Master: fc902de5b2274103ab5e76f967503afb (127.0.105.2:46339)
I20260503 19:53:44.515584 2075 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58818:
table { table_name: "default.Table_15b3325c8812438786ade0fa5c759e21" } modify_external_catalogs: true
I20260503 19:53:44.515887 2075 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58818:
table { table_name: "default.Table_15b3325c8812438786ade0fa5c759e21" } modify_external_catalogs: true
I20260503 19:53:44.515914 2080 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58818:
name: "default.Table_e9dee4f0693a4aca81d75d2c25cefd09"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:44.516355 2076 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58818:
table { table_name: "default.Table_db1c67410d564682adca12ef0e942ca2" } modify_external_catalogs: true
I20260503 19:53:44.516582 2076 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58818:
table { table_name: "default.Table_db1c67410d564682adca12ef0e942ca2" } modify_external_catalogs: true
I20260503 19:53:44.516817 2079 master_service.cc:954] ReplaceTablet: received request to replace tablet ed5861fcf94c47199eb1a81d4702f19b from {username='slave'} at 127.0.0.1:58818
I20260503 19:53:44.517275 2078 master_service.cc:954] ReplaceTablet: received request to replace tablet f391e07ff0e24bc5b3cf7e81342161fe from {username='slave'} at 127.0.0.1:58818
I20260503 19:53:44.517378 2077 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58818:
name: "default.Table_ad6e238ff40f4be99d7cc2d556a40a6b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:44.518153 2078 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:58818:
table { table_name: "default.Table_0fc3df0b61324d7eac247468c05ce87f" } new_table_name: "default.Table_dd74e45a321f4e7f92e8b514d7378c86" modify_external_catalogs: true
I20260503 19:53:44.519233 2081 master_service.cc:954] ReplaceTablet: received request to replace tablet ed5861fcf94c47199eb1a81d4702f19b from {username='slave'} at 127.0.0.1:58818
I20260503 19:53:44.524881 1953 raft_consensus.cc:1275] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 6 FOLLOWER]: Refusing update from remote peer 2ba485a545f44fdd8217c3f86c79d0ca: Log matching property violated. Preceding OpId in replica: term: 3 index: 39. Preceding OpId from leader: term: 6 index: 46. (index mismatch)
F20260503 19:53:44.525650 2134 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 10 received events: NotificationEvent(eventId=1, eventTime=1777838011, eventType=CREATE_TABLE, dbName=default, tableName=table_cb2fc46da15e4d3cbc25e42aac22a26c, message={"server":"","servicePrincipal":"","db":"default","table":"table_cb2fc46da15e4d3cbc25e42aac22a26c","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_cb2fc46da15e4d3cbc25e42aac22a26c\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1777838010},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/warehouse/table_cb2fc46da15e4d3cbc25e42aac22a26c\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_cb2fc46da15e4d3cbc25e42aac22a26c\",\"kudu.table_id\":\"f40076244f874a50b8ee159c9c1b0786\",\"transient_lastDdlTime\":\"1777838010\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"636a1bc8fd6746489a9e892cd3f8ca43\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1777838011,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1777838011, eventType=CREATE_TABLE, dbName=default, tableName=table_f423a73d5c224651bb5eb676ee98c5e3, message={"server":"","servicePrincipal":"","db":"default","table":"table_f423a73d5c224651bb5eb676ee98c5e3","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_f423a73d5c224651bb5eb676ee98c5e3\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1777838010},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/warehouse/table_f423a73d5c224651bb5eb676ee98c5e3\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_f423a73d5c224651bb5eb676ee98c5e3\",\"kudu.table_id\":\"f8aedcc7a5c34071a86b38b46361455f\",\"transient_lastDdlTime\":\"1777838010\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"636a1bc8fd6746489a9e892cd3f8ca43\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1777838011,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
W20260503 19:53:44.526154 2062 consensus_peers.cc:597] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca -> Peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Couldn't send request to peer b0cad11e600c48a3af9a3c56115696d5. Status: Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260503 19:53:44.526425 2142 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 45, Last known committed idx: 39, Time since last communication: 0.000s
@ 0x7f3e72f2fdcd google::LogMessage::Fail() at ??:0
@ 0x7f3e72f33b93 google::LogMessage::SendToLog() at ??:0
@ 0x7f3e72f2f7cc google::LogMessage::Flush() at ??:0
@ 0x7f3e72f2fa11 google::LogMessage::~LogMessage() at ??:0
@ 0x7f3e89a08574 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f3e89a06e21 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f3e89a0ea9c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f3e89a0e919 std::_Function_handler<>::_M_invoke() at ??:0
@ 0xaa4cad std::function<>::operator()() at ??:?
@ 0x7f3e75cc8529 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f3e792b66db start_thread at ??:0
@ 0x7f3e71de061f clone at ??:0
I20260503 19:53:44.589568 2147 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ba485a545f44fdd8217c3f86c79d0ca. Latest consensus state: current_term: 6 leader_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:44.589812 2147 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260503 19:53:44.590189 2153 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:44.590474 2153 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [sys.catalog]: This master's current role is: FOLLOWER
W20260503 19:53:44.756321 1927 connection.cc:570] server connection from 127.0.0.1:43944 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260503 19:53:44.756955 1261 heartbeater.cc:412] Failed 1 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260503 19:53:44.757441 1404 connection.cc:570] client connection to 127.0.105.61:35345 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260503 19:53:44.758241 1404 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.105.61:35345)
W20260503 19:53:44.758638 1404 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.105.61:35345)
W20260503 19:53:44.758833 1404 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.105.61:35345)
W20260503 19:53:44.759261 1404 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.105.61:35345)
I20260503 19:53:45.031155 2153 raft_consensus.cc:493] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2ba485a545f44fdd8217c3f86c79d0ca)
I20260503 19:53:45.031422 2153 raft_consensus.cc:515] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:45.032056 2153 leader_election.cc:290] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:53:45.034579 1927 leader_election.cc:336] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345): Network error: Client connection negotiation failed: client connection to 127.0.105.61:35345: connect: Connection refused (error 111)
W20260503 19:53:45.034816 1927 leader_election.cc:336] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:45.034916 1927 leader_election.cc:304] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3a96e537cec46df958468878509b9f9; no voters: 2ba485a545f44fdd8217c3f86c79d0ca, b0cad11e600c48a3af9a3c56115696d5
I20260503 19:53:45.035180 2153 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260503 19:53:45.282620 1907 master_runner.cc:186] Error getting master registration for 127.0.105.61:35345: Network error: Client connection negotiation failed: client connection to 127.0.105.61:35345: connect: Connection refused (error 111),
I20260503 19:53:45.282900 1907 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:53:45.646019 2171 raft_consensus.cc:493] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2ba485a545f44fdd8217c3f86c79d0ca)
I20260503 19:53:45.646268 2171 raft_consensus.cc:515] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:45.646950 2171 leader_election.cc:290] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:53:45.650523 1927 leader_election.cc:336] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345): Network error: Client connection negotiation failed: client connection to 127.0.105.61:35345: connect: Connection refused (error 111)
W20260503 19:53:45.650785 1927 leader_election.cc:336] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:45.650905 1927 leader_election.cc:304] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3a96e537cec46df958468878509b9f9; no voters: 2ba485a545f44fdd8217c3f86c79d0ca, b0cad11e600c48a3af9a3c56115696d5
I20260503 19:53:45.651261 2171 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260503 19:53:46.178268 2174 raft_consensus.cc:493] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2ba485a545f44fdd8217c3f86c79d0ca)
I20260503 19:53:46.178452 2174 raft_consensus.cc:515] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:46.179054 2174 leader_election.cc:290] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:53:46.180167 1927 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.61:35345: connect: Connection refused (error 111) [suppressed 13 similar messages]
W20260503 19:53:46.181550 1927 leader_election.cc:336] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345): Network error: Client connection negotiation failed: client connection to 127.0.105.61:35345: connect: Connection refused (error 111)
W20260503 19:53:46.181763 1927 leader_election.cc:336] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:46.181883 1927 leader_election.cc:304] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3a96e537cec46df958468878509b9f9; no voters: 2ba485a545f44fdd8217c3f86c79d0ca, b0cad11e600c48a3af9a3c56115696d5
I20260503 19:53:46.182214 2174 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260503 19:53:46.702828 2178 raft_consensus.cc:493] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2ba485a545f44fdd8217c3f86c79d0ca)
I20260503 19:53:46.703049 2178 raft_consensus.cc:515] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:46.703797 2178 leader_election.cc:290] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:53:46.706159 1927 leader_election.cc:336] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345): Network error: Client connection negotiation failed: client connection to 127.0.105.61:35345: connect: Connection refused (error 111)
W20260503 19:53:46.706390 1927 leader_election.cc:336] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:46.706475 1927 leader_election.cc:304] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3a96e537cec46df958468878509b9f9; no voters: 2ba485a545f44fdd8217c3f86c79d0ca, b0cad11e600c48a3af9a3c56115696d5
I20260503 19:53:46.706686 2178 raft_consensus.cc:2749] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260503 19:53:46.951028 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-task12vCql/build/asan/bin/kudu
/tmp/dist-test-task12vCql/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.61:35345
--webserver_interface=127.0.105.61
--webserver_port=39263
--builtin_ntp_servers=127.0.105.20:36687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203
--hive_metastore_uris=thrift://127.0.0.1:38395
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260503 19:53:47.298631 2181 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260503 19:53:47.298954 2181 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260503 19:53:47.299126 2181 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260503 19:53:47.309566 2181 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260503 19:53:47.309712 2181 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260503 19:53:47.309886 2181 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260503 19:53:47.309985 2181 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260503 19:53:47.321911 2181 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.105.20:36687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:38395
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.61:35345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.105.61
--webserver_port=39263
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 0ba7f54e15c9f673bf3e350f30b8d5f1a4037b79
build type FASTDEBUG
built by None at 03 May 2026 19:43:25 UTC on bdcb31816ec0
build id 11730
ASAN enabled
I20260503 19:53:47.323635 2181 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260503 19:53:47.325623 2181 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260503 19:53:47.334805 2187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260503 19:53:47.334860 2186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260503 19:53:47.334766 2189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260503 19:53:47.336207 2181 server_base.cc:1061] running on GCE node
I20260503 19:53:47.337212 2181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260503 19:53:47.338738 2181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260503 19:53:47.339978 2181 hybrid_clock.cc:648] HybridClock initialized: now 1777838027339936 us; error 41 us; skew 500 ppm
I20260503 19:53:47.340426 2181 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260503 19:53:47.342854 2181 webserver.cc:492] Webserver started at http://127.0.105.61:39263/ using document root <none> and password file <none>
I20260503 19:53:47.343616 2181 fs_manager.cc:362] Metadata directory not provided
I20260503 19:53:47.343716 2181 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260503 19:53:47.349253 2181 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260503 19:53:47.352447 2195 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260503 19:53:47.354094 2181 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260503 19:53:47.354296 2181 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data,/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/wal
uuid: "2ba485a545f44fdd8217c3f86c79d0ca"
format_stamp: "Formatted at 2026-05-03 19:53:27 on dist-test-slave-gl77"
I20260503 19:53:47.355074 2181 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260503 19:53:47.381417 2181 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260503 19:53:47.382377 2181 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260503 19:53:47.382717 2181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260503 19:53:47.403448 2181 rpc_server.cc:307] RPC server started. Bound to: 127.0.105.61:35345
I20260503 19:53:47.403496 2246 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.105.61:35345 every 8 connection(s)
I20260503 19:53:47.405248 2181 server_base.cc:1193] Dumped server information to /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data/info.pb
I20260503 19:53:47.411056 420 external_mini_cluster.cc:1428] Started /tmp/dist-test-task12vCql/build/asan/bin/kudu as pid 2181
I20260503 19:53:47.412334 2247 sys_catalog.cc:263] Verifying existing consensus state
I20260503 19:53:47.416510 2247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: Bootstrap starting.
I20260503 19:53:47.430135 2247 log.cc:826] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: Log is configured to *not* fsync() on all Append() calls
I20260503 19:53:47.467084 2247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=46 overwritten=0 applied=43 ignored=0} inserts{seen=31 ignored=0} mutations{seen=27 ignored=0} orphaned_commits=0. Pending: 3 replicates
I20260503 19:53:47.467923 2247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: Bootstrap complete.
I20260503 19:53:47.473683 2247 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 6 FOLLOWER]: Replica starting. Triggering 3 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:47.476213 2247 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba485a545f44fdd8217c3f86c79d0ca, State: Initialized, Role: FOLLOWER
I20260503 19:53:47.476958 2247 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 43, Last appended: 6.46, Last appended by leader: 46, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:47.477867 2252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:47.478143 2252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: This master's current role is: FOLLOWER
I20260503 19:53:47.478900 2247 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: configured and running, proceeding with master startup.
I20260503 19:53:47.491226 2266 catalog_manager.cc:1269] Loaded cluster ID: 636a1bc8fd6746489a9e892cd3f8ca43
I20260503 19:53:47.491422 2266 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: loading cluster ID for follower catalog manager: success
I20260503 19:53:47.496560 2266 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: acquiring CA information for follower catalog manager: success
I20260503 19:53:47.500481 2266 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260503 19:53:47.525471 2181 master_runner.cc:186] Error getting master registration for 127.0.105.60:42203: Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111),
I20260503 19:53:47.531941 2181 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:53:47.782042 1396 heartbeater.cc:344] Connected to a master server at 127.0.105.61:35345
I20260503 19:53:47.782291 1261 heartbeater.cc:344] Connected to a master server at 127.0.105.61:35345
I20260503 19:53:47.782444 1261 heartbeater.cc:461] Registering TS with master...
I20260503 19:53:47.782720 1261 heartbeater.cc:499] Master 127.0.105.61:35345 was elected leader, sending a full tablet report...
I20260503 19:53:47.785480 2212 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" instance_seqno: 1777838010067456) as {username='slave'} at 127.0.105.3:39433; Asking this server to re-register.
I20260503 19:53:47.785970 2211 ts_manager.cc:194] Registered new tserver with Master: fc902de5b2274103ab5e76f967503afb (127.0.105.2:46339)
I20260503 19:53:47.786312 1396 heartbeater.cc:461] Registering TS with master...
I20260503 19:53:47.787840 2211 ts_manager.cc:194] Registered new tserver with Master: b68b1efd1cc44e15b8c69e730229dbdd (127.0.105.3:41575)
I20260503 19:53:47.788379 1117 heartbeater.cc:344] Connected to a master server at 127.0.105.61:35345
I20260503 19:53:47.789578 2212 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "b8594a7337c54dc58d701259120d361b" instance_seqno: 1777838008819588) as {username='slave'} at 127.0.105.1:34453; Asking this server to re-register.
I20260503 19:53:47.790092 1117 heartbeater.cc:461] Registering TS with master...
I20260503 19:53:47.791458 2212 ts_manager.cc:194] Registered new tserver with Master: b8594a7337c54dc58d701259120d361b (127.0.105.1:40289)
I20260503 19:53:47.814886 2274 raft_consensus.cc:493] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2ba485a545f44fdd8217c3f86c79d0ca)
I20260503 19:53:47.815201 2274 raft_consensus.cc:515] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:47.816011 2274 leader_election.cc:290] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:53:47.820160 1927 leader_election.cc:336] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:47.824208 2222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f3a96e537cec46df958468878509b9f9" candidate_term: 7 candidate_status { last_received { term: 6 index: 46 } } ignore_live_leader: false dest_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" is_pre_election: true
I20260503 19:53:47.824842 2222 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3a96e537cec46df958468878509b9f9 in term 6.
I20260503 19:53:47.825512 1927 leader_election.cc:304] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca, f3a96e537cec46df958468878509b9f9; no voters: b0cad11e600c48a3af9a3c56115696d5
I20260503 19:53:47.825881 2274 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260503 19:53:47.826009 2274 raft_consensus.cc:493] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 2ba485a545f44fdd8217c3f86c79d0ca)
I20260503 19:53:47.826138 2274 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 6 FOLLOWER]: Advancing to term 7
I20260503 19:53:47.828133 2274 raft_consensus.cc:515] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:47.828804 2274 leader_election.cc:290] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 7 election: Requested vote from peers 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
I20260503 19:53:47.829408 2222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f3a96e537cec46df958468878509b9f9" candidate_term: 7 candidate_status { last_received { term: 6 index: 46 } } ignore_live_leader: false dest_uuid: "2ba485a545f44fdd8217c3f86c79d0ca"
I20260503 19:53:47.829651 2222 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 6 FOLLOWER]: Advancing to term 7
W20260503 19:53:47.830253 1927 leader_election.cc:336] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:47.835320 2222 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3a96e537cec46df958468878509b9f9 in term 7.
I20260503 19:53:47.835876 1927 leader_election.cc:304] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca, f3a96e537cec46df958468878509b9f9; no voters: b0cad11e600c48a3af9a3c56115696d5
I20260503 19:53:47.836328 2274 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 7 FOLLOWER]: Leader election won for term 7
I20260503 19:53:47.836809 2274 raft_consensus.cc:697] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [term 7 LEADER]: Becoming Leader. State: Replica: f3a96e537cec46df958468878509b9f9, State: Running, Role: LEADER
I20260503 19:53:47.837442 2274 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 43, Committed index: 43, Last appended: 6.46, Last appended by leader: 46, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:47.840101 2278 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3a96e537cec46df958468878509b9f9. Latest consensus state: current_term: 7 leader_uuid: "f3a96e537cec46df958468878509b9f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:47.840859 2278 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [sys.catalog]: This master's current role is: LEADER
I20260503 19:53:48.251160 2222 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 7 FOLLOWER]: Refusing update from remote peer f3a96e537cec46df958468878509b9f9: Log matching property violated. Preceding OpId in replica: term: 6 index: 46. Preceding OpId from leader: term: 7 index: 47. (index mismatch)
I20260503 19:53:48.252044 2278 consensus_queue.cc:1048] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 47, Last known committed idx: 43, Time since last communication: 0.000s
I20260503 19:53:48.254602 2281 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3a96e537cec46df958468878509b9f9. Latest consensus state: current_term: 7 leader_uuid: "f3a96e537cec46df958468878509b9f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:48.254853 2281 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: This master's current role is: FOLLOWER
I20260503 19:53:48.258132 2278 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "f3a96e537cec46df958468878509b9f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:48.258356 2278 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [sys.catalog]: This master's current role is: LEADER
I20260503 19:53:48.258596 2278 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 7 leader_uuid: "f3a96e537cec46df958468878509b9f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:48.258821 2278 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [sys.catalog]: This master's current role is: LEADER
I20260503 19:53:48.258692 2284 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "f3a96e537cec46df958468878509b9f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:48.258896 2284 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 [sys.catalog]: This master's current role is: LEADER
W20260503 19:53:48.259907 1927 consensus_peers.cc:597] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9 -> Peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Couldn't send request to peer b0cad11e600c48a3af9a3c56115696d5. Status: Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260503 19:53:48.260305 2290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "f3a96e537cec46df958468878509b9f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:48.260612 2290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: This master's current role is: FOLLOWER
I20260503 19:53:48.260960 2281 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "f3a96e537cec46df958468878509b9f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:48.260936 2287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "f3a96e537cec46df958468878509b9f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:48.261193 2281 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: This master's current role is: FOLLOWER
I20260503 19:53:48.261197 2287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: This master's current role is: FOLLOWER
I20260503 19:53:48.358103 2280 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260503 19:53:48.360924 2280 catalog_manager.cc:679] Loaded metadata for table default.table_15b3325c8812438786ade0fa5c759e21 [id=04d62d016fc24931ae6f439b1c6152c5]
I20260503 19:53:48.361501 2280 catalog_manager.cc:679] Loaded metadata for table default.table_f91787a2979849d78ba6ba4daf0b4e70 [id=40f19f1e7eaa4640b8c54d5e9e01c320]
I20260503 19:53:48.361835 2280 catalog_manager.cc:679] Loaded metadata for table default.table_0fc3df0b61324d7eac247468c05ce87f [id=54840346b61f4174a481e91d1f745c71]
I20260503 19:53:48.362151 2280 catalog_manager.cc:679] Loaded metadata for table default.table_9c30ca5293344be3964460b578f9d6df [id=8d8f5993b50b4a59aaa004b29cf7c6dd]
I20260503 19:53:48.362419 2280 catalog_manager.cc:679] Loaded metadata for table default.table_db1c67410d564682adca12ef0e942ca2 [id=bd45994325af4ed9ba34633dea7e140e]
I20260503 19:53:48.362704 2280 catalog_manager.cc:679] Loaded metadata for table default.table_dafc7a171f954df18eee7c2911f793f1 [id=da8af64ba75b4650858bb82728b61cd5]
I20260503 19:53:48.363011 2280 catalog_manager.cc:679] Loaded metadata for table default.table_1148c991089047a88d36862e99224cac [id=dce4d9d47ba04a3187de97305003ba5d]
I20260503 19:53:48.363304 2280 catalog_manager.cc:679] Loaded metadata for table default.table_f423a73d5c224651bb5eb676ee98c5e3 [id=f8aedcc7a5c34071a86b38b46361455f]
I20260503 19:53:48.363580 2280 catalog_manager.cc:679] Loaded metadata for table default.table_11964e32037841689422d2a453fff70f [id=f9d69e676109494f8ee9b41f557f4201]
I20260503 19:53:48.365464 2280 tablet_loader.cc:96] loaded metadata for tablet 3ef45a9e31414d209ed6021a0c3f2a04 (table default.table_9c30ca5293344be3964460b578f9d6df [id=8d8f5993b50b4a59aaa004b29cf7c6dd])
I20260503 19:53:48.365978 2280 tablet_loader.cc:96] loaded metadata for tablet 542ff627b557426ab448a2339ef1c665 (table default.table_0fc3df0b61324d7eac247468c05ce87f [id=54840346b61f4174a481e91d1f745c71])
I20260503 19:53:48.366250 2280 tablet_loader.cc:96] loaded metadata for tablet 5bd678d9621b4262b8cf0e8fb504e01e (table default.table_dafc7a171f954df18eee7c2911f793f1 [id=da8af64ba75b4650858bb82728b61cd5])
I20260503 19:53:48.366529 2280 tablet_loader.cc:96] loaded metadata for tablet 61730fd814344d6f85f8411bda525df0 (table default.table_db1c67410d564682adca12ef0e942ca2 [id=bd45994325af4ed9ba34633dea7e140e])
I20260503 19:53:48.366961 2280 tablet_loader.cc:96] loaded metadata for tablet 6600b15c33ca480292d1c6eb3002eb86 (table default.table_1148c991089047a88d36862e99224cac [id=dce4d9d47ba04a3187de97305003ba5d])
I20260503 19:53:48.367254 2280 tablet_loader.cc:96] loaded metadata for tablet 6726412cbfd947d2849ac9774d4e86f4 (table default.table_15b3325c8812438786ade0fa5c759e21 [id=04d62d016fc24931ae6f439b1c6152c5])
I20260503 19:53:48.367820 2280 tablet_loader.cc:96] loaded metadata for tablet 7f7923ecc9354b73bb202b969d51e8fa (table default.table_f91787a2979849d78ba6ba4daf0b4e70 [id=40f19f1e7eaa4640b8c54d5e9e01c320])
I20260503 19:53:48.368382 2280 tablet_loader.cc:96] loaded metadata for tablet ac7d7fffebf5463abd8484bffa42bdc1 (table default.table_f423a73d5c224651bb5eb676ee98c5e3 [id=f8aedcc7a5c34071a86b38b46361455f])
I20260503 19:53:48.369338 2280 tablet_loader.cc:96] loaded metadata for tablet f8fb50ad87bf46dcbefe765ae02b017f (table default.table_11964e32037841689422d2a453fff70f [id=f9d69e676109494f8ee9b41f557f4201])
I20260503 19:53:48.369599 2280 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260503 19:53:48.370548 2280 catalog_manager.cc:1269] Loaded cluster ID: 636a1bc8fd6746489a9e892cd3f8ca43
I20260503 19:53:48.370649 2280 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260503 19:53:48.371838 2280 catalog_manager.cc:1514] Loading token signing keys...
I20260503 19:53:48.372674 2280 catalog_manager.cc:6055] T 00000000000000000000000000000000 P f3a96e537cec46df958468878509b9f9: Loaded TSK: 0
I20260503 19:53:48.373675 2280 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260503 19:53:48.374518 2280 catalog_manager.cc:1537] Loading latest processed Hive Metastore notification log event ID...
I20260503 19:53:48.375334 2280 catalog_manager.cc:5893] Last processed Hive Metastore notification event ID: -1
I20260503 19:53:48.418591 1943 master_service.cc:954] ReplaceTablet: received request to replace tablet c4ef894c0fdb4414a6799627641a8857 from {username='slave'} at 127.0.0.1:49092
I20260503 19:53:48.419840 1935 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49092:
table { table_name: "default.Table_1148c991089047a88d36862e99224cac" } new_table_name: "default.Table_a0d72b6ac2cf408296e54ae1978d1ff2" modify_external_catalogs: true
I20260503 19:53:48.420339 1939 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49092:
name: "default.Table_b578e2eeb11a4b698a7b51e206e1d015"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:48.420686 1936 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49092:
name: "default.Table_ba1a53aa8c174babb4bc2b60b558a8c1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:48.421310 1941 master_service.cc:954] ReplaceTablet: received request to replace tablet a6cc88aae841477d990ed433c705a803 from {username='slave'} at 127.0.0.1:49092
I20260503 19:53:48.421018 1934 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49092:
table { table_name: "default.Table_db1c67410d564682adca12ef0e942ca2" } modify_external_catalogs: true
I20260503 19:53:48.421664 1942 master_service.cc:954] ReplaceTablet: received request to replace tablet ed5861fcf94c47199eb1a81d4702f19b from {username='slave'} at 127.0.0.1:49092
I20260503 19:53:48.423247 1943 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49092:
name: "default.Table_49084d26aae440ac958d415c2b291948"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:48.422899 1934 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49092:
table { table_name: "default.Table_db1c67410d564682adca12ef0e942ca2" } modify_external_catalogs: true
I20260503 19:53:48.421893 1940 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49092:
table { table_name: "default.Table_9c30ca5293344be3964460b578f9d6df" } new_table_name: "default.Table_90313d18e9a542119fe8e0e296d4396c" modify_external_catalogs: true
I20260503 19:53:48.419867 1938 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49092:
table { table_name: "default.Table_dafc7a171f954df18eee7c2911f793f1" } new_table_name: "default.Table_d2f4bbb38f1d4d0191643de99eeb35b6" modify_external_catalogs: true
I20260503 19:53:48.422668 1937 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49092:
name: "default.Table_3f2252b7f4da4d9a9752ecd7ce54019f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:48.424594 1941 master_service.cc:954] ReplaceTablet: received request to replace tablet ed5861fcf94c47199eb1a81d4702f19b from {username='slave'} at 127.0.0.1:49092
I20260503 19:53:48.424371 1942 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49092:
name: "default.Table_ad6e238ff40f4be99d7cc2d556a40a6b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:48.425035 1941 master_service.cc:954] ReplaceTablet: received request to replace tablet ed5861fcf94c47199eb1a81d4702f19b from {username='slave'} at 127.0.0.1:49092
I20260503 19:53:48.425449 1941 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49092:
table { table_name: "default.Table_15b3325c8812438786ade0fa5c759e21" } modify_external_catalogs: true
I20260503 19:53:48.425639 1941 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49092:
table { table_name: "default.Table_15b3325c8812438786ade0fa5c759e21" } modify_external_catalogs: true
F20260503 19:53:48.429976 1995 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 10 received events: NotificationEvent(eventId=1, eventTime=1777838011, eventType=CREATE_TABLE, dbName=default, tableName=table_cb2fc46da15e4d3cbc25e42aac22a26c, message={"server":"","servicePrincipal":"","db":"default","table":"table_cb2fc46da15e4d3cbc25e42aac22a26c","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_cb2fc46da15e4d3cbc25e42aac22a26c\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1777838010},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/warehouse/table_cb2fc46da15e4d3cbc25e42aac22a26c\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_cb2fc46da15e4d3cbc25e42aac22a26c\",\"kudu.table_id\":\"f40076244f874a50b8ee159c9c1b0786\",\"transient_lastDdlTime\":\"1777838010\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"636a1bc8fd6746489a9e892cd3f8ca43\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1777838011,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1777838011, eventType=CREATE_TABLE, dbName=default, tableName=table_f423a73d5c224651bb5eb676ee98c5e3, message={"server":"","servicePrincipal":"","db":"default","table":"table_f423a73d5c224651bb5eb676ee98c5e3","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_f423a73d5c224651bb5eb676ee98c5e3\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1777838010},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/warehouse/table_f423a73d5c224651bb5eb676ee98c5e3\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_f423a73d5c224651bb5eb676ee98c5e3\",\"kudu.table_id\":\"f8aedcc7a5c34071a86b38b46361455f\",\"transient_lastDdlTime\":\"1777838010\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"636a1bc8fd6746489a9e892cd3f8ca43\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1777838011,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
@ 0x7fceb2f57dcd google::LogMessage::Fail() at ??:0
@ 0x7fceb2f5bb93 google::LogMessage::SendToLog() at ??:0
@ 0x7fceb2f577cc google::LogMessage::Flush() at ??:0
@ 0x7fceb2f57a11 google::LogMessage::~LogMessage() at ??:0
@ 0x7fcec9a30574 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7fcec9a2ee21 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7fcec9a36a9c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7fcec9a36919 std::_Function_handler<>::_M_invoke() at ??:0
@ 0xaa4cad std::function<>::operator()() at ??:?
@ 0x7fceb5cf0529 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fceb92de6db start_thread at ??:0
@ 0x7fceb1e0861f clone at ??:0
W20260503 19:53:48.629123 1002 connection.cc:570] client connection to 127.0.105.62:33821 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260503 19:53:48.629135 1282 connection.cc:570] client connection to 127.0.105.62:33821 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260503 19:53:48.629166 1143 connection.cc:570] client connection to 127.0.105.62:33821 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260503 19:53:48.630302 1404 connection.cc:570] client connection to 127.0.105.62:33821 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260503 19:53:48.631181 1404 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.105.62:33821)
W20260503 19:53:48.631604 1404 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.105.62:33821)
W20260503 19:53:48.631793 1404 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.105.62:33821)
W20260503 19:53:48.632071 1404 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.105.62:33821)
I20260503 19:53:48.757438 2287 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f3a96e537cec46df958468878509b9f9)
I20260503 19:53:48.757709 2287 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:48.759589 2287 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:53:48.761010 2196 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:53:48.762470 2196 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:53:48.762737 2196 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:48.762880 2196 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:53:48.763252 2287 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260503 19:53:49.357748 2312 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f3a96e537cec46df958468878509b9f9)
I20260503 19:53:49.357951 2312 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:49.358614 2312 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:53:49.361550 2196 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:53:49.361789 2196 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:49.361915 2196 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:53:49.362222 2312 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260503 19:53:49.868288 2317 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f3a96e537cec46df958468878509b9f9)
I20260503 19:53:49.868528 2317 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:49.869352 2317 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:53:49.871613 2196 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:53:49.871878 2196 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:49.872036 2196 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:53:49.872283 2317 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260503 19:53:50.492496 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-task12vCql/build/asan/bin/kudu
/tmp/dist-test-task12vCql/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.60:42203
--webserver_interface=127.0.105.60
--webserver_port=36851
--builtin_ntp_servers=127.0.105.20:36687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203
--hive_metastore_uris=thrift://127.0.0.1:38395
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260503 19:53:50.832499 2321 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260503 19:53:50.832864 2321 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260503 19:53:50.833006 2321 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260503 19:53:50.841926 2321 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260503 19:53:50.842021 2321 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260503 19:53:50.842114 2321 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260503 19:53:50.842195 2321 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260503 19:53:50.852443 2321 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.105.20:36687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:38395
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.60:42203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.105.60
--webserver_port=36851
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 0ba7f54e15c9f673bf3e350f30b8d5f1a4037b79
build type FASTDEBUG
built by None at 03 May 2026 19:43:25 UTC on bdcb31816ec0
build id 11730
ASAN enabled
I20260503 19:53:50.854163 2321 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260503 19:53:50.855967 2321 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260503 19:53:50.863970 2328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260503 19:53:50.864006 2327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260503 19:53:50.864655 2330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260503 19:53:50.865252 2321 server_base.cc:1061] running on GCE node
I20260503 19:53:50.866304 2321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260503 19:53:50.867949 2321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260503 19:53:50.869201 2321 hybrid_clock.cc:648] HybridClock initialized: now 1777838030869133 us; error 54 us; skew 500 ppm
I20260503 19:53:50.869597 2321 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260503 19:53:50.871924 2321 webserver.cc:492] Webserver started at http://127.0.105.60:36851/ using document root <none> and password file <none>
I20260503 19:53:50.872676 2321 fs_manager.cc:362] Metadata directory not provided
I20260503 19:53:50.872804 2321 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260503 19:53:50.878412 2321 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260503 19:53:50.881520 2336 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260503 19:53:50.883168 2321 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260503 19:53:50.883356 2321 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/data,/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/wal
uuid: "b0cad11e600c48a3af9a3c56115696d5"
format_stamp: "Formatted at 2026-05-03 19:53:27 on dist-test-slave-gl77"
I20260503 19:53:50.884071 2321 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260503 19:53:50.905766 2321 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260503 19:53:50.906842 2321 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260503 19:53:50.907172 2321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260503 19:53:50.929698 2387 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.105.60:42203 every 8 connection(s)
I20260503 19:53:50.929700 2321 rpc_server.cc:307] RPC server started. Bound to: 127.0.105.60:42203
I20260503 19:53:50.931548 2321 server_base.cc:1193] Dumped server information to /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-2/data/info.pb
I20260503 19:53:50.932467 420 external_mini_cluster.cc:1428] Started /tmp/dist-test-task12vCql/build/asan/bin/kudu as pid 2321
I20260503 19:53:50.946892 2388 sys_catalog.cc:263] Verifying existing consensus state
I20260503 19:53:50.949640 2392 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f3a96e537cec46df958468878509b9f9)
I20260503 19:53:50.949869 2392 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:50.950589 2392 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
I20260503 19:53:50.952479 2388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5: Bootstrap starting.
W20260503 19:53:50.972702 2196 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
I20260503 19:53:50.973623 2388 log.cc:826] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5: Log is configured to *not* fsync() on all Append() calls
I20260503 19:53:51.001857 1256 heartbeater.cc:344] Connected to a master server at 127.0.105.60:42203
I20260503 19:53:51.014300 2388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5: Bootstrap replayed 1/1 log segments. Stats: ops{read=43 overwritten=0 applied=40 ignored=0} inserts{seen=28 ignored=0} mutations{seen=24 ignored=0} orphaned_commits=0. Pending: 3 replicates
I20260503 19:53:51.015343 2388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5: Bootstrap complete.
I20260503 19:53:51.021788 2388 raft_consensus.cc:359] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 4 FOLLOWER]: Replica starting. Triggering 3 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:51.025698 2388 raft_consensus.cc:740] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0cad11e600c48a3af9a3c56115696d5, State: Initialized, Role: FOLLOWER
I20260503 19:53:51.026510 2388 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 40, Last appended: 4.43, Last appended by leader: 43, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:51.027470 2399 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:51.027887 2399 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260503 19:53:51.028995 2388 sys_catalog.cc:565] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260503 19:53:51.029299 2363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" candidate_term: 8 candidate_status { last_received { term: 7 index: 47 } } ignore_live_leader: false dest_uuid: "b0cad11e600c48a3af9a3c56115696d5" is_pre_election: true
I20260503 19:53:51.029882 2363 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ba485a545f44fdd8217c3f86c79d0ca in term 4.
I20260503 19:53:51.030687 2196 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca, b0cad11e600c48a3af9a3c56115696d5; no voters: f3a96e537cec46df958468878509b9f9
I20260503 19:53:51.031827 2392 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260503 19:53:51.031958 2392 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 7 FOLLOWER]: Starting leader election (detected failure of leader f3a96e537cec46df958468878509b9f9)
I20260503 19:53:51.032080 2392 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 7 FOLLOWER]: Advancing to term 8
I20260503 19:53:51.033507 1394 heartbeater.cc:344] Connected to a master server at 127.0.105.60:42203
I20260503 19:53:51.034734 2392 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:51.035480 2392 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 8 election: Requested vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
I20260503 19:53:51.036661 2363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" candidate_term: 8 candidate_status { last_received { term: 7 index: 47 } } ignore_live_leader: false dest_uuid: "b0cad11e600c48a3af9a3c56115696d5"
I20260503 19:53:51.036891 2363 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 4 FOLLOWER]: Advancing to term 8
W20260503 19:53:51.038076 2196 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
I20260503 19:53:51.041672 2363 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ba485a545f44fdd8217c3f86c79d0ca in term 8.
I20260503 19:53:51.042311 2196 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca, b0cad11e600c48a3af9a3c56115696d5; no voters: f3a96e537cec46df958468878509b9f9
I20260503 19:53:51.042690 2392 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 8 FOLLOWER]: Leader election won for term 8
I20260503 19:53:51.042891 1115 heartbeater.cc:344] Connected to a master server at 127.0.105.60:42203
I20260503 19:53:51.043205 2392 raft_consensus.cc:697] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 8 LEADER]: Becoming Leader. State: Replica: 2ba485a545f44fdd8217c3f86c79d0ca, State: Running, Role: LEADER
I20260503 19:53:51.043777 2392 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 47, Committed index: 47, Last appended: 7.47, Last appended by leader: 47, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:51.045697 2410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ba485a545f44fdd8217c3f86c79d0ca. Latest consensus state: current_term: 8 leader_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:51.046149 2410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: This master's current role is: LEADER
I20260503 19:53:51.046940 2414 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260503 19:53:51.051172 2414 catalog_manager.cc:679] Loaded metadata for table default.table_15b3325c8812438786ade0fa5c759e21 [id=04d62d016fc24931ae6f439b1c6152c5]
I20260503 19:53:51.052107 2414 catalog_manager.cc:679] Loaded metadata for table default.table_f91787a2979849d78ba6ba4daf0b4e70 [id=40f19f1e7eaa4640b8c54d5e9e01c320]
I20260503 19:53:51.052795 2414 catalog_manager.cc:679] Loaded metadata for table default.table_0fc3df0b61324d7eac247468c05ce87f [id=54840346b61f4174a481e91d1f745c71]
I20260503 19:53:51.053493 2414 catalog_manager.cc:679] Loaded metadata for table default.table_9c30ca5293344be3964460b578f9d6df [id=8d8f5993b50b4a59aaa004b29cf7c6dd]
I20260503 19:53:51.054101 2414 catalog_manager.cc:679] Loaded metadata for table default.table_db1c67410d564682adca12ef0e942ca2 [id=bd45994325af4ed9ba34633dea7e140e]
I20260503 19:53:51.054725 2414 catalog_manager.cc:679] Loaded metadata for table default.table_dafc7a171f954df18eee7c2911f793f1 [id=da8af64ba75b4650858bb82728b61cd5]
I20260503 19:53:51.055351 2414 catalog_manager.cc:679] Loaded metadata for table default.table_1148c991089047a88d36862e99224cac [id=dce4d9d47ba04a3187de97305003ba5d]
I20260503 19:53:51.055985 2414 catalog_manager.cc:679] Loaded metadata for table default.table_f423a73d5c224651bb5eb676ee98c5e3 [id=f8aedcc7a5c34071a86b38b46361455f]
I20260503 19:53:51.056591 2414 catalog_manager.cc:679] Loaded metadata for table default.table_11964e32037841689422d2a453fff70f [id=f9d69e676109494f8ee9b41f557f4201]
I20260503 19:53:51.059917 2414 tablet_loader.cc:96] loaded metadata for tablet 3ef45a9e31414d209ed6021a0c3f2a04 (table default.table_9c30ca5293344be3964460b578f9d6df [id=8d8f5993b50b4a59aaa004b29cf7c6dd])
I20260503 19:53:51.060771 2416 catalog_manager.cc:1269] Loaded cluster ID: 636a1bc8fd6746489a9e892cd3f8ca43
I20260503 19:53:51.060771 2414 tablet_loader.cc:96] loaded metadata for tablet 542ff627b557426ab448a2339ef1c665 (table default.table_0fc3df0b61324d7eac247468c05ce87f [id=54840346b61f4174a481e91d1f745c71])
I20260503 19:53:51.060973 2416 catalog_manager.cc:1562] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5: loading cluster ID for follower catalog manager: success
I20260503 19:53:51.061174 2414 tablet_loader.cc:96] loaded metadata for tablet 5bd678d9621b4262b8cf0e8fb504e01e (table default.table_dafc7a171f954df18eee7c2911f793f1 [id=da8af64ba75b4650858bb82728b61cd5])
I20260503 19:53:51.061673 2414 tablet_loader.cc:96] loaded metadata for tablet 61730fd814344d6f85f8411bda525df0 (table default.table_db1c67410d564682adca12ef0e942ca2 [id=bd45994325af4ed9ba34633dea7e140e])
I20260503 19:53:51.062361 2414 tablet_loader.cc:96] loaded metadata for tablet 6600b15c33ca480292d1c6eb3002eb86 (table default.table_1148c991089047a88d36862e99224cac [id=dce4d9d47ba04a3187de97305003ba5d])
I20260503 19:53:51.062861 2414 tablet_loader.cc:96] loaded metadata for tablet 6726412cbfd947d2849ac9774d4e86f4 (table default.table_15b3325c8812438786ade0fa5c759e21 [id=04d62d016fc24931ae6f439b1c6152c5])
I20260503 19:53:51.063956 2414 tablet_loader.cc:96] loaded metadata for tablet 7f7923ecc9354b73bb202b969d51e8fa (table default.table_f91787a2979849d78ba6ba4daf0b4e70 [id=40f19f1e7eaa4640b8c54d5e9e01c320])
I20260503 19:53:51.065003 2414 tablet_loader.cc:96] loaded metadata for tablet ac7d7fffebf5463abd8484bffa42bdc1 (table default.table_f423a73d5c224651bb5eb676ee98c5e3 [id=f8aedcc7a5c34071a86b38b46361455f])
I20260503 19:53:51.065527 2416 catalog_manager.cc:1584] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5: acquiring CA information for follower catalog manager: success
I20260503 19:53:51.066502 2414 tablet_loader.cc:96] loaded metadata for tablet f8fb50ad87bf46dcbefe765ae02b017f (table default.table_11964e32037841689422d2a453fff70f [id=f9d69e676109494f8ee9b41f557f4201])
I20260503 19:53:51.066800 2414 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260503 19:53:51.067761 2414 catalog_manager.cc:1269] Loaded cluster ID: 636a1bc8fd6746489a9e892cd3f8ca43
I20260503 19:53:51.067874 2414 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260503 19:53:51.069347 2414 catalog_manager.cc:1514] Loading token signing keys...
I20260503 19:53:51.070497 2414 catalog_manager.cc:6055] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: Loaded TSK: 0
I20260503 19:53:51.071363 2414 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260503 19:53:51.072360 2414 catalog_manager.cc:1537] Loading latest processed Hive Metastore notification log event ID...
I20260503 19:53:51.073266 2414 catalog_manager.cc:5893] Last processed Hive Metastore notification event ID: -1
I20260503 19:53:51.077396 2416 catalog_manager.cc:1612] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260503 19:53:51.073897 2321 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:53:51.078608 2321 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:53:51.117817 2212 master_service.cc:954] ReplaceTablet: received request to replace tablet 6726412cbfd947d2849ac9774d4e86f4 from {username='slave'} at 127.0.0.1:59494
I20260503 19:53:51.118854 2205 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59494:
table { table_name: "default.Table_11964e32037841689422d2a453fff70f" } modify_external_catalogs: true
I20260503 19:53:51.119185 2205 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59494:
table { table_name: "default.Table_11964e32037841689422d2a453fff70f" } modify_external_catalogs: true
I20260503 19:53:51.119614 2208 master_service.cc:954] ReplaceTablet: received request to replace tablet ed5861fcf94c47199eb1a81d4702f19b from {username='slave'} at 127.0.0.1:59494
I20260503 19:53:51.120316 2211 master_service.cc:954] ReplaceTablet: received request to replace tablet a6cc88aae841477d990ed433c705a803 from {username='slave'} at 127.0.0.1:59494
I20260503 19:53:51.120606 2208 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:59494:
table { table_name: "default.Table_0fc3df0b61324d7eac247468c05ce87f" } new_table_name: "default.Table_dd74e45a321f4e7f92e8b514d7378c86" modify_external_catalogs: true
I20260503 19:53:51.121186 2211 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59494:
name: "default.Table_e9dee4f0693a4aca81d75d2c25cefd09"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:51.121747 2209 master_service.cc:954] ReplaceTablet: received request to replace tablet ed5861fcf94c47199eb1a81d4702f19b from {username='slave'} at 127.0.0.1:59494
I20260503 19:53:51.122339 2209 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59494:
name: "default.Table_8a28426e04cf42b8ae2288a7c2950b1b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:51.122889 2207 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59494:
table { table_name: "default.Table_f91787a2979849d78ba6ba4daf0b4e70" } modify_external_catalogs: true
I20260503 19:53:51.123085 2207 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59494:
table { table_name: "default.Table_f91787a2979849d78ba6ba4daf0b4e70" } modify_external_catalogs: true
I20260503 19:53:51.123512 2206 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59494:
name: "default.Table_f28cafcdb9394779ac1e1d270bab59fb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:51.124123 2204 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59494:
name: "default.Table_fa0eb6f31c344d07a5532aced2019227"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:51.124724 2203 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:59494:
table { table_name: "default.Table_f423a73d5c224651bb5eb676ee98c5e3" } new_table_name: "default.Table_90069e6755454566a407bbb659d59d9e" modify_external_catalogs: true
I20260503 19:53:51.126780 2210 master_service.cc:954] ReplaceTablet: received request to replace tablet c4ef894c0fdb4414a6799627641a8857 from {username='slave'} at 127.0.0.1:59494
I20260503 19:53:51.127424 2210 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:59494:
table { table_name: "default.Table_dafc7a171f954df18eee7c2911f793f1" } new_table_name: "default.Table_d2f4bbb38f1d4d0191643de99eeb35b6" modify_external_catalogs: true
I20260503 19:53:51.128927 2363 raft_consensus.cc:1275] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 8 FOLLOWER]: Refusing update from remote peer 2ba485a545f44fdd8217c3f86c79d0ca: Log matching property violated. Preceding OpId in replica: term: 4 index: 43. Preceding OpId from leader: term: 8 index: 49. (index mismatch)
W20260503 19:53:51.129110 2196 consensus_peers.cc:597] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca -> Peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Couldn't send request to peer f3a96e537cec46df958468878509b9f9. Status: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260503 19:53:51.129787 2410 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 48, Last known committed idx: 40, Time since last communication: 0.000s
F20260503 19:53:51.133606 2265 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 10 received events: NotificationEvent(eventId=1, eventTime=1777838011, eventType=CREATE_TABLE, dbName=default, tableName=table_cb2fc46da15e4d3cbc25e42aac22a26c, message={"server":"","servicePrincipal":"","db":"default","table":"table_cb2fc46da15e4d3cbc25e42aac22a26c","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_cb2fc46da15e4d3cbc25e42aac22a26c\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1777838010},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/warehouse/table_cb2fc46da15e4d3cbc25e42aac22a26c\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_cb2fc46da15e4d3cbc25e42aac22a26c\",\"kudu.table_id\":\"f40076244f874a50b8ee159c9c1b0786\",\"transient_lastDdlTime\":\"1777838010\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"636a1bc8fd6746489a9e892cd3f8ca43\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1777838011,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1777838011, eventType=CREATE_TABLE, dbName=default, tableName=table_f423a73d5c224651bb5eb676ee98c5e3, message={"server":"","servicePrincipal":"","db":"default","table":"table_f423a73d5c224651bb5eb676ee98c5e3","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_f423a73d5c224651bb5eb676ee98c5e3\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1777838010},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/warehouse/table_f423a73d5c224651bb5eb676ee98c5e3\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_f423a73d5c224651bb5eb676ee98c5e3\",\"kudu.table_id\":\"f8aedcc7a5c34071a86b38b46361455f\",\"transient_lastDdlTime\":\"1777838010\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"636a1bc8fd6746489a9e892cd3f8ca43\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1777838011,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
@ 0x7f523b192dcd google::LogMessage::Fail() at ??:0
@ 0x7f523b196b93 google::LogMessage::SendToLog() at ??:0
@ 0x7f523b1927cc google::LogMessage::Flush() at ??:0
@ 0x7f523b192a11 google::LogMessage::~LogMessage() at ??:0
@ 0x7f5251c6b574 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f5251c69e21 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260503 19:53:51.163376 2425 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 8 leader_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:51.163520 2399 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ba485a545f44fdd8217c3f86c79d0ca. Latest consensus state: current_term: 8 leader_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:51.163723 2425 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260503 19:53:51.163785 2399 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260503 19:53:51.164119 2427 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 8 leader_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:51.164425 2427 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260503 19:53:51.164522 2420 mvcc.cc:204] Tried to move back new op lower bound from 7282024575493656576 to 7282024575158906880. Current Snapshot: MvccSnapshot[applied={T|T < 7282024575493656576}]
I20260503 19:53:51.164582 2428 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 8 leader_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:51.164865 2428 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260503 19:53:51.165644 2410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 8 leader_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:51.165864 2410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: This master's current role is: LEADER
I20260503 19:53:51.166805 2212 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: Sending DeleteTablet for 3 replicas of tablet 6726412cbfd947d2849ac9774d4e86f4
I20260503 19:53:51.167524 2428 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 8 leader_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:51.167685 2428 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260503 19:53:51.168748 2212 catalog_manager.cc:6741] ReplaceTablet: tablet 6726412cbfd947d2849ac9774d4e86f4 deleted and replaced by tablet 6b5cf0d25b1443fb8ba69fb08aec4973
@ 0x7f5251c71a9c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260503 19:53:51.169193 2212 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59494:
name: "default.Table_b578e2eeb11a4b698a7b51e206e1d015"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
@ 0x7f5251c71919 std::_Function_handler<>::_M_invoke() at ??:0
@ 0xaa4cad std::function<>::operator()() at ??:?
@ 0x7f523df2b529 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f52415196db start_thread at ??:0
@ 0x7f523a04361f clone at ??:0
W20260503 19:53:51.378073 2339 connection.cc:570] server connection from 127.0.0.1:51304 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260503 19:53:51.378373 2431 negotiation.cc:336] Failed RPC negotiation. Trace:
0503 19:53:51.169717 (+ 0us) reactor.cc:678] Submitting negotiation task for server connection from 127.0.0.1:58560 (local address 127.0.105.2:46339)
0503 19:53:51.191925 (+ 22208us) server_negotiation.cc:207] Beginning negotiation
0503 19:53:51.191932 (+ 7us) server_negotiation.cc:400] Waiting for connection header
0503 19:53:51.191955 (+ 23us) server_negotiation.cc:408] Connection header received
0503 19:53:51.192029 (+ 74us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0503 19:53:51.192035 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
0503 19:53:51.192133 (+ 98us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0503 19:53:51.192257 (+ 124us) server_negotiation.cc:227] Negotiated authn=CERTIFICATE
0503 19:53:51.377999 (+185742us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58560: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":22086,"thread_start_us":21977,"threads_started":1}
W20260503 19:53:51.378484 2433 negotiation.cc:336] Failed RPC negotiation. Trace:
0503 19:53:51.173641 (+ 0us) reactor.cc:678] Submitting negotiation task for server connection from 127.0.0.1:52506 (local address 127.0.105.3:41575)
0503 19:53:51.191159 (+ 17518us) server_negotiation.cc:207] Beginning negotiation
0503 19:53:51.191168 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0503 19:53:51.378077 (+186909us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52506: BlockingRecv error: recv got EOF from 127.0.0.1:52506 (error 108)
Metrics: {"server-negotiator.queue_time_us":17338,"thread_start_us":17532,"threads_started":1}
W20260503 19:53:51.378535 1282 connection.cc:570] client connection to 127.0.105.61:35345 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260503 19:53:51.378866 1002 connection.cc:570] client connection to 127.0.105.61:35345 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260503 19:53:51.379220 1143 connection.cc:570] client connection to 127.0.105.61:35345 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260503 19:53:51.380193 1404 connection.cc:570] client connection to 127.0.105.61:35345 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260503 19:53:51.380929 1404 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.105.61:35345)
W20260503 19:53:51.381779 1404 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.105.61:35345)
W20260503 19:53:51.382040 1404 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.105.61:35345)
W20260503 19:53:51.382097 2430 negotiation.cc:336] Failed RPC negotiation. Trace:
0503 19:53:51.169295 (+ 0us) reactor.cc:678] Submitting negotiation task for server connection from 127.0.0.1:58998 (local address 127.0.105.1:40289)
0503 19:53:51.191652 (+ 22357us) server_negotiation.cc:207] Beginning negotiation
0503 19:53:51.191660 (+ 8us) server_negotiation.cc:400] Waiting for connection header
0503 19:53:51.191685 (+ 25us) server_negotiation.cc:408] Connection header received
0503 19:53:51.191773 (+ 88us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0503 19:53:51.191778 (+ 5us) server_negotiation.cc:462] Received NEGOTIATE request from client
0503 19:53:51.191866 (+ 88us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0503 19:53:51.191962 (+ 96us) server_negotiation.cc:227] Negotiated authn=CERTIFICATE
0503 19:53:51.380945 (+188983us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58998: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":22216,"thread_start_us":22105,"threads_started":1}
W20260503 19:53:51.382629 1404 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.105.61:35345)
I20260503 19:53:51.506649 2353 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "fc902de5b2274103ab5e76f967503afb" instance_seqno: 1777838009464378) as {username='slave'} at 127.0.105.2:36469; Asking this server to re-register.
I20260503 19:53:51.507441 1256 heartbeater.cc:461] Registering TS with master...
I20260503 19:53:51.509541 2353 ts_manager.cc:194] Registered new tserver with Master: fc902de5b2274103ab5e76f967503afb (127.0.105.2:46339)
I20260503 19:53:51.536900 2353 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" instance_seqno: 1777838010067456) as {username='slave'} at 127.0.105.3:45431; Asking this server to re-register.
I20260503 19:53:51.537516 1394 heartbeater.cc:461] Registering TS with master...
I20260503 19:53:51.538745 2353 ts_manager.cc:194] Registered new tserver with Master: b68b1efd1cc44e15b8c69e730229dbdd (127.0.105.3:41575)
I20260503 19:53:51.545976 2353 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "b8594a7337c54dc58d701259120d361b" instance_seqno: 1777838008819588) as {username='slave'} at 127.0.105.1:51999; Asking this server to re-register.
I20260503 19:53:51.546648 1115 heartbeater.cc:461] Registering TS with master...
I20260503 19:53:51.548055 2353 ts_manager.cc:194] Registered new tserver with Master: b8594a7337c54dc58d701259120d361b (127.0.105.1:40289)
I20260503 19:53:51.688618 2447 raft_consensus.cc:493] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2ba485a545f44fdd8217c3f86c79d0ca)
I20260503 19:53:51.688907 2447 raft_consensus.cc:515] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:51.691058 2447 leader_election.cc:290] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345)
W20260503 19:53:51.691993 2337 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:53:51.693632 2337 leader_election.cc:336] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:53:51.693933 2337 leader_election.cc:336] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345): Network error: Client connection negotiation failed: client connection to 127.0.105.61:35345: connect: Connection refused (error 111)
I20260503 19:53:51.694048 2337 leader_election.cc:304] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0cad11e600c48a3af9a3c56115696d5; no voters: 2ba485a545f44fdd8217c3f86c79d0ca, f3a96e537cec46df958468878509b9f9
I20260503 19:53:51.694494 2447 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260503 19:53:52.310556 2450 raft_consensus.cc:493] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2ba485a545f44fdd8217c3f86c79d0ca)
I20260503 19:53:52.310801 2450 raft_consensus.cc:515] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:52.311517 2450 leader_election.cc:290] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345)
W20260503 19:53:52.313818 2337 leader_election.cc:336] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:53:52.314061 2337 leader_election.cc:336] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345): Network error: Client connection negotiation failed: client connection to 127.0.105.61:35345: connect: Connection refused (error 111)
I20260503 19:53:52.314190 2337 leader_election.cc:304] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0cad11e600c48a3af9a3c56115696d5; no voters: 2ba485a545f44fdd8217c3f86c79d0ca, f3a96e537cec46df958468878509b9f9
I20260503 19:53:52.314533 2450 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260503 19:53:53.105526 2454 raft_consensus.cc:493] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2ba485a545f44fdd8217c3f86c79d0ca)
I20260503 19:53:53.105742 2454 raft_consensus.cc:515] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:53.106396 2454 leader_election.cc:290] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345)
W20260503 19:53:53.109053 2337 leader_election.cc:336] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:53:53.109326 2337 leader_election.cc:336] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345): Network error: Client connection negotiation failed: client connection to 127.0.105.61:35345: connect: Connection refused (error 111)
I20260503 19:53:53.109422 2337 leader_election.cc:304] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0cad11e600c48a3af9a3c56115696d5; no voters: 2ba485a545f44fdd8217c3f86c79d0ca, f3a96e537cec46df958468878509b9f9
I20260503 19:53:53.109722 2454 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260503 19:53:54.197353 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-task12vCql/build/asan/bin/kudu
/tmp/dist-test-task12vCql/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.61:35345
--webserver_interface=127.0.105.61
--webserver_port=39263
--builtin_ntp_servers=127.0.105.20:36687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203
--hive_metastore_uris=thrift://127.0.0.1:38395
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260503 19:53:54.423363 2460 raft_consensus.cc:493] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2ba485a545f44fdd8217c3f86c79d0ca)
I20260503 19:53:54.423559 2460 raft_consensus.cc:515] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:54.424324 2460 leader_election.cc:290] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345)
W20260503 19:53:54.426776 2337 leader_election.cc:336] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:53:54.427004 2337 leader_election.cc:336] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345): Network error: Client connection negotiation failed: client connection to 127.0.105.61:35345: connect: Connection refused (error 111)
I20260503 19:53:54.427130 2337 leader_election.cc:304] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0cad11e600c48a3af9a3c56115696d5; no voters: 2ba485a545f44fdd8217c3f86c79d0ca, f3a96e537cec46df958468878509b9f9
I20260503 19:53:54.427395 2460 raft_consensus.cc:2749] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260503 19:53:54.540918 2459 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260503 19:53:54.541324 2459 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260503 19:53:54.541481 2459 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260503 19:53:54.550246 2459 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260503 19:53:54.550343 2459 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260503 19:53:54.550434 2459 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260503 19:53:54.550535 2459 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260503 19:53:54.562857 2459 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.105.20:36687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:38395
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.61:35345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.105.61
--webserver_port=39263
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 0ba7f54e15c9f673bf3e350f30b8d5f1a4037b79
build type FASTDEBUG
built by None at 03 May 2026 19:43:25 UTC on bdcb31816ec0
build id 11730
ASAN enabled
I20260503 19:53:54.565265 2459 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260503 19:53:54.568013 2459 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260503 19:53:54.579383 2468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260503 19:53:54.579741 2459 server_base.cc:1061] running on GCE node
W20260503 19:53:54.579983 2469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260503 19:53:54.580617 2471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260503 19:53:54.581641 2459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260503 19:53:54.583231 2459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260503 19:53:54.584456 2459 hybrid_clock.cc:648] HybridClock initialized: now 1777838034584399 us; error 45 us; skew 500 ppm
I20260503 19:53:54.584865 2459 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260503 19:53:54.587294 2459 webserver.cc:492] Webserver started at http://127.0.105.61:39263/ using document root <none> and password file <none>
I20260503 19:53:54.588155 2459 fs_manager.cc:362] Metadata directory not provided
I20260503 19:53:54.588258 2459 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260503 19:53:54.594018 2459 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260503 19:53:54.596985 2477 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260503 19:53:54.598479 2459 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260503 19:53:54.598666 2459 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data,/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/wal
uuid: "2ba485a545f44fdd8217c3f86c79d0ca"
format_stamp: "Formatted at 2026-05-03 19:53:27 on dist-test-slave-gl77"
I20260503 19:53:54.599406 2459 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260503 19:53:54.611364 2459 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260503 19:53:54.612229 2459 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260503 19:53:54.612546 2459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260503 19:53:54.634622 2459 rpc_server.cc:307] RPC server started. Bound to: 127.0.105.61:35345
I20260503 19:53:54.634650 2529 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.105.61:35345 every 8 connection(s)
I20260503 19:53:54.636468 2459 server_base.cc:1193] Dumped server information to /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data/info.pb
I20260503 19:53:54.637326 420 external_mini_cluster.cc:1428] Started /tmp/dist-test-task12vCql/build/asan/bin/kudu as pid 2459
I20260503 19:53:54.644335 2530 sys_catalog.cc:263] Verifying existing consensus state
I20260503 19:53:54.648801 2530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: Bootstrap starting.
I20260503 19:53:54.663455 2530 log.cc:826] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: Log is configured to *not* fsync() on all Append() calls
I20260503 19:53:54.700770 2530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=50 overwritten=0 applied=49 ignored=0} inserts{seen=33 ignored=0} mutations{seen=29 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260503 19:53:54.701472 2530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: Bootstrap complete.
I20260503 19:53:54.706981 2530 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 8 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:54.709545 2530 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 8 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba485a545f44fdd8217c3f86c79d0ca, State: Initialized, Role: FOLLOWER
I20260503 19:53:54.710260 2530 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 49, Last appended: 8.50, Last appended by leader: 50, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:54.711053 2535 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 8 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:54.711380 2535 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: This master's current role is: FOLLOWER
I20260503 19:53:54.712136 2530 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: configured and running, proceeding with master startup.
I20260503 19:53:54.724114 2549 catalog_manager.cc:1269] Loaded cluster ID: 636a1bc8fd6746489a9e892cd3f8ca43
I20260503 19:53:54.724272 2549 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: loading cluster ID for follower catalog manager: success
I20260503 19:53:54.728324 2549 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: acquiring CA information for follower catalog manager: success
I20260503 19:53:54.731369 2549 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260503 19:53:54.732559 2459 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:53:54.738446 2459 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:53:54.896942 1396 heartbeater.cc:344] Connected to a master server at 127.0.105.61:35345
I20260503 19:53:54.900364 1117 heartbeater.cc:344] Connected to a master server at 127.0.105.61:35345
I20260503 19:53:54.900388 1261 heartbeater.cc:344] Connected to a master server at 127.0.105.61:35345
I20260503 19:53:54.900357 2495 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" instance_seqno: 1777838010067456) as {username='slave'} at 127.0.105.3:34861; Asking this server to re-register.
I20260503 19:53:54.901371 1396 heartbeater.cc:461] Registering TS with master...
I20260503 19:53:54.901844 2495 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "fc902de5b2274103ab5e76f967503afb" instance_seqno: 1777838009464378) as {username='slave'} at 127.0.105.2:53295; Asking this server to re-register.
I20260503 19:53:54.902031 2493 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "b8594a7337c54dc58d701259120d361b" instance_seqno: 1777838008819588) as {username='slave'} at 127.0.105.1:55761; Asking this server to re-register.
I20260503 19:53:54.902493 1261 heartbeater.cc:461] Registering TS with master...
I20260503 19:53:54.902671 1117 heartbeater.cc:461] Registering TS with master...
I20260503 19:53:54.903542 2493 ts_manager.cc:194] Registered new tserver with Master: b68b1efd1cc44e15b8c69e730229dbdd (127.0.105.3:41575)
I20260503 19:53:54.903996 2495 ts_manager.cc:194] Registered new tserver with Master: fc902de5b2274103ab5e76f967503afb (127.0.105.2:46339)
I20260503 19:53:54.904491 2494 ts_manager.cc:194] Registered new tserver with Master: b8594a7337c54dc58d701259120d361b (127.0.105.1:40289)
I20260503 19:53:54.989845 2555 raft_consensus.cc:493] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2ba485a545f44fdd8217c3f86c79d0ca)
I20260503 19:53:54.990187 2555 raft_consensus.cc:515] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:54.991106 2555 leader_election.cc:290] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345)
W20260503 19:53:54.997833 2337 leader_election.cc:336] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
I20260503 19:53:55.000869 2505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b0cad11e600c48a3af9a3c56115696d5" candidate_term: 9 candidate_status { last_received { term: 8 index: 50 } } ignore_live_leader: false dest_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" is_pre_election: true
I20260503 19:53:55.001616 2505 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0cad11e600c48a3af9a3c56115696d5 in term 8.
I20260503 19:53:55.002404 2337 leader_election.cc:304] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca, b0cad11e600c48a3af9a3c56115696d5; no voters: f3a96e537cec46df958468878509b9f9
I20260503 19:53:55.002749 2555 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260503 19:53:55.002862 2555 raft_consensus.cc:493] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 2ba485a545f44fdd8217c3f86c79d0ca)
I20260503 19:53:55.002956 2555 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 8 FOLLOWER]: Advancing to term 9
I20260503 19:53:55.005972 2555 raft_consensus.cc:515] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:55.006793 2555 leader_election.cc:290] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 9 election: Requested vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), 2ba485a545f44fdd8217c3f86c79d0ca (127.0.105.61:35345)
I20260503 19:53:55.007769 2505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b0cad11e600c48a3af9a3c56115696d5" candidate_term: 9 candidate_status { last_received { term: 8 index: 50 } } ignore_live_leader: false dest_uuid: "2ba485a545f44fdd8217c3f86c79d0ca"
I20260503 19:53:55.008006 2505 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 8 FOLLOWER]: Advancing to term 9
W20260503 19:53:55.008322 2337 leader_election.cc:336] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
I20260503 19:53:55.012058 2505 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0cad11e600c48a3af9a3c56115696d5 in term 9.
I20260503 19:53:55.012669 2337 leader_election.cc:304] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca, b0cad11e600c48a3af9a3c56115696d5; no voters: f3a96e537cec46df958468878509b9f9
I20260503 19:53:55.013120 2555 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 9 FOLLOWER]: Leader election won for term 9
I20260503 19:53:55.013649 2555 raft_consensus.cc:697] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [term 9 LEADER]: Becoming Leader. State: Replica: b0cad11e600c48a3af9a3c56115696d5, State: Running, Role: LEADER
I20260503 19:53:55.014277 2555 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 49, Committed index: 49, Last appended: 8.50, Last appended by leader: 50, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:55.016601 2559 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0cad11e600c48a3af9a3c56115696d5. Latest consensus state: current_term: 9 leader_uuid: "b0cad11e600c48a3af9a3c56115696d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:55.016958 2559 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: This master's current role is: LEADER
W20260503 19:53:55.425942 2337 consensus_peers.cc:597] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 -> Peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Couldn't send request to peer f3a96e537cec46df958468878509b9f9. Status: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260503 19:53:55.572101 2505 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Refusing update from remote peer b0cad11e600c48a3af9a3c56115696d5: Log matching property violated. Preceding OpId in replica: term: 8 index: 50. Preceding OpId from leader: term: 9 index: 51. (index mismatch)
I20260503 19:53:55.572917 2559 consensus_queue.cc:1048] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 51, Last known committed idx: 49, Time since last communication: 0.000s
I20260503 19:53:55.575464 2563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0cad11e600c48a3af9a3c56115696d5. Latest consensus state: current_term: 9 leader_uuid: "b0cad11e600c48a3af9a3c56115696d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:55.575727 2563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: This master's current role is: FOLLOWER
I20260503 19:53:55.579599 2559 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 9 leader_uuid: "b0cad11e600c48a3af9a3c56115696d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:55.579862 2559 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5 [sys.catalog]: This master's current role is: LEADER
I20260503 19:53:55.580168 2563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 9 leader_uuid: "b0cad11e600c48a3af9a3c56115696d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:55.580415 2563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: This master's current role is: FOLLOWER
I20260503 19:53:55.663491 2561 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260503 19:53:55.666249 2561 catalog_manager.cc:679] Loaded metadata for table default.table_15b3325c8812438786ade0fa5c759e21 [id=04d62d016fc24931ae6f439b1c6152c5]
I20260503 19:53:55.666785 2561 catalog_manager.cc:679] Loaded metadata for table default.table_f91787a2979849d78ba6ba4daf0b4e70 [id=40f19f1e7eaa4640b8c54d5e9e01c320]
I20260503 19:53:55.667258 2561 catalog_manager.cc:679] Loaded metadata for table default.table_0fc3df0b61324d7eac247468c05ce87f [id=54840346b61f4174a481e91d1f745c71]
I20260503 19:53:55.667649 2561 catalog_manager.cc:679] Loaded metadata for table default.table_9c30ca5293344be3964460b578f9d6df [id=8d8f5993b50b4a59aaa004b29cf7c6dd]
I20260503 19:53:55.667991 2561 catalog_manager.cc:679] Loaded metadata for table default.table_db1c67410d564682adca12ef0e942ca2 [id=bd45994325af4ed9ba34633dea7e140e]
I20260503 19:53:55.668370 2561 catalog_manager.cc:679] Loaded metadata for table default.table_dafc7a171f954df18eee7c2911f793f1 [id=da8af64ba75b4650858bb82728b61cd5]
I20260503 19:53:55.668704 2561 catalog_manager.cc:679] Loaded metadata for table default.table_1148c991089047a88d36862e99224cac [id=dce4d9d47ba04a3187de97305003ba5d]
I20260503 19:53:55.669062 2561 catalog_manager.cc:679] Loaded metadata for table default.table_f423a73d5c224651bb5eb676ee98c5e3 [id=f8aedcc7a5c34071a86b38b46361455f]
I20260503 19:53:55.669505 2561 catalog_manager.cc:679] Loaded metadata for table default.table_11964e32037841689422d2a453fff70f [id=f9d69e676109494f8ee9b41f557f4201]
I20260503 19:53:55.671461 2561 tablet_loader.cc:96] loaded metadata for tablet 3ef45a9e31414d209ed6021a0c3f2a04 (table default.table_9c30ca5293344be3964460b578f9d6df [id=8d8f5993b50b4a59aaa004b29cf7c6dd])
I20260503 19:53:55.672005 2561 tablet_loader.cc:96] loaded metadata for tablet 542ff627b557426ab448a2339ef1c665 (table default.table_0fc3df0b61324d7eac247468c05ce87f [id=54840346b61f4174a481e91d1f745c71])
I20260503 19:53:55.672328 2561 tablet_loader.cc:96] loaded metadata for tablet 5bd678d9621b4262b8cf0e8fb504e01e (table default.table_dafc7a171f954df18eee7c2911f793f1 [id=da8af64ba75b4650858bb82728b61cd5])
I20260503 19:53:55.672670 2561 tablet_loader.cc:96] loaded metadata for tablet 61730fd814344d6f85f8411bda525df0 (table default.table_db1c67410d564682adca12ef0e942ca2 [id=bd45994325af4ed9ba34633dea7e140e])
I20260503 19:53:55.673199 2561 tablet_loader.cc:96] loaded metadata for tablet 6600b15c33ca480292d1c6eb3002eb86 (table default.table_1148c991089047a88d36862e99224cac [id=dce4d9d47ba04a3187de97305003ba5d])
I20260503 19:53:55.673861 2561 tablet_loader.cc:96] loaded metadata for tablet 6b5cf0d25b1443fb8ba69fb08aec4973 (table default.table_15b3325c8812438786ade0fa5c759e21 [id=04d62d016fc24931ae6f439b1c6152c5])
I20260503 19:53:55.674337 2561 tablet_loader.cc:96] loaded metadata for tablet 7f7923ecc9354b73bb202b969d51e8fa (table default.table_f91787a2979849d78ba6ba4daf0b4e70 [id=40f19f1e7eaa4640b8c54d5e9e01c320])
I20260503 19:53:55.674969 2561 tablet_loader.cc:96] loaded metadata for tablet ac7d7fffebf5463abd8484bffa42bdc1 (table default.table_f423a73d5c224651bb5eb676ee98c5e3 [id=f8aedcc7a5c34071a86b38b46361455f])
I20260503 19:53:55.675899 2561 tablet_loader.cc:96] loaded metadata for tablet f8fb50ad87bf46dcbefe765ae02b017f (table default.table_11964e32037841689422d2a453fff70f [id=f9d69e676109494f8ee9b41f557f4201])
I20260503 19:53:55.676180 2561 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260503 19:53:55.677310 2561 catalog_manager.cc:1269] Loaded cluster ID: 636a1bc8fd6746489a9e892cd3f8ca43
I20260503 19:53:55.677409 2561 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260503 19:53:55.678962 2561 catalog_manager.cc:1514] Loading token signing keys...
I20260503 19:53:55.679966 2561 catalog_manager.cc:6055] T 00000000000000000000000000000000 P b0cad11e600c48a3af9a3c56115696d5: Loaded TSK: 0
I20260503 19:53:55.680874 2561 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260503 19:53:55.681844 2561 catalog_manager.cc:1537] Loading latest processed Hive Metastore notification log event ID...
I20260503 19:53:55.682793 2561 catalog_manager.cc:5893] Last processed Hive Metastore notification event ID: -1
I20260503 19:53:55.719211 2353 master_service.cc:954] ReplaceTablet: received request to replace tablet a6cc88aae841477d990ed433c705a803 from {username='slave'} at 127.0.0.1:51302
I20260503 19:53:55.719470 2352 master_service.cc:954] ReplaceTablet: received request to replace tablet a6cc88aae841477d990ed433c705a803 from {username='slave'} at 127.0.0.1:51302
I20260503 19:53:55.719859 2351 master_service.cc:954] ReplaceTablet: received request to replace tablet f391e07ff0e24bc5b3cf7e81342161fe from {username='slave'} at 127.0.0.1:51302
I20260503 19:53:55.720009 2349 master_service.cc:954] ReplaceTablet: received request to replace tablet a6cc88aae841477d990ed433c705a803 from {username='slave'} at 127.0.0.1:51302
I20260503 19:53:55.720685 2350 master_service.cc:954] ReplaceTablet: received request to replace tablet f391e07ff0e24bc5b3cf7e81342161fe from {username='slave'} at 127.0.0.1:51302
I20260503 19:53:55.720826 2347 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51302:
table { table_name: "default.Table_15b3325c8812438786ade0fa5c759e21" } modify_external_catalogs: true
I20260503 19:53:55.721202 2347 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51302:
table { table_name: "default.Table_15b3325c8812438786ade0fa5c759e21" } modify_external_catalogs: true
I20260503 19:53:55.720975 2348 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51302:
name: "default.Table_1eff9f94b0c84e7180c54349d66db8e4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:55.721457 2346 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51302:
name: "default.Table_ad6e238ff40f4be99d7cc2d556a40a6b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:55.722683 2352 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51302:
name: "default.Table_49084d26aae440ac958d415c2b291948"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:55.722769 2349 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51302:
name: "default.Table_3f2252b7f4da4d9a9752ecd7ce54019f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:55.722711 2351 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51302:
name: "default.Table_ba1a53aa8c174babb4bc2b60b558a8c1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:55.723680 2350 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51302:
table { table_name: "default.Table_1148c991089047a88d36862e99224cac" } new_table_name: "default.Table_a0d72b6ac2cf408296e54ae1978d1ff2" modify_external_catalogs: true
I20260503 19:53:55.723986 2344 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51302:
name: "default.Table_f28cafcdb9394779ac1e1d270bab59fb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260503 19:53:55.724515 2345 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51302:
table { table_name: "default.Table_f91787a2979849d78ba6ba4daf0b4e70" } modify_external_catalogs: true
I20260503 19:53:55.724783 2345 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51302:
table { table_name: "default.Table_f91787a2979849d78ba6ba4daf0b4e70" } modify_external_catalogs: true
I20260503 19:53:55.722214 2353 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51302:
table { table_name: "default.Table_db1c67410d564682adca12ef0e942ca2" } modify_external_catalogs: true
I20260503 19:53:55.725325 2353 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51302:
table { table_name: "default.Table_db1c67410d564682adca12ef0e942ca2" } modify_external_catalogs: true
F20260503 19:53:55.729238 2415 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 10 received events: NotificationEvent(eventId=1, eventTime=1777838011, eventType=CREATE_TABLE, dbName=default, tableName=table_cb2fc46da15e4d3cbc25e42aac22a26c, message={"server":"","servicePrincipal":"","db":"default","table":"table_cb2fc46da15e4d3cbc25e42aac22a26c","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_cb2fc46da15e4d3cbc25e42aac22a26c\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1777838010},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/warehouse/table_cb2fc46da15e4d3cbc25e42aac22a26c\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_cb2fc46da15e4d3cbc25e42aac22a26c\",\"kudu.table_id\":\"f40076244f874a50b8ee159c9c1b0786\",\"transient_lastDdlTime\":\"1777838010\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"636a1bc8fd6746489a9e892cd3f8ca43\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1777838011,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1777838011, eventType=CREATE_TABLE, dbName=default, tableName=table_f423a73d5c224651bb5eb676ee98c5e3, message={"server":"","servicePrincipal":"","db":"default","table":"table_f423a73d5c224651bb5eb676ee98c5e3","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_f423a73d5c224651bb5eb676ee98c5e3\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1777838010},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/warehouse/table_f423a73d5c224651bb5eb676ee98c5e3\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_f423a73d5c224651bb5eb676ee98c5e3\",\"kudu.table_id\":\"f8aedcc7a5c34071a86b38b46361455f\",\"transient_lastDdlTime\":\"1777838010\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"636a1bc8fd6746489a9e892cd3f8ca43\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1777838011,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
@ 0x7f255432edcd google::LogMessage::Fail() at ??:0
@ 0x7f2554332b93 google::LogMessage::SendToLog() at ??:0
@ 0x7f255432e7cc google::LogMessage::Flush() at ??:0
@ 0x7f255432ea11 google::LogMessage::~LogMessage() at ??:0
@ 0x7f256ae07574 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f256ae05e21 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f256ae0da9c kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f256ae0d919 std::_Function_handler<>::_M_invoke() at ??:0
@ 0xaa4cad std::function<>::operator()() at ??:?
@ 0x7f25570c7529 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f255a6b56db start_thread at ??:0
@ 0x7f25531df61f clone at ??:0
W20260503 19:53:55.934984 1404 connection.cc:570] client connection to 127.0.105.60:42203 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260503 19:53:55.935631 1404 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.105.60:42203)
W20260503 19:53:55.936277 1404 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.105.60:42203)
W20260503 19:53:55.936480 1404 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.105.60:42203)
W20260503 19:53:55.937242 1404 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.105.60:42203)
I20260503 19:53:56.079046 2563 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b0cad11e600c48a3af9a3c56115696d5)
I20260503 19:53:56.079254 2563 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:56.081559 2563 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:53:56.082429 2478 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:53:56.083964 2478 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:53:56.084227 2478 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:56.084375 2478 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:53:56.084738 2563 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260503 19:53:56.615943 2588 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b0cad11e600c48a3af9a3c56115696d5)
I20260503 19:53:56.616168 2588 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:56.616888 2588 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:53:56.619262 2478 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:53:56.619496 2478 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:56.619621 2478 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:53:56.619920 2588 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260503 19:53:56.982699 420 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-task12vCql/build/asan/bin/kudu pid 2459
I20260503 19:53:56.996194 2525 generic_service.cc:196] Checking for leaks (request via RPC)
I20260503 19:53:57.109045 420 external_mini_cluster.cc:1658] Killing /tmp/dist-test-task12vCql/build/asan/bin/kudu with pid 2459
I20260503 19:53:57.710152 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-task12vCql/build/asan/bin/kudu
/tmp/dist-test-task12vCql/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.61:35345
--webserver_interface=127.0.105.61
--webserver_port=39263
--builtin_ntp_servers=127.0.105.20:36687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203
--hive_metastore_uris=thrift://127.0.0.1:38395
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260503 19:53:58.049315 2598 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260503 19:53:58.049669 2598 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260503 19:53:58.049804 2598 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260503 19:53:58.058458 2598 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260503 19:53:58.058560 2598 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260503 19:53:58.058667 2598 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260503 19:53:58.058759 2598 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260503 19:53:58.069504 2598 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.105.20:36687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:38395
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.61:35345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.105.61
--webserver_port=39263
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 0ba7f54e15c9f673bf3e350f30b8d5f1a4037b79
build type FASTDEBUG
built by None at 03 May 2026 19:43:25 UTC on bdcb31816ec0
build id 11730
ASAN enabled
I20260503 19:53:58.071395 2598 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260503 19:53:58.073278 2598 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260503 19:53:58.081517 2605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260503 19:53:58.081946 2607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260503 19:53:58.082316 2604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260503 19:53:58.083031 2598 server_base.cc:1061] running on GCE node
I20260503 19:53:58.083839 2598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260503 19:53:58.085453 2598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260503 19:53:58.086714 2598 hybrid_clock.cc:648] HybridClock initialized: now 1777838038086651 us; error 59 us; skew 500 ppm
I20260503 19:53:58.087108 2598 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260503 19:53:58.094312 2598 webserver.cc:492] Webserver started at http://127.0.105.61:39263/ using document root <none> and password file <none>
I20260503 19:53:58.095184 2598 fs_manager.cc:362] Metadata directory not provided
I20260503 19:53:58.095323 2598 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260503 19:53:58.101511 2598 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260503 19:53:58.104756 2613 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260503 19:53:58.106487 2598 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260503 19:53:58.106700 2598 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data,/tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/wal
uuid: "2ba485a545f44fdd8217c3f86c79d0ca"
format_stamp: "Formatted at 2026-05-03 19:53:27 on dist-test-slave-gl77"
I20260503 19:53:58.107508 2598 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260503 19:53:58.120256 2598 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260503 19:53:58.121248 2598 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260503 19:53:58.121563 2598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260503 19:53:58.141222 2598 rpc_server.cc:307] RPC server started. Bound to: 127.0.105.61:35345
I20260503 19:53:58.141271 2664 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.105.61:35345 every 8 connection(s)
I20260503 19:53:58.142894 2598 server_base.cc:1193] Dumped server information to /tmp/dist-test-task12vCql/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1777837987938840-420-0/minicluster-data/master-1/data/info.pb
I20260503 19:53:58.149536 2665 sys_catalog.cc:263] Verifying existing consensus state
I20260503 19:53:58.150923 420 external_mini_cluster.cc:1428] Started /tmp/dist-test-task12vCql/build/asan/bin/kudu as pid 2598
I20260503 19:53:58.153676 2665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: Bootstrap starting.
I20260503 19:53:58.170066 2665 log.cc:826] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: Log is configured to *not* fsync() on all Append() calls
I20260503 19:53:58.206758 2665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=51 overwritten=0 applied=51 ignored=0} inserts{seen=33 ignored=0} mutations{seen=37 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260503 19:53:58.207423 2665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: Bootstrap complete.
I20260503 19:53:58.212616 2665 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:58.213388 2665 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ba485a545f44fdd8217c3f86c79d0ca, State: Initialized, Role: FOLLOWER
I20260503 19:53:58.214110 2665 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 51, Last appended: 9.51, Last appended by leader: 51, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:58.214936 2670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 9 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } } }
I20260503 19:53:58.215224 2670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: This master's current role is: FOLLOWER
I20260503 19:53:58.216080 2665 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [sys.catalog]: configured and running, proceeding with master startup.
I20260503 19:53:58.228854 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:53:58.232807 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:53:58.243989 2684 catalog_manager.cc:1269] Loaded cluster ID: 636a1bc8fd6746489a9e892cd3f8ca43
I20260503 19:53:58.244128 2684 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: loading cluster ID for follower catalog manager: success
I20260503 19:53:58.248565 2684 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: acquiring CA information for follower catalog manager: success
I20260503 19:53:58.251780 2684 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260503 19:53:58.427335 1117 heartbeater.cc:344] Connected to a master server at 127.0.105.61:35345
I20260503 19:53:58.428728 1396 heartbeater.cc:344] Connected to a master server at 127.0.105.61:35345
I20260503 19:53:58.430142 1261 heartbeater.cc:344] Connected to a master server at 127.0.105.61:35345
I20260503 19:53:58.431222 2630 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "b68b1efd1cc44e15b8c69e730229dbdd" instance_seqno: 1777838010067456) as {username='slave'} at 127.0.105.3:53539; Asking this server to re-register.
I20260503 19:53:58.431216 2629 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "b8594a7337c54dc58d701259120d361b" instance_seqno: 1777838008819588) as {username='slave'} at 127.0.105.1:45799; Asking this server to re-register.
I20260503 19:53:58.431659 2628 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "fc902de5b2274103ab5e76f967503afb" instance_seqno: 1777838009464378) as {username='slave'} at 127.0.105.2:51099; Asking this server to re-register.
I20260503 19:53:58.432296 1396 heartbeater.cc:461] Registering TS with master...
I20260503 19:53:58.432300 1117 heartbeater.cc:461] Registering TS with master...
I20260503 19:53:58.432382 1261 heartbeater.cc:461] Registering TS with master...
I20260503 19:53:58.434473 2629 ts_manager.cc:194] Registered new tserver with Master: b8594a7337c54dc58d701259120d361b (127.0.105.1:40289)
I20260503 19:53:58.434653 2628 ts_manager.cc:194] Registered new tserver with Master: b68b1efd1cc44e15b8c69e730229dbdd (127.0.105.3:41575)
I20260503 19:53:58.434772 2630 ts_manager.cc:194] Registered new tserver with Master: fc902de5b2274103ab5e76f967503afb (127.0.105.2:46339)
I20260503 19:53:58.600056 2670 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:53:58.600351 2670 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:58.602667 2670 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:53:58.603295 2614 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:53:58.605373 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:53:58.605685 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:58.605871 2614 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:53:58.606298 2670 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260503 19:53:59.271068 2692 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:53:59.271409 2692 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:53:59.272324 2692 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:53:59.278240 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:53:59.278481 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:53:59.278604 2614 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:53:59.278924 2692 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260503 19:54:00.091091 2698 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:54:00.091315 2698 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:54:00.092001 2698 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:54:00.094235 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:54:00.094452 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:54:00.094555 2614 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:54:00.094885 2698 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260503 19:54:00.319097 420 master-stress-test.cc:577] Killing test thread 1/25
I20260503 19:54:01.365703 2702 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:54:01.365888 2702 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:54:01.366670 2702 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:54:01.369017 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:54:01.369194 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:54:01.369283 2614 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:54:01.369555 2702 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260503 19:54:02.274741 2708 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:54:02.274945 2708 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:54:02.275588 2708 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:54:02.278220 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:54:02.278508 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:54:02.278601 2614 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:54:02.278932 2708 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260503 19:54:03.234428 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:54:03.234685 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:54:04.676115 2716 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:54:04.676335 2716 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:54:04.676962 2716 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:54:04.678170 2614 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20260503 19:54:04.679517 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:54:04.679756 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:54:04.679850 2614 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:54:04.680153 2716 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260503 19:54:06.726625 2725 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:54:06.727073 2725 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:54:06.727837 2725 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:54:06.730237 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:54:06.730456 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:54:06.730579 2614 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:54:06.730824 2725 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260503 19:54:08.236259 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:54:08.236445 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:54:13.238173 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:54:13.238368 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:54:14.282431 2746 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:54:14.282694 2746 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:54:14.283408 2746 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:54:14.284559 2614 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260503 19:54:14.286270 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:54:14.286486 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:54:14.286625 2614 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:54:14.287031 2746 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260503 19:54:18.239995 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:54:18.240240 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:54:18.931074 2758 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:54:18.931376 2758 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:54:18.932178 2758 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:54:18.934756 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:54:18.934955 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:54:18.935032 2614 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:54:18.935261 2758 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260503 19:54:23.241840 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:54:23.242060 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:54:28.243770 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:54:28.243997 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260503 19:54:30.620180 1256 heartbeater.cc:646] Failed to heartbeat to 127.0.105.60:42203 (69 consecutive failures): Network error: Failed to ping master at 127.0.105.60:42203: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
W20260503 19:54:30.646894 1394 heartbeater.cc:646] Failed to heartbeat to 127.0.105.60:42203 (69 consecutive failures): Network error: Failed to ping master at 127.0.105.60:42203: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
W20260503 19:54:30.652112 1115 heartbeater.cc:646] Failed to heartbeat to 127.0.105.60:42203 (69 consecutive failures): Network error: Failed to ping master at 127.0.105.60:42203: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:54:33.246011 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:54:33.246269 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:54:36.439550 2793 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:54:36.439818 2793 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:54:36.440678 2793 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:54:36.442267 2614 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260503 19:54:36.446437 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:54:36.446705 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:54:36.446810 2614 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:54:36.447131 2793 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260503 19:54:38.247874 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:54:38.248118 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:54:41.542636 2806 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:54:41.542830 2806 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:54:41.543558 2806 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:54:41.544669 2614 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260503 19:54:41.546041 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:54:41.546275 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:54:41.546353 2614 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:54:41.546592 2806 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260503 19:54:43.249719 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:54:43.249912 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:54:46.388721 2819 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:54:46.388993 2819 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:54:46.389752 2819 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:54:46.394348 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:54:46.394579 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:54:46.394711 2614 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:54:46.395001 2819 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260503 19:54:48.251478 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:54:48.251690 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:54:53.253440 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:54:53.253661 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:54:58.255301 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:54:58.255504 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:55:03.257314 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:55:03.257543 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:55:05.197723 2857 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:55:05.197937 2857 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:55:05.198916 2857 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:55:05.200392 2614 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260503 19:55:05.202042 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:55:05.202288 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:55:05.202421 2614 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:55:05.202718 2857 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260503 19:55:08.259284 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:55:08.260046 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:55:11.431254 2871 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:55:11.431519 2871 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:55:11.432205 2871 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:55:11.433317 2614 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260503 19:55:11.434706 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:55:11.434926 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:55:11.435065 2614 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:55:11.435389 2871 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260503 19:55:13.261700 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:55:13.261960 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:55:18.268869 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:55:18.269131 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:55:19.132012 2888 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:55:19.132208 2888 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:55:19.132820 2888 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:55:19.134096 2614 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260503 19:55:19.137458 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:55:19.137686 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:55:19.137804 2614 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:55:19.138080 2888 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260503 19:55:23.270535 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:55:23.270753 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:55:28.272411 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:55:28.272639 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:55:28.406827 2909 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:55:28.407027 2909 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:55:28.407680 2909 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:55:28.408347 2614 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260503 19:55:28.409776 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:55:28.409971 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:55:28.410102 2614 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:55:28.410377 2909 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260503 19:55:30.776788 1256 heartbeater.cc:646] Failed to heartbeat to 127.0.105.60:42203 (189 consecutive failures): Network error: Failed to ping master at 127.0.105.60:42203: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
W20260503 19:55:30.802377 1115 heartbeater.cc:646] Failed to heartbeat to 127.0.105.60:42203 (189 consecutive failures): Network error: Failed to ping master at 127.0.105.60:42203: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
W20260503 19:55:30.802464 1394 heartbeater.cc:646] Failed to heartbeat to 127.0.105.60:42203 (189 consecutive failures): Network error: Failed to ping master at 127.0.105.60:42203: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:55:33.274751 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:55:33.274969 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:55:38.276769 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:55:38.276993 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:55:43.278746 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:55:43.279008 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:55:45.179768 2937 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:55:45.179965 2937 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:55:45.180765 2937 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:55:45.182204 2614 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260503 19:55:45.186496 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:55:45.186728 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:55:45.186856 2614 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:55:45.187170 2937 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260503 19:55:48.280700 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:55:48.280915 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:55:53.282877 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:55:53.283104 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:55:58.285405 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:55:58.285830 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:55:58.306366 2619 maintenance_manager.cc:419] P 2ba485a545f44fdd8217c3f86c79d0ca: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033359
I20260503 19:55:58.358367 2618 maintenance_manager.cc:643] P 2ba485a545f44fdd8217c3f86c79d0ca: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.051s user 0.029s sys 0.000s Metrics: {"bytes_written":15260,"cfile_init":1,"dirs.queue_time_us":472,"dirs.run_cpu_time_us":241,"dirs.run_wall_time_us":759,"drs_written":1,"lbm_read_time_us":68,"lbm_reads_lt_1ms":4,"lbm_write_time_us":714,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":33,"thread_start_us":574,"threads_started":2,"wal-append.queue_time_us":412}
I20260503 19:55:58.359737 2619 maintenance_manager.cc:419] P 2ba485a545f44fdd8217c3f86c79d0ca: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1644 bytes on disk
I20260503 19:55:58.360514 2618 maintenance_manager.cc:643] P 2ba485a545f44fdd8217c3f86c79d0ca: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":73,"lbm_reads_lt_1ms":4}
I20260503 19:56:03.287658 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:56:03.287923 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:56:04.033813 2979 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:56:04.034082 2979 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:56:04.035020 2979 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:56:04.036535 2614 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260503 19:56:04.039202 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:56:04.039408 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:56:04.039531 2614 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:56:04.039849 2979 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260503 19:56:08.289722 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:56:08.290017 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:56:13.291725 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:56:13.291949 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:56:14.233355 3025 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:56:14.233577 3025 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:56:14.234262 3025 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:56:14.235582 2614 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260503 19:56:14.237186 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:56:14.237392 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:56:14.237520 2614 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:56:14.237844 3025 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260503 19:56:18.293678 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:56:18.293913 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:56:23.295724 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:56:23.295956 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:56:28.298063 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:56:28.298326 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:56:29.363029 3089 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:56:29.363303 3089 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:56:29.364166 3089 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:56:29.365571 2614 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260503 19:56:29.370705 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:56:29.370955 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:56:29.371099 2614 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:56:29.371482 3089 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260503 19:56:30.932996 1256 heartbeater.cc:646] Failed to heartbeat to 127.0.105.60:42203 (309 consecutive failures): Network error: Failed to ping master at 127.0.105.60:42203: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
W20260503 19:56:30.964387 1394 heartbeater.cc:646] Failed to heartbeat to 127.0.105.60:42203 (309 consecutive failures): Network error: Failed to ping master at 127.0.105.60:42203: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
W20260503 19:56:30.964421 1115 heartbeater.cc:646] Failed to heartbeat to 127.0.105.60:42203 (309 consecutive failures): Network error: Failed to ping master at 127.0.105.60:42203: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:56:33.299968 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:56:33.300191 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:56:38.302752 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:56:38.302968 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:56:43.304690 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:56:43.304957 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:56:47.901468 3161 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:56:47.901700 3161 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:56:47.902619 3161 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:56:47.904109 2614 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260503 19:56:47.906224 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:56:47.906443 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:56:47.906565 2614 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:56:47.906875 3161 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260503 19:56:48.306294 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:56:48.306576 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:56:51.727031 3178 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:56:51.727296 3178 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:56:51.727922 3178 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:56:51.731817 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:56:51.732086 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:56:51.732197 2614 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:56:51.732496 3178 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260503 19:56:53.308375 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:56:53.308862 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:56:55.822371 3198 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:56:55.822574 3198 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:56:55.823289 3198 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:56:55.824682 2614 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260503 19:56:55.826292 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:56:55.826517 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:56:55.826612 2614 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:56:55.826839 3198 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260503 19:56:58.310612 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:56:58.310858 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:57:01.406046 3224 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:57:01.406236 3224 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:57:01.406981 3224 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:57:01.408128 2614 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260503 19:57:01.409492 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:57:01.409670 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:57:01.409742 2614 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:57:01.409983 3224 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260503 19:57:03.312567 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:57:03.312922 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:57:08.314381 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:57:08.314613 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:57:13.317832 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:57:13.318051 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:57:18.301965 3286 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:57:18.302372 3286 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:57:18.303622 3286 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:57:18.313230 2614 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260503 19:57:18.314822 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:57:18.315096 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:57:18.315217 2614 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:57:18.315512 3286 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260503 19:57:18.319088 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:57:18.319264 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:57:23.320945 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:57:23.321233 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:57:28.322721 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:57:28.322942 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:57:30.834172 3326 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:57:30.834365 3326 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:57:30.835105 3326 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:57:30.836634 2614 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260503 19:57:30.838236 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:57:30.838476 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:57:30.838564 2614 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:57:30.839004 3326 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260503 19:57:31.091148 1256 heartbeater.cc:646] Failed to heartbeat to 127.0.105.60:42203 (429 consecutive failures): Network error: Failed to ping master at 127.0.105.60:42203: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
W20260503 19:57:31.124496 1115 heartbeater.cc:646] Failed to heartbeat to 127.0.105.60:42203 (429 consecutive failures): Network error: Failed to ping master at 127.0.105.60:42203: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
W20260503 19:57:31.124945 1394 heartbeater.cc:646] Failed to heartbeat to 127.0.105.60:42203 (429 consecutive failures): Network error: Failed to ping master at 127.0.105.60:42203: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:57:33.324563 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:57:33.324795 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:57:38.326534 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:57:38.326766 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:57:43.328378 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:57:43.328562 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:57:48.330021 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:57:48.330271 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:57:48.529052 3393 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:57:48.529415 3393 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:57:48.530438 3393 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:57:48.531373 2614 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260503 19:57:48.533051 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:57:48.533381 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:57:48.533496 2614 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:57:48.533802 3393 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260503 19:57:53.331957 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:57:53.332149 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:57:58.333674 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:57:58.333863 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:58:03.335428 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:58:03.335614 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:58:06.997524 3453 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:58:06.997747 3453 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:58:06.998526 3453 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:58:07.000339 2614 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260503 19:58:07.002826 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:58:07.003095 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:58:07.003221 2614 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:58:07.003543 3453 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260503 19:58:08.337824 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:58:08.338081 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:58:13.339573 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:58:13.339814 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:58:18.341498 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:58:18.341722 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:58:21.599800 3509 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:58:21.600016 3509 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:58:21.600755 3509 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:58:21.602053 2614 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260503 19:58:21.603428 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:58:21.603655 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:58:21.603752 2614 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:58:21.604000 3509 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260503 19:58:23.343734 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:58:23.343974 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:58:28.345769 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:58:28.345991 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260503 19:58:31.248436 1256 heartbeater.cc:646] Failed to heartbeat to 127.0.105.60:42203 (549 consecutive failures): Network error: Failed to ping master at 127.0.105.60:42203: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
W20260503 19:58:31.286376 1394 heartbeater.cc:646] Failed to heartbeat to 127.0.105.60:42203 (549 consecutive failures): Network error: Failed to ping master at 127.0.105.60:42203: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
W20260503 19:58:31.286375 1115 heartbeater.cc:646] Failed to heartbeat to 127.0.105.60:42203 (549 consecutive failures): Network error: Failed to ping master at 127.0.105.60:42203: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:58:31.350206 3548 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:58:31.350418 3548 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:58:31.351042 3548 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:58:31.352416 2614 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260503 19:58:31.354467 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:58:31.354712 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:58:31.354871 2614 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:58:31.355211 3548 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260503 19:58:33.347726 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:58:33.347926 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:58:38.349606 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:58:38.349795 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:58:43.351869 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:58:43.352069 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:58:48.353763 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:58:48.354048 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260503 19:58:49.676898 3609 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260503 19:58:49.677212 3609 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3a96e537cec46df958468878509b9f9" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33821 } } peers { permanent_uuid: "2ba485a545f44fdd8217c3f86c79d0ca" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 35345 } } peers { permanent_uuid: "b0cad11e600c48a3af9a3c56115696d5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 42203 } }
I20260503 19:58:49.678165 3609 leader_election.cc:290] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821), b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203)
W20260503 19:58:49.679662 2614 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260503 19:58:49.682996 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3a96e537cec46df958468878509b9f9 (127.0.105.62:33821): Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111)
W20260503 19:58:49.683257 2614 leader_election.cc:336] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b0cad11e600c48a3af9a3c56115696d5 (127.0.105.60:42203): Network error: Client connection negotiation failed: client connection to 127.0.105.60:42203: connect: Connection refused (error 111)
I20260503 19:58:49.683350 2614 leader_election.cc:304] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ba485a545f44fdd8217c3f86c79d0ca; no voters: b0cad11e600c48a3af9a3c56115696d5, f3a96e537cec46df958468878509b9f9
I20260503 19:58:49.683703 3609 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 2ba485a545f44fdd8217c3f86c79d0ca [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260503 19:58:51.385784 2619 maintenance_manager.cc:419] P 2ba485a545f44fdd8217c3f86c79d0ca: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1644 bytes on disk
I20260503 19:58:51.392050 2618 maintenance_manager.cc:643] P 2ba485a545f44fdd8217c3f86c79d0ca: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.006s user 0.006s sys 0.000s Metrics: {"dirs.queue_time_us":363,"dirs.run_cpu_time_us":137,"dirs.run_wall_time_us":137,"thread_start_us":577,"threads_started":2,"wal-append.queue_time_us":414}
I20260503 19:58:53.355674 2598 master_runner.cc:186] Error getting master registration for 127.0.105.62:33821: Network error: Client connection negotiation failed: client connection to 127.0.105.62:33821: connect: Connection refused (error 111),
I20260503 19:58:53.356384 2598 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260503 19:58:56.011281 1404 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)
W20260503 19:58:56.011543 1404 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)
W20260503 19:58:56.011632 1404 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)
W20260503 19:58:56.011736 1404 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)
F20260503 19:58:56.022641 1436 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)
*** Check failure stack trace: ***
*** Aborted at 1777838336 (unix time) try "date -d @1777838336" if you are using GNU date ***
F20260503 19:58:56.023970 1412 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8a28426e04cf42b8ae2288a7c2950b1b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)
*** Check failure stack trace: ***
F20260503 19:58:56.023970 1412 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8a28426e04cf42b8ae2288a7c2950b1b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025200 1438 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)
*** Check failure stack trace: ***
*** Check failure stack trace: ***
F20260503 19:58:56.023970 1412 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8a28426e04cf42b8ae2288a7c2950b1b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025200 1438 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025446 1434 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025494 1427 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)
*** Check failure stack trace: ***
F20260503 19:58:56.023970 1412 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8a28426e04cf42b8ae2288a7c2950b1b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025200 1438 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025446 1434 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025494 1427 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025976 1416 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ad6e238ff40f4be99d7cc2d556a40a6b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)
*** Check failure stack trace: ***
F20260503 19:58:56.023970 1412 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8a28426e04cf42b8ae2288a7c2950b1b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025200 1438 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025446 1434 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025494 1427 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025976 1416 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ad6e238ff40f4be99d7cc2d556a40a6b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.026080 1414 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ba1a53aa8c174babb4bc2b60b558a8c1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)
*** Check failure stack trace: ***
F20260503 19:58:56.023970 1412 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8a28426e04cf42b8ae2288a7c2950b1b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025200 1438 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025446 1434 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025494 1427 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025976 1416 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ad6e238ff40f4be99d7cc2d556a40a6b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.026080 1414 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ba1a53aa8c174babb4bc2b60b558a8c1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027262 1419 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f28cafcdb9394779ac1e1d270bab59fb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)
*** Check failure stack trace: ***
F20260503 19:58:56.023970 1412 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8a28426e04cf42b8ae2288a7c2950b1b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025200 1438 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025446 1434 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025494 1427 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025976 1416 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ad6e238ff40f4be99d7cc2d556a40a6b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.026080 1414 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ba1a53aa8c174babb4bc2b60b558a8c1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027262 1419 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f28cafcdb9394779ac1e1d270bab59fb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027302 1428 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)
*** Check failure stack trace: ***
F20260503 19:58:56.023970 1412 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8a28426e04cf42b8ae2288a7c2950b1b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025200 1438 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025446 1434 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025494 1427 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025976 1416 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ad6e238ff40f4be99d7cc2d556a40a6b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.026080 1414 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ba1a53aa8c174babb4bc2b60b558a8c1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027262 1419 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f28cafcdb9394779ac1e1d270bab59fb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027302 1428 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027299 1415 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b578e2eeb11a4b698a7b51e206e1d015 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)
*** Check failure stack trace: ***
F20260503 19:58:56.023970 1412 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8a28426e04cf42b8ae2288a7c2950b1b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025200 1438 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025446 1434 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025494 1427 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025976 1416 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ad6e238ff40f4be99d7cc2d556a40a6b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.026080 1414 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ba1a53aa8c174babb4bc2b60b558a8c1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027262 1419 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f28cafcdb9394779ac1e1d270bab59fb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027302 1428 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027299 1415 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b578e2eeb11a4b698a7b51e206e1d015 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027734 1432 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)
*** Check failure stack trace: ***
F20260503 19:58:56.023970 1412 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8a28426e04cf42b8ae2288a7c2950b1b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025200 1438 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025446 1434 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025494 1427 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025976 1416 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ad6e238ff40f4be99d7cc2d556a40a6b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.026080 1414 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ba1a53aa8c174babb4bc2b60b558a8c1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027262 1419 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f28cafcdb9394779ac1e1d270bab59fb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027302 1428 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027299 1415 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b578e2eeb11a4b698a7b51e206e1d015 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027734 1432 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027954 1418 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fa0eb6f31c344d07a5532aced2019227 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)
*** Check failure stack trace: ***
F20260503 19:58:56.023970 1412 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8a28426e04cf42b8ae2288a7c2950b1b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025200 1438 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025446 1434 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025494 1427 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025976 1416 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ad6e238ff40f4be99d7cc2d556a40a6b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.026080 1414 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ba1a53aa8c174babb4bc2b60b558a8c1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027262 1419 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f28cafcdb9394779ac1e1d270bab59fb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027302 1428 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027299 1415 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b578e2eeb11a4b698a7b51e206e1d015 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027734 1432 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027954 1418 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fa0eb6f31c344d07a5532aced2019227 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028113 1425 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)
*** Check failure stack trace: ***
F20260503 19:58:56.023970 1412 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8a28426e04cf42b8ae2288a7c2950b1b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025200 1438 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025446 1434 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025494 1427 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025976 1416 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ad6e238ff40f4be99d7cc2d556a40a6b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.026080 1414 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ba1a53aa8c174babb4bc2b60b558a8c1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027262 1419 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f28cafcdb9394779ac1e1d270bab59fb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027302 1428 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027299 1415 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b578e2eeb11a4b698a7b51e206e1d015 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027734 1432 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027954 1418 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fa0eb6f31c344d07a5532aced2019227 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028113 1425 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028539 1422 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)
*** Check failure stack trace: ***
F20260503 19:58:56.023970 1412 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8a28426e04cf42b8ae2288a7c2950b1b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025200 1438 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025446 1434 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025494 1427 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025976 1416 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ad6e238ff40f4be99d7cc2d556a40a6b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.026080 1414 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ba1a53aa8c174babb4bc2b60b558a8c1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027262 1419 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f28cafcdb9394779ac1e1d270bab59fb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027302 1428 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027299 1415 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b578e2eeb11a4b698a7b51e206e1d015 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027734 1432 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027954 1418 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fa0eb6f31c344d07a5532aced2019227 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028113 1425 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028539 1422 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028620 1423 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)
*** Check failure stack trace: ***
F20260503 19:58:56.023970 1412 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8a28426e04cf42b8ae2288a7c2950b1b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025200 1438 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025446 1434 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025494 1427 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025976 1416 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ad6e238ff40f4be99d7cc2d556a40a6b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.026080 1414 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ba1a53aa8c174babb4bc2b60b558a8c1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027262 1419 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f28cafcdb9394779ac1e1d270bab59fb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027302 1428 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027299 1415 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b578e2eeb11a4b698a7b51e206e1d015 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027734 1432 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027954 1418 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fa0eb6f31c344d07a5532aced2019227 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028113 1425 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028539 1422 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028620 1423 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028690 1413 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_3f2252b7f4da4d9a9752ecd7ce54019f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)
*** Check failure stack trace: ***
PC: @ 0x0 (unknown)
F20260503 19:58:56.023970 1412 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8a28426e04cf42b8ae2288a7c2950b1b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025200 1438 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025446 1434 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025494 1427 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025976 1416 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ad6e238ff40f4be99d7cc2d556a40a6b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.026080 1414 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ba1a53aa8c174babb4bc2b60b558a8c1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027262 1419 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f28cafcdb9394779ac1e1d270bab59fb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027302 1428 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027299 1415 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b578e2eeb11a4b698a7b51e206e1d015 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027734 1432 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027954 1418 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fa0eb6f31c344d07a5532aced2019227 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028113 1425 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028539 1422 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028620 1423 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028690 1413 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_3f2252b7f4da4d9a9752ecd7ce54019f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.029892 1424 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)
*** Check failure stack trace: ***
F20260503 19:58:56.023970 1412 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8a28426e04cf42b8ae2288a7c2950b1b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025200 1438 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025446 1434 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025494 1427 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025976 1416 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ad6e238ff40f4be99d7cc2d556a40a6b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.026080 1414 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ba1a53aa8c174babb4bc2b60b558a8c1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027262 1419 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f28cafcdb9394779ac1e1d270bab59fb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027302 1428 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027299 1415 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b578e2eeb11a4b698a7b51e206e1d015 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027734 1432 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027954 1418 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fa0eb6f31c344d07a5532aced2019227 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028113 1425 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028539 1422 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028620 1423 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028690 1413 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_3f2252b7f4da4d9a9752ecd7ce54019f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.029892 1424 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.029958 1429 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)
*** SIGABRT (@0x3e8000001a4) received by PID 420 (TID 0x7f8960e4b700) from PID 420; stack trace: ***
*** Check failure stack trace: ***
F20260503 19:58:56.023970 1412 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8a28426e04cf42b8ae2288a7c2950b1b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025200 1438 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025446 1434 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025494 1427 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025976 1416 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ad6e238ff40f4be99d7cc2d556a40a6b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.026080 1414 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ba1a53aa8c174babb4bc2b60b558a8c1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027262 1419 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f28cafcdb9394779ac1e1d270bab59fb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027302 1428 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027299 1415 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b578e2eeb11a4b698a7b51e206e1d015 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027734 1432 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027954 1418 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fa0eb6f31c344d07a5532aced2019227 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028113 1425 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028539 1422 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028620 1423 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028690 1413 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_3f2252b7f4da4d9a9752ecd7ce54019f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.029892 1424 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.029958 1429 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.030045 1417 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_49084d26aae440ac958d415c2b291948 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)
*** Check failure stack trace: ***
F20260503 19:58:56.023970 1412 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8a28426e04cf42b8ae2288a7c2950b1b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025200 1438 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025446 1434 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025494 1427 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025976 1416 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ad6e238ff40f4be99d7cc2d556a40a6b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.026080 1414 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ba1a53aa8c174babb4bc2b60b558a8c1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027262 1419 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f28cafcdb9394779ac1e1d270bab59fb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027302 1428 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027299 1415 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b578e2eeb11a4b698a7b51e206e1d015 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027734 1432 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027954 1418 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fa0eb6f31c344d07a5532aced2019227 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028113 1425 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028539 1422 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028620 1423 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028690 1413 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_3f2252b7f4da4d9a9752ecd7ce54019f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.029892 1424 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.029958 1429 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.030045 1417 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_49084d26aae440ac958d415c2b291948 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.030510 1420 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1eff9f94b0c84e7180c54349d66db8e4 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)
*** Check failure stack trace: ***
F20260503 19:58:56.023970 1412 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8a28426e04cf42b8ae2288a7c2950b1b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025200 1438 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025446 1434 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025494 1427 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025976 1416 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ad6e238ff40f4be99d7cc2d556a40a6b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.026080 1414 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ba1a53aa8c174babb4bc2b60b558a8c1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027262 1419 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f28cafcdb9394779ac1e1d270bab59fb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027302 1428 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027299 1415 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b578e2eeb11a4b698a7b51e206e1d015 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027734 1432 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027954 1418 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fa0eb6f31c344d07a5532aced2019227 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028113 1425 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028539 1422 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028620 1423 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028690 1413 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_3f2252b7f4da4d9a9752ecd7ce54019f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.029892 1424 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.029958 1429 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.030045 1417 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_49084d26aae440ac958d415c2b291948 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.030510 1420 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1eff9f94b0c84e7180c54349d66db8e4 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.030565 1426 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)
*** Check failure stack trace: ***
F20260503 19:58:56.023970 1412 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8a28426e04cf42b8ae2288a7c2950b1b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025200 1438 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025446 1434 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025494 1427 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025976 1416 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ad6e238ff40f4be99d7cc2d556a40a6b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.026080 1414 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ba1a53aa8c174babb4bc2b60b558a8c1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027262 1419 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f28cafcdb9394779ac1e1d270bab59fb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027302 1428 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027299 1415 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b578e2eeb11a4b698a7b51e206e1d015 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027734 1432 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027954 1418 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fa0eb6f31c344d07a5532aced2019227 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028113 1425 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028539 1422 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028620 1423 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028690 1413 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_3f2252b7f4da4d9a9752ecd7ce54019f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.029892 1424 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.029958 1429 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.030045 1417 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_49084d26aae440ac958d415c2b291948 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.030510 1420 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1eff9f94b0c84e7180c54349d66db8e4 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.030565 1426 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.030586 1430 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)
*** Check failure stack trace: ***
@ 0x7f897fadc980 (unknown) at ??:0
F20260503 19:58:56.023970 1412 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8a28426e04cf42b8ae2288a7c2950b1b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025200 1438 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025446 1434 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025494 1427 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025976 1416 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ad6e238ff40f4be99d7cc2d556a40a6b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.026080 1414 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ba1a53aa8c174babb4bc2b60b558a8c1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027262 1419 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f28cafcdb9394779ac1e1d270bab59fb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027302 1428 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027299 1415 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b578e2eeb11a4b698a7b51e206e1d015 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027734 1432 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027954 1418 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fa0eb6f31c344d07a5532aced2019227 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028113 1425 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028539 1422 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028620 1423 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028690 1413 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_3f2252b7f4da4d9a9752ecd7ce54019f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.029892 1424 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.029958 1429 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.030045 1417 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_49084d26aae440ac958d415c2b291948 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.030510 1420 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1eff9f94b0c84e7180c54349d66db8e4 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.030565 1426 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.030586 1430 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.032083 1433 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)
*** Check failure stack trace: ***
F20260503 19:58:56.023970 1412 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8a28426e04cf42b8ae2288a7c2950b1b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025200 1438 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025446 1434 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025494 1427 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.025976 1416 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ad6e238ff40f4be99d7cc2d556a40a6b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.026080 1414 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ba1a53aa8c174babb4bc2b60b558a8c1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027262 1419 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_f28cafcdb9394779ac1e1d270bab59fb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027302 1428 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027299 1415 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b578e2eeb11a4b698a7b51e206e1d015 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027734 1432 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.027954 1418 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fa0eb6f31c344d07a5532aced2019227 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028113 1425 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028539 1422 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028620 1423 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.028690 1413 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_3f2252b7f4da4d9a9752ecd7ce54019f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.029892 1424 master-stress-test.cc:328] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.029958 1429 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.030045 1417 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_49084d26aae440ac958d415c2b291948 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.030510 1420 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1eff9f94b0c84e7180c54349d66db8e4 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.030565 1426 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.030586 1430 master-stress-test.cc:351] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.032083 1433 master-stress-test.cc:391] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)F20260503 19:58:56.032226 1421 master-stress-test.cc:297] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e9dee4f0693a4aca81d75d2c25cefd09 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 769) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.105.62:33821,127.0.105.61:35345,127.0.105.60:42203, num_attempts: 1)
*** Check failure stack trace: ***
@ 0x7f89788cce87 gsignal at ??:0
@ 0x7f89788ce7f1 abort at ??:0
@ 0x7f897b606dcd google::LogMessage::Fail() at ??:0
@ 0x7f897b60ab93 google::LogMessage::SendToLog() at ??:0
@ 0x7f897b6067cc google::LogMessage::Flush() at ??:0
@ 0x7f897b607f59 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x40d9f5 kudu::MasterStressTest::ReplaceTabletThread() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/master-stress-test.cc:391
@ 0x44aaf4 std::__invoke_impl<>() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/invoke.h:73
@ 0x44a9ad std::__invoke<>() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/invoke.h:95
@ 0x44a96a _ZNSt6thread8_InvokerISt5tupleIJMN4kudu16MasterStressTestEFvvEPNS2_26MasterStressTest_Test_TestEEEE9_M_invokeIJLm0ELm1EEEEDTclsr3stdE8__invokespcl10_S_declvalIXT_EEEEESt12_Index_tupleIJXspT_EEE at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:234
@ 0x44a91e std::thread::_Invoker<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:243
@ 0x44a659 std::thread::_State_impl<>::_M_run() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:186
@ 0x7f8978f546df (unknown) at ??:0
@ 0x7f897fad16db start_thread at ??:0
@ 0x7f89789af61f clone at ??:0