Note: This is test shard 1 of 3.
[==========] Running 2 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 2 tests from CatalogManagerConfigurations/MasterStressTest
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/0
2026-09-25T13:55:53Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:55:53Z Disabled control of system clock
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260925 13:55:53.545296 19163 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskYmJttM/build/tsan/bin/kudu
/tmp/dist-test-taskYmJttM/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.182.254:40825
--webserver_interface=127.18.182.254
--webserver_port=0
--builtin_ntp_servers=127.18.182.212:43091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.182.254:40825,127.18.182.253:39641,127.18.182.252:36181
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260925 13:55:53.848682 19172 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:55:53.849249 19172 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:55:53.849609 19172 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260925 13:55:53.882424 19172 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:55:53.882711 19172 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:55:53.882918 19172 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:55:53.883145 19172 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:55:53.920333 19172 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.182.212:43091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-0/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.18.182.254:40825,127.18.182.253:39641,127.18.182.252:36181
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.182.254:40825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.182.254
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 102683f65af844d269565345d7d8c6ebc55c92c5
build type FASTDEBUG
built by None at 25 Sep 2026 13:43:25 UTC on e7f111948823
build id 13994
TSAN enabled
I20260925 13:55:53.921555 19172 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:55:53.923089 19172 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260925 13:55:53.934904 19179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:55:53.935727 19178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:55:53.938462 19181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:55:55.162341 19180 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1224 milliseconds
I20260925 13:55:55.162451 19172 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260925 13:55:55.163769 19172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:55:55.167968 19172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:55:55.169390 19172 hybrid_clock.cc:648] HybridClock initialized: now 1790344555169348 us; error 61 us; skew 500 ppm
I20260925 13:55:55.170152 19172 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 13:55:55.176340 19172 webserver.cc:533] Webserver started at http://127.18.182.254:44181/ using document root <none> and password file <none>
I20260925 13:55:55.177202 19172 fs_manager.cc:362] Metadata directory not provided
I20260925 13:55:55.177382 19172 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:55:55.177763 19172 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:55:55.181888 19172 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-0/data/instance:
uuid: "7dcc50a8531144a69ab0cc00df1f0df4"
format_stamp: "Formatted at 2026-09-25 13:55:55 on dist-test-slave-nlqf"
I20260925 13:55:55.182871 19172 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-0/wal/instance:
uuid: "7dcc50a8531144a69ab0cc00df1f0df4"
format_stamp: "Formatted at 2026-09-25 13:55:55 on dist-test-slave-nlqf"
I20260925 13:55:55.189888 19172 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260925 13:55:55.195691 19188 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:55:55.196780 19172 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 13:55:55.197134 19172 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-0/data,/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-0/wal
uuid: "7dcc50a8531144a69ab0cc00df1f0df4"
format_stamp: "Formatted at 2026-09-25 13:55:55 on dist-test-slave-nlqf"
I20260925 13:55:55.197489 19172 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:55:55.247117 19172 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260925 13:55:55.249246 19172 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:55:55.249755 19172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:55:55.320160 19172 rpc_server.cc:307] RPC server started. Bound to: 127.18.182.254:40825
I20260925 13:55:55.320259 19239 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.182.254:40825 every 8 connection(s)
I20260925 13:55:55.322875 19172 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-0/data/info.pb
I20260925 13:55:55.326169 19163 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskYmJttM/build/tsan/bin/kudu as pid 19172
I20260925 13:55:55.326692 19163 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-0/wal/instance
I20260925 13:55:55.329985 19240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:55:55.332129 19163 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskYmJttM/build/tsan/bin/kudu
/tmp/dist-test-taskYmJttM/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.182.253:39641
--webserver_interface=127.18.182.253
--webserver_port=0
--builtin_ntp_servers=127.18.182.212:43091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.182.254:40825,127.18.182.253:39641,127.18.182.252:36181
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260925 13:55:55.342976 19240 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:55:55.385915 19240 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } has no permanent_uuid. Determining permanent_uuid...
W20260925 13:55:55.388981 19190 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.182.253:39641: connect: Connection refused (error 111)
W20260925 13:55:55.393327 19240 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.253:39641: Network error: Client connection negotiation failed: client connection to 127.18.182.253:39641: connect: Connection refused (error 111)
I20260925 13:55:55.442823 19240 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } attempt: 1
W20260925 13:55:55.446820 19240 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.253:39641: Network error: Client connection negotiation failed: client connection to 127.18.182.253:39641: connect: Connection refused (error 111)
I20260925 13:55:55.515210 19240 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } attempt: 2
W20260925 13:55:55.519502 19240 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.253:39641: Network error: Client connection negotiation failed: client connection to 127.18.182.253:39641: connect: Connection refused (error 111)
I20260925 13:55:55.610885 19240 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } attempt: 3
W20260925 13:55:55.615022 19240 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.253:39641: Network error: Client connection negotiation failed: client connection to 127.18.182.253:39641: connect: Connection refused (error 111)
W20260925 13:55:55.644073 19242 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:55:55.644724 19242 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:55:55.645186 19242 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260925 13:55:55.678661 19242 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:55:55.679035 19242 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:55:55.679359 19242 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:55:55.679636 19242 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:55:55.717337 19242 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.182.212:43091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-1/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.18.182.254:40825,127.18.182.253:39641,127.18.182.252:36181
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.182.253:39641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.182.253
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 102683f65af844d269565345d7d8c6ebc55c92c5
build type FASTDEBUG
built by None at 25 Sep 2026 13:43:25 UTC on e7f111948823
build id 13994
TSAN enabled
I20260925 13:55:55.718730 19242 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:55:55.720613 19242 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260925 13:55:55.732403 19251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:55:55.758424 19240 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } attempt: 4
W20260925 13:55:55.766100 19240 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.253:39641: Network error: Client connection negotiation failed: client connection to 127.18.182.253:39641: connect: Connection refused (error 111)
I20260925 13:55:56.065670 19240 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } attempt: 5
W20260925 13:55:56.073089 19240 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.253:39641: Network error: Client connection negotiation failed: client connection to 127.18.182.253:39641: connect: Connection refused (error 111)
I20260925 13:55:56.620651 19240 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } attempt: 6
W20260925 13:55:56.650925 19240 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.253:39641: Network error: Client connection negotiation failed: client connection to 127.18.182.253:39641: connect: Connection refused (error 111)
W20260925 13:55:55.732896 19252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:55:57.136950 19250 debug-util.cc:398] Leaking SignalData structure 0x7b08000379a0 after lost signal to thread 19242
W20260925 13:55:57.395980 19242 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.664s user 0.594s sys 1.065s
W20260925 13:55:57.397166 19242 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.665s user 0.594s sys 1.065s
W20260925 13:55:57.398240 19253 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1665 milliseconds
I20260925 13:55:57.398330 19242 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
W20260925 13:55:57.398396 19254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:55:57.402405 19242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:55:57.404875 19242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:55:57.406301 19242 hybrid_clock.cc:648] HybridClock initialized: now 1790344557406262 us; error 40 us; skew 500 ppm
I20260925 13:55:57.407230 19242 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 13:55:57.413851 19242 webserver.cc:533] Webserver started at http://127.18.182.253:33169/ using document root <none> and password file <none>
I20260925 13:55:57.414891 19242 fs_manager.cc:362] Metadata directory not provided
I20260925 13:55:57.415148 19242 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:55:57.415707 19242 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:55:57.420252 19242 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-1/data/instance:
uuid: "bcc92728d49643efab5528d77c3d7828"
format_stamp: "Formatted at 2026-09-25 13:55:57 on dist-test-slave-nlqf"
I20260925 13:55:57.421391 19242 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-1/wal/instance:
uuid: "bcc92728d49643efab5528d77c3d7828"
format_stamp: "Formatted at 2026-09-25 13:55:57 on dist-test-slave-nlqf"
I20260925 13:55:57.428610 19242 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260925 13:55:57.434134 19263 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:55:57.435222 19242 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260925 13:55:57.435575 19242 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-1/data,/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-1/wal
uuid: "bcc92728d49643efab5528d77c3d7828"
format_stamp: "Formatted at 2026-09-25 13:55:57 on dist-test-slave-nlqf"
I20260925 13:55:57.435925 19242 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:55:57.483052 19242 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260925 13:55:57.484529 19242 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:55:57.484985 19242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:55:57.555668 19242 rpc_server.cc:307] RPC server started. Bound to: 127.18.182.253:39641
I20260925 13:55:57.555809 19314 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.182.253:39641 every 8 connection(s)
I20260925 13:55:57.558553 19242 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-1/data/info.pb
I20260925 13:55:57.565331 19315 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:55:57.566233 19163 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskYmJttM/build/tsan/bin/kudu as pid 19242
I20260925 13:55:57.566704 19163 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-1/wal/instance
I20260925 13:55:57.569875 19163 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskYmJttM/build/tsan/bin/kudu
/tmp/dist-test-taskYmJttM/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.182.252:36181
--webserver_interface=127.18.182.252
--webserver_port=0
--builtin_ntp_servers=127.18.182.212:43091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.182.254:40825,127.18.182.253:39641,127.18.182.252:36181
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260925 13:55:57.579479 19315 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:55:57.600711 19315 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:55:57.614724 19315 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } has no permanent_uuid. Determining permanent_uuid...
W20260925 13:55:57.617543 19266 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.182.252:36181: connect: Connection refused (error 111)
W20260925 13:55:57.621999 19315 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.252:36181: Network error: Client connection negotiation failed: client connection to 127.18.182.252:36181: connect: Connection refused (error 111)
I20260925 13:55:57.671521 19315 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } attempt: 1
W20260925 13:55:57.676164 19315 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.252:36181: Network error: Client connection negotiation failed: client connection to 127.18.182.252:36181: connect: Connection refused (error 111)
I20260925 13:55:57.711503 19240 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } attempt: 7
I20260925 13:55:57.724557 19240 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } has no permanent_uuid. Determining permanent_uuid...
W20260925 13:55:57.730178 19240 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.252:36181: Network error: Client connection negotiation failed: client connection to 127.18.182.252:36181: connect: Connection refused (error 111)
I20260925 13:55:57.744563 19315 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } attempt: 2
W20260925 13:55:57.749066 19315 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.252:36181: Network error: Client connection negotiation failed: client connection to 127.18.182.252:36181: connect: Connection refused (error 111)
I20260925 13:55:57.788609 19240 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } attempt: 1
W20260925 13:55:57.792943 19240 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.252:36181: Network error: Client connection negotiation failed: client connection to 127.18.182.252:36181: connect: Connection refused (error 111)
I20260925 13:55:57.840605 19315 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } attempt: 3
W20260925 13:55:57.845023 19315 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.252:36181: Network error: Client connection negotiation failed: client connection to 127.18.182.252:36181: connect: Connection refused (error 111)
I20260925 13:55:57.874327 19240 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } attempt: 2
W20260925 13:55:57.878603 19240 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.252:36181: Network error: Client connection negotiation failed: client connection to 127.18.182.252:36181: connect: Connection refused (error 111)
W20260925 13:55:57.890190 19317 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:55:57.890838 19317 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:55:57.891289 19317 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260925 13:55:57.924705 19317 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:55:57.925087 19317 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:55:57.925369 19317 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:55:57.925635 19317 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:55:57.963075 19317 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.182.212:43091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-2/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.18.182.254:40825,127.18.182.253:39641,127.18.182.252:36181
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.182.252:36181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.182.252
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 102683f65af844d269565345d7d8c6ebc55c92c5
build type FASTDEBUG
built by None at 25 Sep 2026 13:43:25 UTC on e7f111948823
build id 13994
TSAN enabled
I20260925 13:55:57.963960 19240 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } attempt: 3
I20260925 13:55:57.964529 19317 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:55:57.966554 19317 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260925 13:55:57.968007 19240 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.252:36181: Network error: Client connection negotiation failed: client connection to 127.18.182.252:36181: connect: Connection refused (error 111)
W20260925 13:55:57.977651 19329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:55:57.988451 19315 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } attempt: 4
W20260925 13:55:57.995692 19315 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.252:36181: Network error: Client connection negotiation failed: client connection to 127.18.182.252:36181: connect: Connection refused (error 111)
I20260925 13:55:58.108369 19240 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } attempt: 4
W20260925 13:55:58.116444 19240 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.252:36181: Network error: Client connection negotiation failed: client connection to 127.18.182.252:36181: connect: Connection refused (error 111)
I20260925 13:55:58.295204 19315 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } attempt: 5
W20260925 13:55:58.302980 19315 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.252:36181: Network error: Client connection negotiation failed: client connection to 127.18.182.252:36181: connect: Connection refused (error 111)
I20260925 13:55:58.399995 19240 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } attempt: 5
W20260925 13:55:58.409389 19240 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.252:36181: Network error: Client connection negotiation failed: client connection to 127.18.182.252:36181: connect: Connection refused (error 111)
I20260925 13:55:58.850531 19315 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } attempt: 6
W20260925 13:55:58.887329 19315 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.252:36181: Network error: Client connection negotiation failed: client connection to 127.18.182.252:36181: connect: Connection refused (error 111)
I20260925 13:55:58.961918 19240 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } attempt: 6
W20260925 13:55:58.992630 19240 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.252:36181: Network error: Client connection negotiation failed: client connection to 127.18.182.252:36181: connect: Connection refused (error 111)
W20260925 13:55:59.383313 19328 debug-util.cc:398] Leaking SignalData structure 0x7b08000379a0 after lost signal to thread 19317
W20260925 13:55:57.978364 19330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:55:59.547636 19317 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.570s user 0.516s sys 1.040s
W20260925 13:55:59.548452 19317 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.571s user 0.516s sys 1.040s
W20260925 13:55:59.550323 19332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:55:59.552510 19331 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1571 milliseconds
I20260925 13:55:59.552591 19317 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260925 13:55:59.554203 19317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:55:59.557299 19317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:55:59.558775 19317 hybrid_clock.cc:648] HybridClock initialized: now 1790344559558732 us; error 38 us; skew 500 ppm
I20260925 13:55:59.559881 19317 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 13:55:59.567613 19317 webserver.cc:533] Webserver started at http://127.18.182.252:42511/ using document root <none> and password file <none>
I20260925 13:55:59.568866 19317 fs_manager.cc:362] Metadata directory not provided
I20260925 13:55:59.569134 19317 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:55:59.569677 19317 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:55:59.573894 19317 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-2/data/instance:
uuid: "fc3953a2ba984a658630c423ec9b9728"
format_stamp: "Formatted at 2026-09-25 13:55:59 on dist-test-slave-nlqf"
I20260925 13:55:59.574919 19317 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-2/wal/instance:
uuid: "fc3953a2ba984a658630c423ec9b9728"
format_stamp: "Formatted at 2026-09-25 13:55:59 on dist-test-slave-nlqf"
I20260925 13:55:59.582109 19317 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260925 13:55:59.587687 19343 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:55:59.588735 19317 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260925 13:55:59.589071 19317 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-2/data,/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-2/wal
uuid: "fc3953a2ba984a658630c423ec9b9728"
format_stamp: "Formatted at 2026-09-25 13:55:59 on dist-test-slave-nlqf"
I20260925 13:55:59.589429 19317 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:55:59.647583 19317 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260925 13:55:59.649148 19317 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:55:59.649667 19317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:55:59.721344 19317 rpc_server.cc:307] RPC server started. Bound to: 127.18.182.252:36181
I20260925 13:55:59.721423 19394 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.182.252:36181 every 8 connection(s)
I20260925 13:55:59.724252 19317 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-2/data/info.pb
I20260925 13:55:59.730409 19395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:55:59.732398 19163 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskYmJttM/build/tsan/bin/kudu as pid 19317
I20260925 13:55:59.732825 19163 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-2/wal/instance
I20260925 13:55:59.741647 19395 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:55:59.765528 19395 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:55:59.778889 19395 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:55:59.801192 19395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728: Bootstrap starting.
I20260925 13:55:59.807976 19395 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728: Neither blocks nor log segments found. Creating new log.
I20260925 13:55:59.809794 19395 log.cc:826] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728: Log is configured to *not* fsync() on all Append() calls
I20260925 13:55:59.815968 19395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728: No bootstrap required, opened a new log
I20260925 13:55:59.834781 19395 raft_consensus.cc:359] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } }
I20260925 13:55:59.835784 19395 raft_consensus.cc:385] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:55:59.836097 19395 raft_consensus.cc:740] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc3953a2ba984a658630c423ec9b9728, State: Initialized, Role: FOLLOWER
I20260925 13:55:59.837178 19395 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } }
I20260925 13:55:59.839548 19406 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } } }
I20260925 13:55:59.840168 19406 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:55:59.840849 19395 sys_catalog.cc:565] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [sys.catalog]: configured and running, proceeding with master startup.
W20260925 13:55:59.867800 19417 catalog_manager.cc:1607] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 13:55:59.868155 19417 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 13:55:59.862339 19317 master_runner.cc:187] Error getting master registration for 127.18.182.254:40825: OK, instance_id { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" instance_seqno: 1790344555301129 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 13:55:59.887247 19317 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260925 13:55:59.904910 19349 tablet.cc:2433] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 13:55:59.947949 19315 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } attempt: 7
I20260925 13:55:59.966854 19315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Bootstrap starting.
I20260925 13:55:59.972295 19315 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Neither blocks nor log segments found. Creating new log.
I20260925 13:55:59.974152 19315 log.cc:826] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Log is configured to *not* fsync() on all Append() calls
I20260925 13:55:59.979244 19315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: No bootstrap required, opened a new log
I20260925 13:55:59.997210 19315 raft_consensus.cc:359] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } }
I20260925 13:55:59.998140 19315 raft_consensus.cc:385] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:55:59.998471 19315 raft_consensus.cc:740] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcc92728d49643efab5528d77c3d7828, State: Initialized, Role: FOLLOWER
I20260925 13:55:59.999202 19315 consensus_queue.cc:260] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } }
I20260925 13:56:00.002151 19421 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } } }
I20260925 13:56:00.002923 19421 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:56:00.003912 19315 sys_catalog.cc:565] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:56:00.026170 19240 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } attempt: 7
W20260925 13:56:00.049695 19432 catalog_manager.cc:1607] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 13:56:00.050164 19432 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 13:56:00.063760 19421 raft_consensus.cc:493] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:00.064679 19421 raft_consensus.cc:515] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } }
I20260925 13:56:00.031692 19242 master_runner.cc:187] Error getting master registration for 127.18.182.254:40825: OK, instance_id { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" instance_seqno: 1790344555301129 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 13:56:00.072258 19242 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 13:56:00.074378 19421 leader_election.cc:290] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dcc50a8531144a69ab0cc00df1f0df4 (127.18.182.254:40825), fc3953a2ba984a658630c423ec9b9728 (127.18.182.252:36181)
I20260925 13:56:00.076459 19370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bcc92728d49643efab5528d77c3d7828" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc3953a2ba984a658630c423ec9b9728" is_pre_election: true
I20260925 13:56:00.077594 19240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4: Bootstrap starting.
I20260925 13:56:00.077374 19370 raft_consensus.cc:2468] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcc92728d49643efab5528d77c3d7828 in term 0.
I20260925 13:56:00.079494 19266 leader_election.cc:304] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcc92728d49643efab5528d77c3d7828, fc3953a2ba984a658630c423ec9b9728; no voters:
I20260925 13:56:00.082165 19421 raft_consensus.cc:2804] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:00.082619 19421 raft_consensus.cc:493] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:00.083096 19421 raft_consensus.cc:3060] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:00.085191 19240 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:00.087716 19240 log.cc:826] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4: Log is configured to *not* fsync() on all Append() calls
I20260925 13:56:00.095122 19421 raft_consensus.cc:515] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } }
I20260925 13:56:00.102561 19370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bcc92728d49643efab5528d77c3d7828" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc3953a2ba984a658630c423ec9b9728"
I20260925 13:56:00.103078 19370 raft_consensus.cc:3060] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:00.105342 19240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4: No bootstrap required, opened a new log
I20260925 13:56:00.115401 19370 raft_consensus.cc:2468] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcc92728d49643efab5528d77c3d7828 in term 1.
I20260925 13:56:00.125195 19266 leader_election.cc:304] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcc92728d49643efab5528d77c3d7828, fc3953a2ba984a658630c423ec9b9728; no voters:
I20260925 13:56:00.129989 19421 leader_election.cc:290] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 [CANDIDATE]: Term 1 election: Requested vote from peers 7dcc50a8531144a69ab0cc00df1f0df4 (127.18.182.254:40825), fc3953a2ba984a658630c423ec9b9728 (127.18.182.252:36181)
I20260925 13:56:00.131310 19421 raft_consensus.cc:2804] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:00.132172 19421 raft_consensus.cc:697] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 [term 1 LEADER]: Becoming Leader. State: Replica: bcc92728d49643efab5528d77c3d7828, State: Running, Role: LEADER
I20260925 13:56:00.133595 19421 consensus_queue.cc:237] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } }
I20260925 13:56:00.152371 19436 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bcc92728d49643efab5528d77c3d7828. Latest consensus state: current_term: 1 leader_uuid: "bcc92728d49643efab5528d77c3d7828" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } } }
I20260925 13:56:00.153208 19436 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 [sys.catalog]: This master's current role is: LEADER
I20260925 13:56:00.162405 19240 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } }
I20260925 13:56:00.164155 19240 raft_consensus.cc:385] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:00.164526 19438 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260925 13:56:00.164767 19240 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dcc50a8531144a69ab0cc00df1f0df4, State: Initialized, Role: FOLLOWER
I20260925 13:56:00.166546 19240 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } }
I20260925 13:56:00.174449 19438 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260925 13:56:00.180460 19439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } } }
I20260925 13:56:00.181253 19439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:56:00.185021 19214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bcc92728d49643efab5528d77c3d7828" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dcc50a8531144a69ab0cc00df1f0df4"
I20260925 13:56:00.185920 19215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bcc92728d49643efab5528d77c3d7828" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" is_pre_election: true
I20260925 13:56:00.187124 19215 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcc92728d49643efab5528d77c3d7828 in term 0.
I20260925 13:56:00.193079 19214 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:00.200174 19214 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcc92728d49643efab5528d77c3d7828 in term 1.
I20260925 13:56:00.200614 19240 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:56:00.206276 19370 raft_consensus.cc:1275] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 1 FOLLOWER]: Refusing update from remote peer bcc92728d49643efab5528d77c3d7828: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:56:00.208065 19436 consensus_queue.cc:1048] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:00.226904 19214 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [term 1 FOLLOWER]: Refusing update from remote peer bcc92728d49643efab5528d77c3d7828: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:56:00.228843 19436 consensus_queue.cc:1048] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 13:56:00.242856 19269 tablet.cc:2433] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 13:56:00.260363 19194 tablet.cc:2433] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 13:56:00.267133 19439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bcc92728d49643efab5528d77c3d7828. Latest consensus state: current_term: 1 leader_uuid: "bcc92728d49643efab5528d77c3d7828" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } } }
I20260925 13:56:00.268074 19439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:56:00.282321 19436 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bcc92728d49643efab5528d77c3d7828" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } } }
I20260925 13:56:00.282982 19436 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 [sys.catalog]: This master's current role is: LEADER
I20260925 13:56:00.284649 19406 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bcc92728d49643efab5528d77c3d7828. Latest consensus state: current_term: 1 leader_uuid: "bcc92728d49643efab5528d77c3d7828" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } } }
I20260925 13:56:00.285311 19406 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:56:00.285822 19438 catalog_manager.cc:1396] Generated new cluster ID: 83d15c6349674bc6880b5fd44887fcd6
I20260925 13:56:00.286087 19438 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260925 13:56:00.289577 19439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bcc92728d49643efab5528d77c3d7828" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } } }
I20260925 13:56:00.290236 19439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:56:00.297086 19421 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bcc92728d49643efab5528d77c3d7828" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } } }
I20260925 13:56:00.297778 19421 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 [sys.catalog]: This master's current role is: LEADER
I20260925 13:56:00.322721 19457 catalog_manager.cc:1308] Loaded cluster ID: 83d15c6349674bc6880b5fd44887fcd6
I20260925 13:56:00.323021 19457 catalog_manager.cc:1601] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4: loading cluster ID for follower catalog manager: success
I20260925 13:56:00.326471 19406 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bcc92728d49643efab5528d77c3d7828" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } } }
W20260925 13:56:00.327471 19457 catalog_manager.cc:1625] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260925 13:56:00.327778 19457 catalog_manager.cc:896] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260925 13:56:00.327158 19406 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:56:00.358852 19438 catalog_manager.cc:1419] Generated new certificate authority record
I20260925 13:56:00.360857 19438 catalog_manager.cc:1553] Loading token signing keys...
I20260925 13:56:00.405428 19438 catalog_manager.cc:6219] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Generated new TSK 0
I20260925 13:56:00.407094 19438 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260925 13:56:00.455119 19163 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskYmJttM/build/tsan/bin/kudu
/tmp/dist-test-taskYmJttM/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.182.193:0
--local_ip_for_outbound_sockets=127.18.182.193
--webserver_interface=127.18.182.193
--webserver_port=0
--tserver_master_addrs=127.18.182.254:40825,127.18.182.253:39641,127.18.182.252:36181
--builtin_ntp_servers=127.18.182.212:43091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260925 13:56:00.873934 19417 catalog_manager.cc:1308] Loaded cluster ID: 83d15c6349674bc6880b5fd44887fcd6
I20260925 13:56:00.874413 19417 catalog_manager.cc:1601] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728: loading cluster ID for follower catalog manager: success
I20260925 13:56:00.888630 19417 catalog_manager.cc:1623] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728: acquiring CA information for follower catalog manager: success
I20260925 13:56:00.898300 19417 catalog_manager.cc:1651] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260925 13:56:01.055578 19460 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:56:01.056339 19460 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:56:01.057287 19460 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260925 13:56:01.119841 19460 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:56:01.121606 19460 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.182.193
I20260925 13:56:01.191814 19460 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.182.212:43091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.182.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.182.193
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.18.182.254:40825,127.18.182.253:39641,127.18.182.252:36181
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.182.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 102683f65af844d269565345d7d8c6ebc55c92c5
build type FASTDEBUG
built by None at 25 Sep 2026 13:43:25 UTC on e7f111948823
build id 13994
TSAN enabled
I20260925 13:56:01.194314 19460 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:56:01.196712 19460 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260925 13:56:01.228793 19466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:56:01.336164 19457 catalog_manager.cc:1623] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4: acquiring CA information for follower catalog manager: success
I20260925 13:56:01.341637 19457 catalog_manager.cc:1651] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 13:56:01.401621 19470 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bcc92728d49643efab5528d77c3d7828)
I20260925 13:56:01.403385 19470 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } }
I20260925 13:56:01.408856 19290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bcc92728d49643efab5528d77c3d7828" is_pre_election: true
I20260925 13:56:01.412882 19370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc3953a2ba984a658630c423ec9b9728" is_pre_election: true
I20260925 13:56:01.412750 19470 leader_election.cc:290] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bcc92728d49643efab5528d77c3d7828 (127.18.182.253:39641), fc3953a2ba984a658630c423ec9b9728 (127.18.182.252:36181)
I20260925 13:56:01.414592 19191 leader_election.cc:304] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7dcc50a8531144a69ab0cc00df1f0df4; no voters: bcc92728d49643efab5528d77c3d7828, fc3953a2ba984a658630c423ec9b9728
I20260925 13:56:01.419773 19470 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:56:01.844146 19471 raft_consensus.cc:493] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bcc92728d49643efab5528d77c3d7828)
I20260925 13:56:01.844681 19471 raft_consensus.cc:515] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } }
I20260925 13:56:01.848239 19215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fc3953a2ba984a658630c423ec9b9728" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" is_pre_election: true
I20260925 13:56:01.848829 19471 leader_election.cc:290] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7dcc50a8531144a69ab0cc00df1f0df4 (127.18.182.254:40825), bcc92728d49643efab5528d77c3d7828 (127.18.182.253:39641)
I20260925 13:56:01.850112 19290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fc3953a2ba984a658630c423ec9b9728" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bcc92728d49643efab5528d77c3d7828" is_pre_election: true
I20260925 13:56:01.851706 19345 leader_election.cc:304] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc3953a2ba984a658630c423ec9b9728; no voters: 7dcc50a8531144a69ab0cc00df1f0df4, bcc92728d49643efab5528d77c3d7828
I20260925 13:56:01.852434 19471 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:56:02.368618 19472 raft_consensus.cc:493] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bcc92728d49643efab5528d77c3d7828)
I20260925 13:56:02.369107 19472 raft_consensus.cc:515] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } }
I20260925 13:56:02.373605 19215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fc3953a2ba984a658630c423ec9b9728" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" is_pre_election: true
I20260925 13:56:02.374907 19472 leader_election.cc:290] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7dcc50a8531144a69ab0cc00df1f0df4 (127.18.182.254:40825), bcc92728d49643efab5528d77c3d7828 (127.18.182.253:39641)
I20260925 13:56:02.376204 19290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fc3953a2ba984a658630c423ec9b9728" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bcc92728d49643efab5528d77c3d7828" is_pre_election: true
I20260925 13:56:02.377669 19345 leader_election.cc:304] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc3953a2ba984a658630c423ec9b9728; no voters: 7dcc50a8531144a69ab0cc00df1f0df4, bcc92728d49643efab5528d77c3d7828
I20260925 13:56:02.378419 19472 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:56:02.465507 19473 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bcc92728d49643efab5528d77c3d7828)
I20260925 13:56:02.469789 19473 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } }
I20260925 13:56:02.472456 19290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bcc92728d49643efab5528d77c3d7828" is_pre_election: true
I20260925 13:56:02.475427 19370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc3953a2ba984a658630c423ec9b9728" is_pre_election: true
I20260925 13:56:02.477016 19191 leader_election.cc:304] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7dcc50a8531144a69ab0cc00df1f0df4; no voters: bcc92728d49643efab5528d77c3d7828, fc3953a2ba984a658630c423ec9b9728
I20260925 13:56:02.479594 19473 leader_election.cc:290] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bcc92728d49643efab5528d77c3d7828 (127.18.182.253:39641), fc3953a2ba984a658630c423ec9b9728 (127.18.182.252:36181)
I20260925 13:56:02.480391 19473 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:56:03.364981 19475 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bcc92728d49643efab5528d77c3d7828)
I20260925 13:56:03.365624 19475 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } }
I20260925 13:56:03.368896 19290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bcc92728d49643efab5528d77c3d7828" is_pre_election: true
I20260925 13:56:03.371245 19475 leader_election.cc:290] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bcc92728d49643efab5528d77c3d7828 (127.18.182.253:39641), fc3953a2ba984a658630c423ec9b9728 (127.18.182.252:36181)
I20260925 13:56:03.372470 19370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc3953a2ba984a658630c423ec9b9728" is_pre_election: true
I20260925 13:56:03.373965 19191 leader_election.cc:304] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7dcc50a8531144a69ab0cc00df1f0df4; no voters: bcc92728d49643efab5528d77c3d7828, fc3953a2ba984a658630c423ec9b9728
I20260925 13:56:03.378319 19475 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:56:03.765393 19477 raft_consensus.cc:493] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bcc92728d49643efab5528d77c3d7828)
I20260925 13:56:03.766139 19477 raft_consensus.cc:515] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } }
I20260925 13:56:03.769747 19215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fc3953a2ba984a658630c423ec9b9728" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" is_pre_election: true
I20260925 13:56:03.779955 19290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fc3953a2ba984a658630c423ec9b9728" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bcc92728d49643efab5528d77c3d7828" is_pre_election: true
I20260925 13:56:03.782213 19345 leader_election.cc:304] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc3953a2ba984a658630c423ec9b9728; no voters: 7dcc50a8531144a69ab0cc00df1f0df4, bcc92728d49643efab5528d77c3d7828
I20260925 13:56:03.784395 19477 leader_election.cc:290] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7dcc50a8531144a69ab0cc00df1f0df4 (127.18.182.254:40825), bcc92728d49643efab5528d77c3d7828 (127.18.182.253:39641)
I20260925 13:56:03.786491 19477 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 13:56:02.620710 19465 debug-util.cc:398] Leaking SignalData structure 0x7b0800037240 after lost signal to thread 19460
W20260925 13:56:03.814030 19460 thread.cc:641] GCE (cloud detector) Time spent creating pthread: real 2.582s user 0.867s sys 1.356s
W20260925 13:56:03.825647 19460 thread.cc:608] GCE (cloud detector) Time spent starting thread: real 2.594s user 0.867s sys 1.357s
I20260925 13:56:03.836906 19460 server_base.cc:1061] running on GCE node
W20260925 13:56:01.235124 19467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:56:03.841463 19480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:56:03.843132 19460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:56:03.845745 19460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:56:03.847218 19460 hybrid_clock.cc:648] HybridClock initialized: now 1790344563847175 us; error 35 us; skew 500 ppm
I20260925 13:56:03.848289 19460 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 13:56:03.861605 19460 webserver.cc:533] Webserver started at http://127.18.182.193:45955/ using document root <none> and password file <none>
I20260925 13:56:03.862912 19460 fs_manager.cc:362] Metadata directory not provided
I20260925 13:56:03.863240 19460 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:56:03.863857 19460 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:56:03.870136 19460 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/data/instance:
uuid: "8f578869bc084a5d8a9a69f143954eb3"
format_stamp: "Formatted at 2026-09-25 13:56:03 on dist-test-slave-nlqf"
I20260925 13:56:03.871601 19460 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/instance:
uuid: "8f578869bc084a5d8a9a69f143954eb3"
format_stamp: "Formatted at 2026-09-25 13:56:03 on dist-test-slave-nlqf"
I20260925 13:56:03.881490 19460 fs_manager.cc:696] Time spent creating directory manager: real 0.009s user 0.004s sys 0.004s
I20260925 13:56:03.889052 19486 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:56:03.890326 19460 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260925 13:56:03.890686 19460 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal
uuid: "8f578869bc084a5d8a9a69f143954eb3"
format_stamp: "Formatted at 2026-09-25 13:56:03 on dist-test-slave-nlqf"
I20260925 13:56:03.891099 19460 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:56:03.927794 19487 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bcc92728d49643efab5528d77c3d7828)
I20260925 13:56:03.928499 19487 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } }
I20260925 13:56:03.930435 19487 leader_election.cc:290] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bcc92728d49643efab5528d77c3d7828 (127.18.182.253:39641), fc3953a2ba984a658630c423ec9b9728 (127.18.182.252:36181)
I20260925 13:56:03.931619 19290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bcc92728d49643efab5528d77c3d7828" is_pre_election: true
I20260925 13:56:03.932396 19370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc3953a2ba984a658630c423ec9b9728" is_pre_election: true
I20260925 13:56:03.933859 19191 leader_election.cc:304] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7dcc50a8531144a69ab0cc00df1f0df4; no voters: bcc92728d49643efab5528d77c3d7828, fc3953a2ba984a658630c423ec9b9728
I20260925 13:56:03.934495 19487 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:56:03.969732 19460 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260925 13:56:03.971285 19460 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:56:03.971776 19460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:56:03.974416 19460 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260925 13:56:03.978757 19460 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:56:03.979027 19460 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:56:03.979306 19460 ts_tablet_manager.cc:616] Registered 0 tablets
I20260925 13:56:03.979518 19460 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260925 13:56:04.136346 19460 rpc_server.cc:307] RPC server started. Bound to: 127.18.182.193:38269
I20260925 13:56:04.136445 19599 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.182.193:38269 every 8 connection(s)
I20260925 13:56:04.138842 19460 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/data/info.pb
I20260925 13:56:04.143226 19163 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskYmJttM/build/tsan/bin/kudu as pid 19460
I20260925 13:56:04.143898 19163 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/instance
I20260925 13:56:04.176684 19163 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskYmJttM/build/tsan/bin/kudu
/tmp/dist-test-taskYmJttM/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.182.194:0
--local_ip_for_outbound_sockets=127.18.182.194
--webserver_interface=127.18.182.194
--webserver_port=0
--tserver_master_addrs=127.18.182.254:40825,127.18.182.253:39641,127.18.182.252:36181
--builtin_ntp_servers=127.18.182.212:43091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260925 13:56:04.195652 19602 heartbeater.cc:344] Connected to a master server at 127.18.182.254:40825
I20260925 13:56:04.196147 19602 heartbeater.cc:461] Registering TS with master...
I20260925 13:56:04.197258 19601 heartbeater.cc:344] Connected to a master server at 127.18.182.253:39641
I20260925 13:56:04.197489 19602 heartbeater.cc:507] Master 127.18.182.254:40825 requested a full tablet report, sending...
I20260925 13:56:04.197595 19601 heartbeater.cc:461] Registering TS with master...
I20260925 13:56:04.198472 19601 heartbeater.cc:507] Master 127.18.182.253:39641 requested a full tablet report, sending...
I20260925 13:56:04.199383 19600 heartbeater.cc:344] Connected to a master server at 127.18.182.252:36181
I20260925 13:56:04.199729 19600 heartbeater.cc:461] Registering TS with master...
I20260925 13:56:04.200611 19600 heartbeater.cc:507] Master 127.18.182.252:36181 requested a full tablet report, sending...
I20260925 13:56:04.201421 19280 ts_manager.cc:194] Registered new tserver with Master: 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:04.201419 19205 ts_manager.cc:194] Registered new tserver with Master: 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:04.203421 19360 ts_manager.cc:194] Registered new tserver with Master: 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:04.203998 19280 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.182.193:37907
W20260925 13:56:04.485055 19610 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:56:04.485586 19610 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:56:04.486172 19610 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260925 13:56:04.518740 19610 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:56:04.519685 19610 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.182.194
I20260925 13:56:04.556769 19610 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.182.212:43091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.182.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.182.194
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.18.182.254:40825,127.18.182.253:39641,127.18.182.252:36181
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.182.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 102683f65af844d269565345d7d8c6ebc55c92c5
build type FASTDEBUG
built by None at 25 Sep 2026 13:43:25 UTC on e7f111948823
build id 13994
TSAN enabled
I20260925 13:56:04.558252 19610 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:56:04.559914 19610 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260925 13:56:04.573733 19616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:56:04.708452 19601 heartbeater.cc:499] Master 127.18.182.253:39641 was elected leader, sending a full tablet report...
I20260925 13:56:04.854188 19622 raft_consensus.cc:493] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bcc92728d49643efab5528d77c3d7828)
I20260925 13:56:04.855396 19622 raft_consensus.cc:515] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } }
I20260925 13:56:04.858228 19622 leader_election.cc:290] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7dcc50a8531144a69ab0cc00df1f0df4 (127.18.182.254:40825), bcc92728d49643efab5528d77c3d7828 (127.18.182.253:39641)
I20260925 13:56:04.859777 19215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fc3953a2ba984a658630c423ec9b9728" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" is_pre_election: true
I20260925 13:56:04.859929 19290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fc3953a2ba984a658630c423ec9b9728" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bcc92728d49643efab5528d77c3d7828" is_pre_election: true
I20260925 13:56:04.862398 19345 leader_election.cc:304] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc3953a2ba984a658630c423ec9b9728; no voters: 7dcc50a8531144a69ab0cc00df1f0df4, bcc92728d49643efab5528d77c3d7828
I20260925 13:56:04.863780 19622 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 13:56:05.975839 19615 debug-util.cc:398] Leaking SignalData structure 0x7b0800037240 after lost signal to thread 19610
W20260925 13:56:06.276049 19615 kernel_stack_watchdog.cc:198] Thread 19610 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:642 for 399ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20260925 13:56:04.575199 19617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:56:06.276940 19610 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.703s user 0.519s sys 1.144s
W20260925 13:56:06.277352 19610 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.704s user 0.519s sys 1.144s
W20260925 13:56:06.278301 19619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:56:06.281209 19618 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1703 milliseconds
I20260925 13:56:06.281260 19610 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260925 13:56:06.282692 19610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:56:06.284950 19610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:56:06.286355 19610 hybrid_clock.cc:648] HybridClock initialized: now 1790344566286313 us; error 36 us; skew 500 ppm
I20260925 13:56:06.287184 19610 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 13:56:06.293093 19610 webserver.cc:533] Webserver started at http://127.18.182.194:39407/ using document root <none> and password file <none>
I20260925 13:56:06.293978 19610 fs_manager.cc:362] Metadata directory not provided
I20260925 13:56:06.294205 19610 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:56:06.294713 19610 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:56:06.298880 19610 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/data/instance:
uuid: "7c1b52895a14482fb54a6758f5054f8f"
format_stamp: "Formatted at 2026-09-25 13:56:06 on dist-test-slave-nlqf"
I20260925 13:56:06.299995 19610 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/instance:
uuid: "7c1b52895a14482fb54a6758f5054f8f"
format_stamp: "Formatted at 2026-09-25 13:56:06 on dist-test-slave-nlqf"
I20260925 13:56:06.307081 19610 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.000s sys 0.006s
I20260925 13:56:06.312402 19628 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:56:06.313426 19610 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260925 13:56:06.313755 19610 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal
uuid: "7c1b52895a14482fb54a6758f5054f8f"
format_stamp: "Formatted at 2026-09-25 13:56:06 on dist-test-slave-nlqf"
I20260925 13:56:06.314077 19610 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:56:06.371834 19610 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260925 13:56:06.373307 19610 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:56:06.373765 19610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:56:06.376331 19610 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260925 13:56:06.380728 19610 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:56:06.380976 19610 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:56:06.381240 19610 ts_tablet_manager.cc:616] Registered 0 tablets
I20260925 13:56:06.381419 19610 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:56:06.512425 19610 rpc_server.cc:307] RPC server started. Bound to: 127.18.182.194:38641
I20260925 13:56:06.512532 19740 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.182.194:38641 every 8 connection(s)
I20260925 13:56:06.514945 19610 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/data/info.pb
I20260925 13:56:06.519500 19163 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskYmJttM/build/tsan/bin/kudu as pid 19610
I20260925 13:56:06.519968 19163 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/instance
I20260925 13:56:06.548177 19163 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskYmJttM/build/tsan/bin/kudu
/tmp/dist-test-taskYmJttM/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.182.195:0
--local_ip_for_outbound_sockets=127.18.182.195
--webserver_interface=127.18.182.195
--webserver_port=0
--tserver_master_addrs=127.18.182.254:40825,127.18.182.253:39641,127.18.182.252:36181
--builtin_ntp_servers=127.18.182.212:43091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260925 13:56:06.552413 19742 heartbeater.cc:344] Connected to a master server at 127.18.182.253:39641
I20260925 13:56:06.552907 19742 heartbeater.cc:461] Registering TS with master...
I20260925 13:56:06.554584 19742 heartbeater.cc:507] Master 127.18.182.253:39641 requested a full tablet report, sending...
I20260925 13:56:06.557703 19280 ts_manager.cc:194] Registered new tserver with Master: 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:06.559664 19280 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.182.194:34125
I20260925 13:56:06.563210 19741 heartbeater.cc:344] Connected to a master server at 127.18.182.252:36181
I20260925 13:56:06.563619 19741 heartbeater.cc:461] Registering TS with master...
I20260925 13:56:06.563840 19743 heartbeater.cc:344] Connected to a master server at 127.18.182.254:40825
I20260925 13:56:06.564177 19743 heartbeater.cc:461] Registering TS with master...
I20260925 13:56:06.564702 19743 heartbeater.cc:507] Master 127.18.182.254:40825 requested a full tablet report, sending...
I20260925 13:56:06.564196 19741 heartbeater.cc:507] Master 127.18.182.252:36181 requested a full tablet report, sending...
I20260925 13:56:06.567103 19205 ts_manager.cc:194] Registered new tserver with Master: 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:06.567853 19360 ts_manager.cc:194] Registered new tserver with Master: 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:06.655270 19752 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bcc92728d49643efab5528d77c3d7828)
I20260925 13:56:06.655768 19752 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } }
I20260925 13:56:06.657388 19752 leader_election.cc:290] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bcc92728d49643efab5528d77c3d7828 (127.18.182.253:39641), fc3953a2ba984a658630c423ec9b9728 (127.18.182.252:36181)
I20260925 13:56:06.658384 19290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bcc92728d49643efab5528d77c3d7828" is_pre_election: true
I20260925 13:56:06.658595 19370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fc3953a2ba984a658630c423ec9b9728" is_pre_election: true
I20260925 13:56:06.660346 19190 leader_election.cc:304] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7dcc50a8531144a69ab0cc00df1f0df4; no voters: bcc92728d49643efab5528d77c3d7828, fc3953a2ba984a658630c423ec9b9728
I20260925 13:56:06.661109 19752 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:56:06.783403 19753 raft_consensus.cc:493] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bcc92728d49643efab5528d77c3d7828)
I20260925 13:56:06.783828 19753 raft_consensus.cc:515] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } }
I20260925 13:56:06.785296 19753 leader_election.cc:290] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7dcc50a8531144a69ab0cc00df1f0df4 (127.18.182.254:40825), bcc92728d49643efab5528d77c3d7828 (127.18.182.253:39641)
I20260925 13:56:06.786126 19215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fc3953a2ba984a658630c423ec9b9728" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" is_pre_election: true
I20260925 13:56:06.786374 19290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fc3953a2ba984a658630c423ec9b9728" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bcc92728d49643efab5528d77c3d7828" is_pre_election: true
I20260925 13:56:06.787701 19345 leader_election.cc:304] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc3953a2ba984a658630c423ec9b9728; no voters: 7dcc50a8531144a69ab0cc00df1f0df4, bcc92728d49643efab5528d77c3d7828
I20260925 13:56:06.788403 19753 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 13:56:06.865880 19751 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:56:06.866344 19751 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:56:06.866811 19751 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260925 13:56:06.899290 19751 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:56:06.900295 19751 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.182.195
I20260925 13:56:06.936115 19751 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.182.212:43091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.182.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.182.195
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.18.182.254:40825,127.18.182.253:39641,127.18.182.252:36181
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.182.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 102683f65af844d269565345d7d8c6ebc55c92c5
build type FASTDEBUG
built by None at 25 Sep 2026 13:43:25 UTC on e7f111948823
build id 13994
TSAN enabled
I20260925 13:56:06.937430 19751 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:56:06.938988 19751 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260925 13:56:06.951180 19759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:56:07.063679 19742 heartbeater.cc:499] Master 127.18.182.253:39641 was elected leader, sending a full tablet report...
I20260925 13:56:07.343966 19765 raft_consensus.cc:493] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bcc92728d49643efab5528d77c3d7828)
I20260925 13:56:07.344447 19765 raft_consensus.cc:515] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } }
I20260925 13:56:07.346247 19765 leader_election.cc:290] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7dcc50a8531144a69ab0cc00df1f0df4 (127.18.182.254:40825), bcc92728d49643efab5528d77c3d7828 (127.18.182.253:39641)
I20260925 13:56:07.348950 19290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fc3953a2ba984a658630c423ec9b9728" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bcc92728d49643efab5528d77c3d7828" is_pre_election: true
I20260925 13:56:07.349956 19215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fc3953a2ba984a658630c423ec9b9728" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" is_pre_election: true
I20260925 13:56:07.355116 19345 leader_election.cc:304] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc3953a2ba984a658630c423ec9b9728; no voters: 7dcc50a8531144a69ab0cc00df1f0df4, bcc92728d49643efab5528d77c3d7828
I20260925 13:56:07.356164 19765 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:56:07.908540 19767 raft_consensus.cc:493] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bcc92728d49643efab5528d77c3d7828)
I20260925 13:56:07.911540 19767 raft_consensus.cc:515] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } }
I20260925 13:56:07.914458 19767 leader_election.cc:290] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7dcc50a8531144a69ab0cc00df1f0df4 (127.18.182.254:40825), bcc92728d49643efab5528d77c3d7828 (127.18.182.253:39641)
I20260925 13:56:07.917810 19215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fc3953a2ba984a658630c423ec9b9728" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" is_pre_election: true
I20260925 13:56:07.919653 19290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fc3953a2ba984a658630c423ec9b9728" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bcc92728d49643efab5528d77c3d7828" is_pre_election: true
I20260925 13:56:07.922665 19345 leader_election.cc:304] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc3953a2ba984a658630c423ec9b9728; no voters: 7dcc50a8531144a69ab0cc00df1f0df4, bcc92728d49643efab5528d77c3d7828
I20260925 13:56:07.930737 19767 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 13:56:08.354449 19758 debug-util.cc:398] Leaking SignalData structure 0x7b0800037240 after lost signal to thread 19751
W20260925 13:56:08.536396 19758 kernel_stack_watchdog.cc:198] Thread 19751 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:642 for 400ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20260925 13:56:06.952270 19760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:56:08.537812 19751 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.586s user 0.546s sys 1.035s
W20260925 13:56:08.537926 19761 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1585 milliseconds
W20260925 13:56:08.538252 19751 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.587s user 0.546s sys 1.035s
W20260925 13:56:08.542527 19762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:56:08.542980 19751 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260925 13:56:08.544090 19751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:56:08.546727 19751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:56:08.548210 19751 hybrid_clock.cc:648] HybridClock initialized: now 1790344568548168 us; error 41 us; skew 500 ppm
I20260925 13:56:08.549064 19751 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 13:56:08.555591 19751 webserver.cc:533] Webserver started at http://127.18.182.195:39735/ using document root <none> and password file <none>
I20260925 13:56:08.556519 19751 fs_manager.cc:362] Metadata directory not provided
I20260925 13:56:08.556756 19751 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:56:08.557216 19751 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:56:08.561367 19751 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/data/instance:
uuid: "f790493ba7ad403993b678f2df2b55be"
format_stamp: "Formatted at 2026-09-25 13:56:08 on dist-test-slave-nlqf"
I20260925 13:56:08.562455 19751 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/instance:
uuid: "f790493ba7ad403993b678f2df2b55be"
format_stamp: "Formatted at 2026-09-25 13:56:08 on dist-test-slave-nlqf"
I20260925 13:56:08.570434 19751 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.005s sys 0.005s
I20260925 13:56:08.576088 19772 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:56:08.577190 19751 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260925 13:56:08.577531 19751 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal
uuid: "f790493ba7ad403993b678f2df2b55be"
format_stamp: "Formatted at 2026-09-25 13:56:08 on dist-test-slave-nlqf"
I20260925 13:56:08.577881 19751 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:56:08.637763 19751 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260925 13:56:08.639446 19751 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:56:08.639936 19751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:56:08.643074 19751 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260925 13:56:08.647518 19751 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:56:08.647784 19751 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:56:08.648085 19751 ts_tablet_manager.cc:616] Registered 0 tablets
I20260925 13:56:08.648293 19751 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:56:08.807564 19751 rpc_server.cc:307] RPC server started. Bound to: 127.18.182.195:46649
I20260925 13:56:08.807686 19884 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.182.195:46649 every 8 connection(s)
I20260925 13:56:08.810091 19751 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/data/info.pb
I20260925 13:56:08.817241 19163 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskYmJttM/build/tsan/bin/kudu as pid 19751
I20260925 13:56:08.817837 19163 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/instance
I20260925 13:56:08.861737 19886 heartbeater.cc:344] Connected to a master server at 127.18.182.253:39641
I20260925 13:56:08.862048 19887 heartbeater.cc:344] Connected to a master server at 127.18.182.254:40825
I20260925 13:56:08.862041 19885 heartbeater.cc:344] Connected to a master server at 127.18.182.252:36181
I20260925 13:56:08.862273 19886 heartbeater.cc:461] Registering TS with master...
I20260925 13:56:08.862469 19887 heartbeater.cc:461] Registering TS with master...
I20260925 13:56:08.862582 19885 heartbeater.cc:461] Registering TS with master...
I20260925 13:56:08.863795 19886 heartbeater.cc:507] Master 127.18.182.253:39641 requested a full tablet report, sending...
I20260925 13:56:08.863834 19887 heartbeater.cc:507] Master 127.18.182.254:40825 requested a full tablet report, sending...
I20260925 13:56:08.863866 19885 heartbeater.cc:507] Master 127.18.182.252:36181 requested a full tablet report, sending...
I20260925 13:56:08.867455 19360 ts_manager.cc:194] Registered new tserver with Master: f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:08.868382 19280 ts_manager.cc:194] Registered new tserver with Master: f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:08.868566 19205 ts_manager.cc:194] Registered new tserver with Master: f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:08.870287 19280 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.182.195:57799
I20260925 13:56:08.888175 19163 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260925 13:56:08.942770 19279 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_19b72b4e5ddb4528808d66bafa08c34a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:08.942678 19280 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_be2861b26e364cc38cb8ccd089946041"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:08.944494 19278 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_47cb5f3b369c4e008b1c4acda6c3c721"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:08.946964 19278 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_47cb5f3b369c4e008b1c4acda6c3c721 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 13:56:08.946921 19279 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_19b72b4e5ddb4528808d66bafa08c34a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 13:56:08.948426 19280 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_be2861b26e364cc38cb8ccd089946041 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:08.950572 19277 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_6460dded9e1b473b811c1f99aace2341"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:08.952836 19277 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6460dded9e1b473b811c1f99aace2341 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:08.998652 19163 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskYmJttM/build/tsan/bin/kudu with pid 19317
I20260925 13:56:09.009860 19280 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_be2861b26e364cc38cb8ccd089946041" } new_table_name: "default.Table_f6235b56e68b46aea4c9bcbefbf85eb1" modify_external_catalogs: true
W20260925 13:56:09.034544 19266 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.182.252:36181 (error 108) [suppressed 6 similar messages]
I20260925 13:56:09.051138 19278 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_47cb5f3b369c4e008b1c4acda6c3c721" } modify_external_catalogs: true
I20260925 13:56:09.052054 19278 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_47cb5f3b369c4e008b1c4acda6c3c721" } modify_external_catalogs: true
I20260925 13:56:09.052241 19277 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_6460dded9e1b473b811c1f99aace2341" } new_table_name: "default.Table_fc503e4e6bb14eaabf279db7fe11e448" modify_external_catalogs: true
I20260925 13:56:09.061777 19279 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_19b72b4e5ddb4528808d66bafa08c34a" } modify_external_catalogs: true
I20260925 13:56:09.062332 19279 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_19b72b4e5ddb4528808d66bafa08c34a" } modify_external_catalogs: true
W20260925 13:56:09.065459 19280 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8bd480f8e7a3487eb6b74a7ac7544b0d consensus state has no leader
W20260925 13:56:09.073354 19277 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 98180fc321cf49b39f9efaf6a86d33f9 has no consensus state
W20260925 13:56:09.086596 19266 consensus_peers.cc:597] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 -> Peer fc3953a2ba984a658630c423ec9b9728 (127.18.182.252:36181): Couldn't send request to peer fc3953a2ba984a658630c423ec9b9728. Status: Network error: Client connection negotiation failed: client connection to 127.18.182.252:36181: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:09.094823 19278 catalog_manager.cc:6128] Not sending DeleteTablet requests; no consensus state for tablet 197f5255220449aa93ff7c2a3171faa9
W20260925 13:56:09.095287 19267 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 8bd480f8e7a3487eb6b74a7ac7544b0d (table default.Table_f6235b56e68b46aea4c9bcbefbf85eb1 [id=3fd25e8e73f946b8ba379c2876e1386e], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 8bd480f8e7a3487eb6b74a7ac7544b0d consensus state has no leader
I20260925 13:56:09.096241 19279 catalog_manager.cc:6128] Not sending DeleteTablet requests; no consensus state for tablet e56bf0153e164f15aefb555bf8524c48
W20260925 13:56:09.113149 19741 heartbeater.cc:646] Failed to heartbeat to 127.18.182.252:36181 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.182.252:36181: connect: Connection refused (error 111)
I20260925 13:56:09.132436 19535 tablet_service.cc:1511] Processing CreateTablet for tablet 8bd480f8e7a3487eb6b74a7ac7544b0d (DEFAULT_TABLE table=default.Table_f6235b56e68b46aea4c9bcbefbf85eb1 [id=3fd25e8e73f946b8ba379c2876e1386e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:09.142894 19820 tablet_service.cc:1511] Processing CreateTablet for tablet 8bd480f8e7a3487eb6b74a7ac7544b0d (DEFAULT_TABLE table=default.Table_f6235b56e68b46aea4c9bcbefbf85eb1 [id=3fd25e8e73f946b8ba379c2876e1386e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:09.142767 19676 tablet_service.cc:1511] Processing CreateTablet for tablet 8bd480f8e7a3487eb6b74a7ac7544b0d (DEFAULT_TABLE table=default.Table_f6235b56e68b46aea4c9bcbefbf85eb1 [id=3fd25e8e73f946b8ba379c2876e1386e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:09.144739 19820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bd480f8e7a3487eb6b74a7ac7544b0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:09.145377 19676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bd480f8e7a3487eb6b74a7ac7544b0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:09.148548 19535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bd480f8e7a3487eb6b74a7ac7544b0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:09.166633 19534 tablet_service.cc:1511] Processing CreateTablet for tablet 98180fc321cf49b39f9efaf6a86d33f9 (DEFAULT_TABLE table=default.Table_fc503e4e6bb14eaabf279db7fe11e448 [id=d1f1ad2082e84994a06d872c94b1d847]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:09.167776 19534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98180fc321cf49b39f9efaf6a86d33f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:09.168772 19675 tablet_service.cc:1511] Processing CreateTablet for tablet 98180fc321cf49b39f9efaf6a86d33f9 (DEFAULT_TABLE table=default.Table_fc503e4e6bb14eaabf279db7fe11e448 [id=d1f1ad2082e84994a06d872c94b1d847]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:09.169858 19675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98180fc321cf49b39f9efaf6a86d33f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:09.171373 19819 tablet_service.cc:1511] Processing CreateTablet for tablet 98180fc321cf49b39f9efaf6a86d33f9 (DEFAULT_TABLE table=default.Table_fc503e4e6bb14eaabf279db7fe11e448 [id=d1f1ad2082e84994a06d872c94b1d847]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:09.173103 19819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98180fc321cf49b39f9efaf6a86d33f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:09.184590 19938 tablet_bootstrap.cc:492] T 8bd480f8e7a3487eb6b74a7ac7544b0d P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:09.192169 19938 tablet_bootstrap.cc:654] T 8bd480f8e7a3487eb6b74a7ac7544b0d P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:09.198868 19938 log.cc:826] T 8bd480f8e7a3487eb6b74a7ac7544b0d P f790493ba7ad403993b678f2df2b55be: Log is configured to *not* fsync() on all Append() calls
I20260925 13:56:09.200668 19939 tablet_bootstrap.cc:492] T 98180fc321cf49b39f9efaf6a86d33f9 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:09.210918 19939 tablet_bootstrap.cc:654] T 98180fc321cf49b39f9efaf6a86d33f9 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:09.210830 19279 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_b998047d4cac4412a6d2872638486667"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:09.213433 19279 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b998047d4cac4412a6d2872638486667 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:09.213819 19939 log.cc:826] T 98180fc321cf49b39f9efaf6a86d33f9 P 8f578869bc084a5d8a9a69f143954eb3: Log is configured to *not* fsync() on all Append() calls
I20260925 13:56:09.215133 19938 tablet_bootstrap.cc:492] T 8bd480f8e7a3487eb6b74a7ac7544b0d P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:09.215830 19938 ts_tablet_manager.cc:1403] T 8bd480f8e7a3487eb6b74a7ac7544b0d P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.032s user 0.013s sys 0.004s
I20260925 13:56:09.232293 19278 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_e9dc3916a32d4c6e8d9517c521a20220"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:09.233982 19278 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e9dc3916a32d4c6e8d9517c521a20220 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:09.235002 19277 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_9b6d9d00ef814e00bd6b75b1f8e16323"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:09.237442 19277 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9b6d9d00ef814e00bd6b75b1f8e16323 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:09.245548 19938 raft_consensus.cc:359] T 8bd480f8e7a3487eb6b74a7ac7544b0d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:09.246467 19938 raft_consensus.cc:385] T 8bd480f8e7a3487eb6b74a7ac7544b0d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:09.246803 19938 raft_consensus.cc:740] T 8bd480f8e7a3487eb6b74a7ac7544b0d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:09.247784 19938 consensus_queue.cc:260] T 8bd480f8e7a3487eb6b74a7ac7544b0d P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:09.261404 19942 tablet_bootstrap.cc:492] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:09.263214 19279 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_cb55c90b6da345868b11789d1177730d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:09.273123 19942 tablet_bootstrap.cc:654] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:09.275887 19280 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_fc503e4e6bb14eaabf279db7fe11e448" } new_table_name: "default.Table_ec0fe54be8374ac391e843ea5f82f061" modify_external_catalogs: true
I20260925 13:56:09.285692 19942 log.cc:826] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 7c1b52895a14482fb54a6758f5054f8f: Log is configured to *not* fsync() on all Append() calls
W20260925 13:56:09.299535 19600 heartbeater.cc:646] Failed to heartbeat to 127.18.182.252:36181 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.182.252:36181: connect: Connection refused (error 111)
I20260925 13:56:09.302261 19942 tablet_bootstrap.cc:492] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:09.303139 19942 ts_tablet_manager.cc:1403] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.044s user 0.018s sys 0.010s
W20260925 13:56:09.344944 19279 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cb55c90b6da345868b11789d1177730d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:09.350528 19275 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_b998047d4cac4412a6d2872638486667" } modify_external_catalogs: true
I20260925 13:56:09.351320 19275 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_b998047d4cac4412a6d2872638486667" } modify_external_catalogs: true
I20260925 13:56:09.351320 19942 raft_consensus.cc:359] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:09.352589 19942 raft_consensus.cc:385] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:09.353009 19942 raft_consensus.cc:740] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:09.353974 19276 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_f6235b56e68b46aea4c9bcbefbf85eb1" } new_table_name: "default.Table_1c876062216b4a3cbfab9086131b4bb0" modify_external_catalogs: true
I20260925 13:56:09.354487 19942 consensus_queue.cc:260] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:09.366532 19942 ts_tablet_manager.cc:1434] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.063s user 0.045s sys 0.006s
W20260925 13:56:09.367894 19280 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 98180fc321cf49b39f9efaf6a86d33f9 consensus state has no leader
I20260925 13:56:09.376415 19280 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_ec0fe54be8374ac391e843ea5f82f061" } modify_external_catalogs: true
I20260925 13:56:09.376930 19886 heartbeater.cc:499] Master 127.18.182.253:39641 was elected leader, sending a full tablet report...
I20260925 13:56:09.377140 19280 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_ec0fe54be8374ac391e843ea5f82f061" } modify_external_catalogs: true
I20260925 13:56:09.392472 19942 tablet_bootstrap.cc:492] T 98180fc321cf49b39f9efaf6a86d33f9 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:09.400605 19939 tablet_bootstrap.cc:492] T 98180fc321cf49b39f9efaf6a86d33f9 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:09.401412 19939 ts_tablet_manager.cc:1403] T 98180fc321cf49b39f9efaf6a86d33f9 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.201s user 0.015s sys 0.010s
W20260925 13:56:09.404381 19885 heartbeater.cc:646] Failed to heartbeat to 127.18.182.252:36181 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.182.252:36181: connect: Connection refused (error 111)
I20260925 13:56:09.409049 19942 tablet_bootstrap.cc:654] T 98180fc321cf49b39f9efaf6a86d33f9 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:09.413229 19535 tablet_service.cc:1511] Processing CreateTablet for tablet ce8fe54ef2a84dbcbec5d866c37a13d5 (DEFAULT_TABLE table=default.Table_b998047d4cac4412a6d2872638486667 [id=b6f3c8cdd85b4a6f87c491267f6cddab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:09.414321 19535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce8fe54ef2a84dbcbec5d866c37a13d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:09.415143 19676 tablet_service.cc:1511] Processing CreateTablet for tablet ce8fe54ef2a84dbcbec5d866c37a13d5 (DEFAULT_TABLE table=default.Table_b998047d4cac4412a6d2872638486667 [id=b6f3c8cdd85b4a6f87c491267f6cddab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:09.416615 19676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce8fe54ef2a84dbcbec5d866c37a13d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:09.420519 19819 tablet_service.cc:1511] Processing CreateTablet for tablet ce8fe54ef2a84dbcbec5d866c37a13d5 (DEFAULT_TABLE table=default.Table_b998047d4cac4412a6d2872638486667 [id=b6f3c8cdd85b4a6f87c491267f6cddab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:09.421743 19819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce8fe54ef2a84dbcbec5d866c37a13d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:09.452641 19939 raft_consensus.cc:359] T 98180fc321cf49b39f9efaf6a86d33f9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:09.456355 19939 raft_consensus.cc:385] T 98180fc321cf49b39f9efaf6a86d33f9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:09.456697 19939 raft_consensus.cc:740] T 98180fc321cf49b39f9efaf6a86d33f9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:09.460628 19274 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_f68a67145c3941479bfbfbbbaf7e69df"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:09.462785 19274 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f68a67145c3941479bfbfbbbaf7e69df in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:09.463258 19942 tablet_bootstrap.cc:492] T 98180fc321cf49b39f9efaf6a86d33f9 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:09.463927 19942 ts_tablet_manager.cc:1403] T 98180fc321cf49b39f9efaf6a86d33f9 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.072s user 0.014s sys 0.005s
I20260925 13:56:09.467739 19942 raft_consensus.cc:359] T 98180fc321cf49b39f9efaf6a86d33f9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:09.468622 19942 raft_consensus.cc:385] T 98180fc321cf49b39f9efaf6a86d33f9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:09.468286 19939 consensus_queue.cc:260] T 98180fc321cf49b39f9efaf6a86d33f9 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:09.468955 19942 raft_consensus.cc:740] T 98180fc321cf49b39f9efaf6a86d33f9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:09.469816 19942 consensus_queue.cc:260] T 98180fc321cf49b39f9efaf6a86d33f9 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:09.475497 19939 ts_tablet_manager.cc:1434] T 98180fc321cf49b39f9efaf6a86d33f9 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.074s user 0.030s sys 0.007s
I20260925 13:56:09.476598 19939 tablet_bootstrap.cc:492] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:09.480868 19942 ts_tablet_manager.cc:1434] T 98180fc321cf49b39f9efaf6a86d33f9 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.017s user 0.006s sys 0.001s
I20260925 13:56:09.489523 19942 tablet_bootstrap.cc:492] T ce8fe54ef2a84dbcbec5d866c37a13d5 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:09.490952 19939 tablet_bootstrap.cc:654] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:09.499257 19942 tablet_bootstrap.cc:654] T ce8fe54ef2a84dbcbec5d866c37a13d5 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:09.501403 19938 ts_tablet_manager.cc:1434] T 8bd480f8e7a3487eb6b74a7ac7544b0d P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.285s user 0.131s sys 0.073s
I20260925 13:56:09.502547 19938 tablet_bootstrap.cc:492] T 98180fc321cf49b39f9efaf6a86d33f9 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:09.508098 19939 tablet_bootstrap.cc:492] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:09.508754 19939 ts_tablet_manager.cc:1403] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.032s user 0.010s sys 0.005s
I20260925 13:56:09.509150 19938 tablet_bootstrap.cc:654] T 98180fc321cf49b39f9efaf6a86d33f9 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:09.512180 19939 raft_consensus.cc:359] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:09.512940 19939 raft_consensus.cc:385] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:09.513114 19942 tablet_bootstrap.cc:492] T ce8fe54ef2a84dbcbec5d866c37a13d5 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:09.513937 19942 ts_tablet_manager.cc:1403] T ce8fe54ef2a84dbcbec5d866c37a13d5 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.025s user 0.013s sys 0.006s
I20260925 13:56:09.515434 19939 raft_consensus.cc:740] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:09.516206 19939 consensus_queue.cc:260] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:09.516980 19942 raft_consensus.cc:359] T ce8fe54ef2a84dbcbec5d866c37a13d5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:09.517763 19942 raft_consensus.cc:385] T ce8fe54ef2a84dbcbec5d866c37a13d5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:09.518096 19942 raft_consensus.cc:740] T ce8fe54ef2a84dbcbec5d866c37a13d5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:09.518604 19939 ts_tablet_manager.cc:1434] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20260925 13:56:09.518989 19942 consensus_queue.cc:260] T ce8fe54ef2a84dbcbec5d866c37a13d5 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:09.528106 19939 tablet_bootstrap.cc:492] T ce8fe54ef2a84dbcbec5d866c37a13d5 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:09.532806 19942 ts_tablet_manager.cc:1434] T ce8fe54ef2a84dbcbec5d866c37a13d5 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.018s user 0.002s sys 0.004s
I20260925 13:56:09.535101 19939 tablet_bootstrap.cc:654] T ce8fe54ef2a84dbcbec5d866c37a13d5 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:09.537741 19938 tablet_bootstrap.cc:492] T 98180fc321cf49b39f9efaf6a86d33f9 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:09.538149 19938 ts_tablet_manager.cc:1403] T 98180fc321cf49b39f9efaf6a86d33f9 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.036s user 0.010s sys 0.015s
I20260925 13:56:09.541592 19938 raft_consensus.cc:359] T 98180fc321cf49b39f9efaf6a86d33f9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:09.542361 19938 raft_consensus.cc:385] T 98180fc321cf49b39f9efaf6a86d33f9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:09.542693 19938 raft_consensus.cc:740] T 98180fc321cf49b39f9efaf6a86d33f9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
W20260925 13:56:09.543375 19748 tablet.cc:2433] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 7c1b52895a14482fb54a6758f5054f8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 13:56:09.543583 19938 consensus_queue.cc:260] T 98180fc321cf49b39f9efaf6a86d33f9 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:09.546072 19938 ts_tablet_manager.cc:1434] T 98180fc321cf49b39f9efaf6a86d33f9 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.008s user 0.003s sys 0.003s
I20260925 13:56:09.547462 19938 tablet_bootstrap.cc:492] T ce8fe54ef2a84dbcbec5d866c37a13d5 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:09.556272 19938 tablet_bootstrap.cc:654] T ce8fe54ef2a84dbcbec5d866c37a13d5 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:09.578413 19275 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet ce8fe54ef2a84dbcbec5d866c37a13d5
I20260925 13:56:09.584388 19272 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_9b6d9d00ef814e00bd6b75b1f8e16323" } new_table_name: "default.Table_72b4fdbfbca745a286dca8c191dc2286" modify_external_catalogs: true
W20260925 13:56:09.587582 19892 tablet.cc:2433] T 8bd480f8e7a3487eb6b74a7ac7544b0d P f790493ba7ad403993b678f2df2b55be: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 13:56:09.588039 19535 tablet_service.cc:1558] Processing DeleteTablet for tablet ce8fe54ef2a84dbcbec5d866c37a13d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:09 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:09.589970 19273 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_bc39392c68044dd5a26e193aaa135333"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:09.594224 19271 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_bb731091526c4e0d95e68171b5b70a60"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:09.596056 19271 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bb731091526c4e0d95e68171b5b70a60 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:09.597218 19924 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bcc92728d49643efab5528d77c3d7828" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } } }
I20260925 13:56:09.597955 19924 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 [sys.catalog]: This master's current role is: LEADER
I20260925 13:56:09.603610 19676 tablet_service.cc:1558] Processing DeleteTablet for tablet ce8fe54ef2a84dbcbec5d866c37a13d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:09 UTC) from {username='slave'} at 127.0.0.1:45948
W20260925 13:56:09.606976 19273 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bc39392c68044dd5a26e193aaa135333 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:09.627573 19819 tablet_service.cc:1558] Processing DeleteTablet for tablet ce8fe54ef2a84dbcbec5d866c37a13d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:09 UTC) from {username='slave'} at 127.0.0.1:51442
W20260925 13:56:09.644663 19266 catalog_manager.cc:5043] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): delete failed for tablet ce8fe54ef2a84dbcbec5d866c37a13d5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ce8fe54ef2a84dbcbec5d866c37a13d5 already in progress: creating tablet
I20260925 13:56:09.648883 19958 tablet_replica.cc:333] T ce8fe54ef2a84dbcbec5d866c37a13d5 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:09.650012 19958 raft_consensus.cc:2243] T ce8fe54ef2a84dbcbec5d866c37a13d5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:09.650669 19958 raft_consensus.cc:2272] T ce8fe54ef2a84dbcbec5d866c37a13d5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus is shut down!
W20260925 13:56:09.653016 19264 catalog_manager.cc:5043] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): delete failed for tablet ce8fe54ef2a84dbcbec5d866c37a13d5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ce8fe54ef2a84dbcbec5d866c37a13d5 already in progress: creating tablet
W20260925 13:56:09.673269 19272 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c0f05269ea6c43a98d2137eae7a9a401 has no consensus state
I20260925 13:56:09.686615 19958 ts_tablet_manager.cc:1916] T ce8fe54ef2a84dbcbec5d866c37a13d5 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 13:56:09.687139 19605 tablet.cc:2433] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 8f578869bc084a5d8a9a69f143954eb3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 13:56:09.691183 19939 tablet_bootstrap.cc:492] T ce8fe54ef2a84dbcbec5d866c37a13d5 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:09.691745 19939 ts_tablet_manager.cc:1403] T ce8fe54ef2a84dbcbec5d866c37a13d5 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.164s user 0.030s sys 0.046s
I20260925 13:56:09.694830 19939 raft_consensus.cc:359] T ce8fe54ef2a84dbcbec5d866c37a13d5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:09.700593 19272 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_c232f8cfe77b43e6ad45e9979d8e3364"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:09.702356 19272 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c232f8cfe77b43e6ad45e9979d8e3364 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:09.706991 19939 raft_consensus.cc:385] T ce8fe54ef2a84dbcbec5d866c37a13d5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:09.713495 19939 raft_consensus.cc:740] T ce8fe54ef2a84dbcbec5d866c37a13d5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:09.715094 19939 consensus_queue.cc:260] T ce8fe54ef2a84dbcbec5d866c37a13d5 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:09.716724 19938 tablet_bootstrap.cc:492] T ce8fe54ef2a84dbcbec5d866c37a13d5 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:09.717201 19938 ts_tablet_manager.cc:1403] T ce8fe54ef2a84dbcbec5d866c37a13d5 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.170s user 0.032s sys 0.109s
I20260925 13:56:09.719992 19938 raft_consensus.cc:359] T ce8fe54ef2a84dbcbec5d866c37a13d5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:09.720741 19938 raft_consensus.cc:385] T ce8fe54ef2a84dbcbec5d866c37a13d5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:09.721067 19938 raft_consensus.cc:740] T ce8fe54ef2a84dbcbec5d866c37a13d5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:09.721860 19938 consensus_queue.cc:260] T ce8fe54ef2a84dbcbec5d866c37a13d5 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:09.728631 19938 ts_tablet_manager.cc:1434] T ce8fe54ef2a84dbcbec5d866c37a13d5 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20260925 13:56:09.748066 19277 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 8bd480f8e7a3487eb6b74a7ac7544b0d (table default.Table_f6235b56e68b46aea4c9bcbefbf85eb1 [id=3fd25e8e73f946b8ba379c2876e1386e]). Expected version 1 got 0
I20260925 13:56:09.753192 19278 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 8bd480f8e7a3487eb6b74a7ac7544b0d (table default.Table_f6235b56e68b46aea4c9bcbefbf85eb1 [id=3fd25e8e73f946b8ba379c2876e1386e]). Expected version 1 got 0
I20260925 13:56:09.754879 19819 tablet_service.cc:1511] Processing CreateTablet for tablet 5a7907a50912430bbeeac44d2ebd47de (DEFAULT_TABLE table=default.Table_e9dc3916a32d4c6e8d9517c521a20220 [id=00c39e12d41b4d27b6cd28a850c76fe4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:09.755983 19819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a7907a50912430bbeeac44d2ebd47de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:09.756173 19939 ts_tablet_manager.cc:1434] T ce8fe54ef2a84dbcbec5d866c37a13d5 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.064s user 0.007s sys 0.000s
I20260925 13:56:09.760133 19676 tablet_service.cc:1511] Processing CreateTablet for tablet 5a7907a50912430bbeeac44d2ebd47de (DEFAULT_TABLE table=default.Table_e9dc3916a32d4c6e8d9517c521a20220 [id=00c39e12d41b4d27b6cd28a850c76fe4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:09.761134 19676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a7907a50912430bbeeac44d2ebd47de. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:09.763937 19276 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8bd480f8e7a3487eb6b74a7ac7544b0d consensus state has no leader
I20260925 13:56:09.765419 19280 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 98180fc321cf49b39f9efaf6a86d33f9
I20260925 13:56:09.769982 19535 tablet_service.cc:1511] Processing CreateTablet for tablet 5a7907a50912430bbeeac44d2ebd47de (DEFAULT_TABLE table=default.Table_e9dc3916a32d4c6e8d9517c521a20220 [id=00c39e12d41b4d27b6cd28a850c76fe4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:09.770967 19535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a7907a50912430bbeeac44d2ebd47de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:09.773595 19820 tablet_service.cc:1558] Processing DeleteTablet for tablet 98180fc321cf49b39f9efaf6a86d33f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:09 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:09.774442 19959 tablet_replica.cc:333] T 98180fc321cf49b39f9efaf6a86d33f9 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:09.775317 19959 raft_consensus.cc:2243] T 98180fc321cf49b39f9efaf6a86d33f9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:09.775907 19959 raft_consensus.cc:2272] T 98180fc321cf49b39f9efaf6a86d33f9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:09.779810 19534 tablet_service.cc:1558] Processing DeleteTablet for tablet 98180fc321cf49b39f9efaf6a86d33f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:09 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:09.779909 19675 tablet_service.cc:1558] Processing DeleteTablet for tablet 98180fc321cf49b39f9efaf6a86d33f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:09 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:09.780701 19955 tablet_replica.cc:333] T 98180fc321cf49b39f9efaf6a86d33f9 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:09.781464 19959 ts_tablet_manager.cc:1916] T 98180fc321cf49b39f9efaf6a86d33f9 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:09.781569 19955 raft_consensus.cc:2243] T 98180fc321cf49b39f9efaf6a86d33f9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:09.782107 19955 raft_consensus.cc:2272] T 98180fc321cf49b39f9efaf6a86d33f9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:09.795120 19955 ts_tablet_manager.cc:1916] T 98180fc321cf49b39f9efaf6a86d33f9 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:09.796455 19279 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet 8bd480f8e7a3487eb6b74a7ac7544b0d (table default.Table_1c876062216b4a3cbfab9086131b4bb0 [id=3fd25e8e73f946b8ba379c2876e1386e]). Expected version 2 got 0
I20260925 13:56:09.797911 19675 tablet_service.cc:1511] Processing CreateTablet for tablet c0f05269ea6c43a98d2137eae7a9a401 (DEFAULT_TABLE table=default.Table_72b4fdbfbca745a286dca8c191dc2286 [id=41786d7cdac8440ca86768cac5b28f49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:09.798969 19675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0f05269ea6c43a98d2137eae7a9a401. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:09.798827 19820 tablet_service.cc:1511] Processing CreateTablet for tablet c0f05269ea6c43a98d2137eae7a9a401 (DEFAULT_TABLE table=default.Table_72b4fdbfbca745a286dca8c191dc2286 [id=41786d7cdac8440ca86768cac5b28f49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:09.799819 19820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0f05269ea6c43a98d2137eae7a9a401. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:09.805884 19958 ts_tablet_manager.cc:1929] T ce8fe54ef2a84dbcbec5d866c37a13d5 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:09.806350 19958 log.cc:1199] T ce8fe54ef2a84dbcbec5d866c37a13d5 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/ce8fe54ef2a84dbcbec5d866c37a13d5
I20260925 13:56:09.807312 19958 ts_tablet_manager.cc:1950] T ce8fe54ef2a84dbcbec5d866c37a13d5 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:09.816514 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet ce8fe54ef2a84dbcbec5d866c37a13d5 (table default.Table_b998047d4cac4412a6d2872638486667 [id=b6f3c8cdd85b4a6f87c491267f6cddab]) successfully deleted
I20260925 13:56:09.819896 19958 tablet_replica.cc:333] T 98180fc321cf49b39f9efaf6a86d33f9 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:09.823261 19959 ts_tablet_manager.cc:1929] T 98180fc321cf49b39f9efaf6a86d33f9 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:09.824218 19958 raft_consensus.cc:2243] T 98180fc321cf49b39f9efaf6a86d33f9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:09.824733 19958 raft_consensus.cc:2272] T 98180fc321cf49b39f9efaf6a86d33f9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:09.829495 19958 ts_tablet_manager.cc:1916] T 98180fc321cf49b39f9efaf6a86d33f9 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:09.826102 19938 tablet_bootstrap.cc:492] T c0f05269ea6c43a98d2137eae7a9a401 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:09.800837 19534 tablet_service.cc:1511] Processing CreateTablet for tablet c0f05269ea6c43a98d2137eae7a9a401 (DEFAULT_TABLE table=default.Table_72b4fdbfbca745a286dca8c191dc2286 [id=41786d7cdac8440ca86768cac5b28f49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:09.832159 19534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0f05269ea6c43a98d2137eae7a9a401. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:09.809971 19533 tablet_service.cc:1558] Processing DeleteTablet for tablet ce8fe54ef2a84dbcbec5d866c37a13d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:09 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:09.840970 19938 tablet_bootstrap.cc:654] T c0f05269ea6c43a98d2137eae7a9a401 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:09.845957 19959 log.cc:1199] T 98180fc321cf49b39f9efaf6a86d33f9 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/98180fc321cf49b39f9efaf6a86d33f9
I20260925 13:56:09.851866 19279 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_cb55c90b6da345868b11789d1177730d" } modify_external_catalogs: true
I20260925 13:56:09.855443 19959 ts_tablet_manager.cc:1950] T 98180fc321cf49b39f9efaf6a86d33f9 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:09.855587 19279 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_cb55c90b6da345868b11789d1177730d" } modify_external_catalogs: true
I20260925 13:56:09.859874 19943 raft_consensus.cc:493] T 8bd480f8e7a3487eb6b74a7ac7544b0d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:09.860383 19943 raft_consensus.cc:515] T 8bd480f8e7a3487eb6b74a7ac7544b0d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:09.862344 19942 tablet_bootstrap.cc:492] T 5a7907a50912430bbeeac44d2ebd47de P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:09.863415 19943 leader_election.cc:290] T 8bd480f8e7a3487eb6b74a7ac7544b0d P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:09.871330 19942 tablet_bootstrap.cc:654] T 5a7907a50912430bbeeac44d2ebd47de P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:09.870108 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 98180fc321cf49b39f9efaf6a86d33f9 (table default.Table_ec0fe54be8374ac391e843ea5f82f061 [id=d1f1ad2082e84994a06d872c94b1d847]) successfully deleted
I20260925 13:56:09.881325 19271 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_064df254a7ee406d92f3ff5d401fdfe7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:09.875263 19278 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 8bd480f8e7a3487eb6b74a7ac7544b0d (table default.Table_1c876062216b4a3cbfab9086131b4bb0 [id=3fd25e8e73f946b8ba379c2876e1386e]). Expected version 2 got 0
I20260925 13:56:09.881240 19272 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_f68a67145c3941479bfbfbbbaf7e69df" } new_table_name: "default.Table_7bd4f946a6a4438692e33e986b637e42" modify_external_catalogs: true
W20260925 13:56:09.882918 19271 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_064df254a7ee406d92f3ff5d401fdfe7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:09.884423 19938 tablet_bootstrap.cc:492] T c0f05269ea6c43a98d2137eae7a9a401 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:09.889561 19958 ts_tablet_manager.cc:1929] T 98180fc321cf49b39f9efaf6a86d33f9 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:09.890180 19958 log.cc:1199] T 98180fc321cf49b39f9efaf6a86d33f9 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/98180fc321cf49b39f9efaf6a86d33f9
I20260925 13:56:09.891157 19958 ts_tablet_manager.cc:1950] T 98180fc321cf49b39f9efaf6a86d33f9 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:09.891031 19533 tablet_service.cc:1511] Processing CreateTablet for tablet b8200d0044434788adc1f7ec29aefa59 (DEFAULT_TABLE table=default.Table_cb55c90b6da345868b11789d1177730d [id=36a1da175eaf48548cdd40e1203d422a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:09.892370 19533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8200d0044434788adc1f7ec29aefa59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:09.899734 19938 ts_tablet_manager.cc:1403] T c0f05269ea6c43a98d2137eae7a9a401 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.074s user 0.010s sys 0.005s
I20260925 13:56:09.903380 19938 raft_consensus.cc:359] T c0f05269ea6c43a98d2137eae7a9a401 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:09.907508 19938 raft_consensus.cc:385] T c0f05269ea6c43a98d2137eae7a9a401 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:09.907648 19277 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_aef7d8ec17544ce79239c2bf81006b3b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:09.908033 19938 raft_consensus.cc:740] T c0f05269ea6c43a98d2137eae7a9a401 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
W20260925 13:56:09.909281 19277 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aef7d8ec17544ce79239c2bf81006b3b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:09.908975 19938 consensus_queue.cc:260] T c0f05269ea6c43a98d2137eae7a9a401 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:09.911532 19938 ts_tablet_manager.cc:1434] T c0f05269ea6c43a98d2137eae7a9a401 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.011s user 0.008s sys 0.000s
I20260925 13:56:09.916513 19938 tablet_bootstrap.cc:492] T 5a7907a50912430bbeeac44d2ebd47de P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
W20260925 13:56:09.929572 19272 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 491b8b5c0f51408b882adee10ce80728 consensus state has no leader
I20260925 13:56:09.930339 19938 tablet_bootstrap.cc:654] T 5a7907a50912430bbeeac44d2ebd47de P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:09.930387 19532 tablet_service.cc:1511] Processing CreateTablet for tablet 491b8b5c0f51408b882adee10ce80728 (DEFAULT_TABLE table=default.Table_f68a67145c3941479bfbfbbbaf7e69df [id=dadfbd02a0a640949b1dc46c01db7bc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:09.931896 19532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 491b8b5c0f51408b882adee10ce80728. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:09.956198 19938 tablet_bootstrap.cc:492] T 5a7907a50912430bbeeac44d2ebd47de P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:09.956816 19938 ts_tablet_manager.cc:1403] T 5a7907a50912430bbeeac44d2ebd47de P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.041s user 0.016s sys 0.001s
I20260925 13:56:09.959781 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 98180fc321cf49b39f9efaf6a86d33f9 (table default.Table_ec0fe54be8374ac391e843ea5f82f061 [id=d1f1ad2082e84994a06d872c94b1d847]) successfully deleted
I20260925 13:56:09.962200 19674 tablet_service.cc:1511] Processing CreateTablet for tablet 491b8b5c0f51408b882adee10ce80728 (DEFAULT_TABLE table=default.Table_7bd4f946a6a4438692e33e986b637e42 [id=dadfbd02a0a640949b1dc46c01db7bc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:09.963416 19674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 491b8b5c0f51408b882adee10ce80728. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:09.965512 19819 tablet_service.cc:1511] Processing CreateTablet for tablet b8200d0044434788adc1f7ec29aefa59 (DEFAULT_TABLE table=default.Table_cb55c90b6da345868b11789d1177730d [id=36a1da175eaf48548cdd40e1203d422a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:09.966527 19819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8200d0044434788adc1f7ec29aefa59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:09.966380 19673 tablet_service.cc:1511] Processing CreateTablet for tablet f0cdc99cde2f4dabb819e3e171c5d8d9 (DEFAULT_TABLE table=default.Table_bc39392c68044dd5a26e193aaa135333 [id=7e7ee975d4bb457eb3cc4632e12f3cd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:09.966368 19938 raft_consensus.cc:359] T 5a7907a50912430bbeeac44d2ebd47de P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:09.967054 19938 raft_consensus.cc:385] T 5a7907a50912430bbeeac44d2ebd47de P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:09.967311 19938 raft_consensus.cc:740] T 5a7907a50912430bbeeac44d2ebd47de P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:09.968041 19938 consensus_queue.cc:260] T 5a7907a50912430bbeeac44d2ebd47de P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:09.970049 19938 ts_tablet_manager.cc:1434] T 5a7907a50912430bbeeac44d2ebd47de P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.013s user 0.004s sys 0.000s
I20260925 13:56:09.970458 19673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0cdc99cde2f4dabb819e3e171c5d8d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:09.977950 19820 tablet_service.cc:1511] Processing CreateTablet for tablet 491b8b5c0f51408b882adee10ce80728 (DEFAULT_TABLE table=default.Table_f68a67145c3941479bfbfbbbaf7e69df [id=dadfbd02a0a640949b1dc46c01db7bc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:09.978863 19820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 491b8b5c0f51408b882adee10ce80728. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:09.983158 19271 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_064df254a7ee406d92f3ff5d401fdfe7" } new_table_name: "default.Table_824449a97a864e3dbbf06e2ad678edc5" modify_external_catalogs: true
I20260925 13:56:09.992223 19818 tablet_service.cc:1511] Processing CreateTablet for tablet f0cdc99cde2f4dabb819e3e171c5d8d9 (DEFAULT_TABLE table=default.Table_bc39392c68044dd5a26e193aaa135333 [id=7e7ee975d4bb457eb3cc4632e12f3cd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:09.993256 19818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0cdc99cde2f4dabb819e3e171c5d8d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:10.004017 19817 tablet_service.cc:1558] Processing DeleteTablet for tablet 98180fc321cf49b39f9efaf6a86d33f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:09 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:10.005074 19817 tablet_service.cc:1558] Processing DeleteTablet for tablet ce8fe54ef2a84dbcbec5d866c37a13d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:09 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:10.005833 19959 tablet_replica.cc:333] T ce8fe54ef2a84dbcbec5d866c37a13d5 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:10.006500 19959 raft_consensus.cc:2243] T ce8fe54ef2a84dbcbec5d866c37a13d5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:10.006897 19959 raft_consensus.cc:2272] T ce8fe54ef2a84dbcbec5d866c37a13d5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus is shut down!
W20260925 13:56:10.007859 19266 catalog_manager.cc:5030] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): delete failed for tablet 98180fc321cf49b39f9efaf6a86d33f9 because the tablet was not found. No further retry: Not found: Tablet not found: 98180fc321cf49b39f9efaf6a86d33f9
I20260925 13:56:10.009070 19959 ts_tablet_manager.cc:1916] T ce8fe54ef2a84dbcbec5d866c37a13d5 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:10.009704 19696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bd480f8e7a3487eb6b74a7ac7544b0d" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:10.010548 19696 raft_consensus.cc:2468] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 0.
I20260925 13:56:10.013224 19773 leader_election.cc:304] T 8bd480f8e7a3487eb6b74a7ac7544b0d P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:10.013567 19938 tablet_bootstrap.cc:492] T 491b8b5c0f51408b882adee10ce80728 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:10.014199 19942 tablet_bootstrap.cc:492] T 5a7907a50912430bbeeac44d2ebd47de P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:10.014585 19942 ts_tablet_manager.cc:1403] T 5a7907a50912430bbeeac44d2ebd47de P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.152s user 0.012s sys 0.021s
I20260925 13:56:10.021399 19942 raft_consensus.cc:359] T 5a7907a50912430bbeeac44d2ebd47de P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:10.022778 19942 raft_consensus.cc:385] T 5a7907a50912430bbeeac44d2ebd47de P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:10.023072 19942 raft_consensus.cc:740] T 5a7907a50912430bbeeac44d2ebd47de P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:10.023041 19943 raft_consensus.cc:2804] T 8bd480f8e7a3487eb6b74a7ac7544b0d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:10.023576 19943 raft_consensus.cc:493] T 8bd480f8e7a3487eb6b74a7ac7544b0d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:10.024019 19943 raft_consensus.cc:3060] T 8bd480f8e7a3487eb6b74a7ac7544b0d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:10.031198 19942 consensus_queue.cc:260] T 5a7907a50912430bbeeac44d2ebd47de P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:10.033852 19938 tablet_bootstrap.cc:654] T 491b8b5c0f51408b882adee10ce80728 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:10.036007 19280 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_f9ae699facd148ca8bae38d481d136df"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:10.037978 19280 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f9ae699facd148ca8bae38d481d136df in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:10.041033 19276 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_3c6823adc2894e7e8048a056415f6a73"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:10.042735 19276 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3c6823adc2894e7e8048a056415f6a73 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:10.043772 19942 ts_tablet_manager.cc:1434] T 5a7907a50912430bbeeac44d2ebd47de P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.029s user 0.006s sys 0.006s
I20260925 13:56:10.064695 19272 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_bb731091526c4e0d95e68171b5b70a60" } modify_external_catalogs: true
I20260925 13:56:10.065430 19272 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_bb731091526c4e0d95e68171b5b70a60" } modify_external_catalogs: true
I20260925 13:56:10.064536 19943 raft_consensus.cc:515] T 8bd480f8e7a3487eb6b74a7ac7544b0d P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:09.962301 19676 tablet_service.cc:1511] Processing CreateTablet for tablet b8200d0044434788adc1f7ec29aefa59 (DEFAULT_TABLE table=default.Table_cb55c90b6da345868b11789d1177730d [id=36a1da175eaf48548cdd40e1203d422a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:10.068966 19676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8200d0044434788adc1f7ec29aefa59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:10.072444 19696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bd480f8e7a3487eb6b74a7ac7544b0d" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:10.072974 19696 raft_consensus.cc:3060] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:10.074227 19943 leader_election.cc:290] T 8bd480f8e7a3487eb6b74a7ac7544b0d P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:10.078730 19531 tablet_service.cc:1511] Processing CreateTablet for tablet f0cdc99cde2f4dabb819e3e171c5d8d9 (DEFAULT_TABLE table=default.Table_bc39392c68044dd5a26e193aaa135333 [id=7e7ee975d4bb457eb3cc4632e12f3cd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:10.080173 19531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0cdc99cde2f4dabb819e3e171c5d8d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:10.082249 19938 tablet_bootstrap.cc:492] T 491b8b5c0f51408b882adee10ce80728 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:10.083009 19938 ts_tablet_manager.cc:1403] T 491b8b5c0f51408b882adee10ce80728 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.070s user 0.017s sys 0.005s
I20260925 13:56:10.087319 19938 raft_consensus.cc:359] T 491b8b5c0f51408b882adee10ce80728 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
W20260925 13:56:10.093523 19271 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f51c5ad86ce64891a6393c1dc8d414e5 has no consensus state
I20260925 13:56:10.093961 19938 raft_consensus.cc:385] T 491b8b5c0f51408b882adee10ce80728 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:10.094558 19938 raft_consensus.cc:740] T 491b8b5c0f51408b882adee10ce80728 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:10.095960 19938 consensus_queue.cc:260] T 491b8b5c0f51408b882adee10ce80728 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:10.102035 19939 tablet_bootstrap.cc:492] T b8200d0044434788adc1f7ec29aefa59 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:10.103542 19938 ts_tablet_manager.cc:1434] T 491b8b5c0f51408b882adee10ce80728 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.020s user 0.005s sys 0.004s
I20260925 13:56:10.106527 19938 tablet_bootstrap.cc:492] T b8200d0044434788adc1f7ec29aefa59 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
W20260925 13:56:10.107843 19267 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 8bd480f8e7a3487eb6b74a7ac7544b0d (table default.Table_1c876062216b4a3cbfab9086131b4bb0 [id=3fd25e8e73f946b8ba379c2876e1386e], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 8bd480f8e7a3487eb6b74a7ac7544b0d consensus state has no leader
I20260925 13:56:10.118045 19939 tablet_bootstrap.cc:654] T b8200d0044434788adc1f7ec29aefa59 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:10.122491 19938 tablet_bootstrap.cc:654] T b8200d0044434788adc1f7ec29aefa59 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:10.133669 19271 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_5a29923b85f84545beefc8120b5b29fd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:10.137323 19277 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_bc39392c68044dd5a26e193aaa135333" } new_table_name: "default.Table_5d6ee0b4200c486ca1fdec8349603746" modify_external_catalogs: true
W20260925 13:56:10.140832 19271 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5a29923b85f84545beefc8120b5b29fd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:10.159292 19938 tablet_bootstrap.cc:492] T b8200d0044434788adc1f7ec29aefa59 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:10.159909 19938 ts_tablet_manager.cc:1403] T b8200d0044434788adc1f7ec29aefa59 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.054s user 0.005s sys 0.014s
I20260925 13:56:10.163333 19938 raft_consensus.cc:359] T b8200d0044434788adc1f7ec29aefa59 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:10.164428 19938 raft_consensus.cc:385] T b8200d0044434788adc1f7ec29aefa59 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:10.164798 19938 raft_consensus.cc:740] T b8200d0044434788adc1f7ec29aefa59 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:10.165683 19938 consensus_queue.cc:260] T b8200d0044434788adc1f7ec29aefa59 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:10.171448 19938 ts_tablet_manager.cc:1434] T b8200d0044434788adc1f7ec29aefa59 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.011s user 0.006s sys 0.003s
I20260925 13:56:10.172786 19938 tablet_bootstrap.cc:492] T f0cdc99cde2f4dabb819e3e171c5d8d9 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:10.174480 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bd480f8e7a3487eb6b74a7ac7544b0d" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:10.174748 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bd480f8e7a3487eb6b74a7ac7544b0d" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:10.175380 19555 raft_consensus.cc:2468] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 0.
I20260925 13:56:10.177515 19955 ts_tablet_manager.cc:1929] T 98180fc321cf49b39f9efaf6a86d33f9 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:10.177932 19955 log.cc:1199] T 98180fc321cf49b39f9efaf6a86d33f9 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/98180fc321cf49b39f9efaf6a86d33f9
I20260925 13:56:10.178815 19955 ts_tablet_manager.cc:1950] T 98180fc321cf49b39f9efaf6a86d33f9 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:10.179100 19938 tablet_bootstrap.cc:654] T f0cdc99cde2f4dabb819e3e171c5d8d9 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:10.181818 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet 98180fc321cf49b39f9efaf6a86d33f9 (table default.Table_ec0fe54be8374ac391e843ea5f82f061 [id=d1f1ad2082e84994a06d872c94b1d847]) successfully deleted
I20260925 13:56:10.191649 19279 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet b8200d0044434788adc1f7ec29aefa59
I20260925 13:56:10.193471 19819 tablet_service.cc:1558] Processing DeleteTablet for tablet b8200d0044434788adc1f7ec29aefa59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:09 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:10.196054 19672 tablet_service.cc:1558] Processing DeleteTablet for tablet b8200d0044434788adc1f7ec29aefa59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:09 UTC) from {username='slave'} at 127.0.0.1:45948
W20260925 13:56:10.203919 19264 catalog_manager.cc:5030] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet b8200d0044434788adc1f7ec29aefa59 because the tablet was not found. No further retry: Not found: Tablet not found: b8200d0044434788adc1f7ec29aefa59
I20260925 13:56:10.206055 19938 tablet_bootstrap.cc:492] T f0cdc99cde2f4dabb819e3e171c5d8d9 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:10.206673 19938 ts_tablet_manager.cc:1403] T f0cdc99cde2f4dabb819e3e171c5d8d9 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.034s user 0.010s sys 0.001s
I20260925 13:56:10.206760 19696 raft_consensus.cc:2468] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 1.
I20260925 13:56:10.208012 19773 leader_election.cc:304] T 8bd480f8e7a3487eb6b74a7ac7544b0d P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters: 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:10.209082 19943 raft_consensus.cc:2804] T 8bd480f8e7a3487eb6b74a7ac7544b0d P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:10.210103 19938 raft_consensus.cc:359] T f0cdc99cde2f4dabb819e3e171c5d8d9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:10.211031 19938 raft_consensus.cc:385] T f0cdc99cde2f4dabb819e3e171c5d8d9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:10.211411 19938 raft_consensus.cc:740] T f0cdc99cde2f4dabb819e3e171c5d8d9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:10.212455 19938 consensus_queue.cc:260] T f0cdc99cde2f4dabb819e3e171c5d8d9 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:10.222513 19938 ts_tablet_manager.cc:1434] T f0cdc99cde2f4dabb819e3e171c5d8d9 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.015s user 0.004s sys 0.008s
I20260925 13:56:10.229420 19939 tablet_bootstrap.cc:492] T b8200d0044434788adc1f7ec29aefa59 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:10.229856 19939 ts_tablet_manager.cc:1403] T b8200d0044434788adc1f7ec29aefa59 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.128s user 0.011s sys 0.000s
I20260925 13:56:10.233266 19978 raft_consensus.cc:493] T 5a7907a50912430bbeeac44d2ebd47de P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:10.234639 19978 raft_consensus.cc:515] T 5a7907a50912430bbeeac44d2ebd47de P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:10.239019 19978 leader_election.cc:290] T 5a7907a50912430bbeeac44d2ebd47de P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:10.239601 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a7907a50912430bbeeac44d2ebd47de" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:10.240851 19696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a7907a50912430bbeeac44d2ebd47de" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:10.241394 19696 raft_consensus.cc:2468] T 5a7907a50912430bbeeac44d2ebd47de P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 0.
I20260925 13:56:10.242451 19773 leader_election.cc:304] T 5a7907a50912430bbeeac44d2ebd47de P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:10.243842 19978 raft_consensus.cc:493] T f0cdc99cde2f4dabb819e3e171c5d8d9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260925 13:56:10.244012 19773 leader_election.cc:343] T 5a7907a50912430bbeeac44d2ebd47de P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:10.244484 19978 raft_consensus.cc:515] T f0cdc99cde2f4dabb819e3e171c5d8d9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:10.247992 19532 tablet_service.cc:1511] Processing CreateTablet for tablet b0dffbfd867040fd99e0fdc77e732f74 (DEFAULT_TABLE table=default.Table_bb731091526c4e0d95e68171b5b70a60 [id=80deba2b504744738a9fca818e546a7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:10.248639 19978 leader_election.cc:290] T f0cdc99cde2f4dabb819e3e171c5d8d9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:10.249186 19532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0dffbfd867040fd99e0fdc77e732f74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:10.249329 19696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0cdc99cde2f4dabb819e3e171c5d8d9" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:10.249549 19978 raft_consensus.cc:2804] T 5a7907a50912430bbeeac44d2ebd47de P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:10.249974 19978 raft_consensus.cc:493] T 5a7907a50912430bbeeac44d2ebd47de P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:10.250430 19978 raft_consensus.cc:3060] T 5a7907a50912430bbeeac44d2ebd47de P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
W20260925 13:56:10.250589 19773 leader_election.cc:343] T f0cdc99cde2f4dabb819e3e171c5d8d9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:10.251535 19773 leader_election.cc:343] T f0cdc99cde2f4dabb819e3e171c5d8d9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Not found: Tablet not found: f0cdc99cde2f4dabb819e3e171c5d8d9
I20260925 13:56:10.251914 19773 leader_election.cc:304] T f0cdc99cde2f4dabb819e3e171c5d8d9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:10.256167 19278 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet c0f05269ea6c43a98d2137eae7a9a401 (table default.Table_72b4fdbfbca745a286dca8c191dc2286 [id=41786d7cdac8440ca86768cac5b28f49]). Expected version 1 got 0
I20260925 13:56:10.260437 19943 raft_consensus.cc:697] T 8bd480f8e7a3487eb6b74a7ac7544b0d P f790493ba7ad403993b678f2df2b55be [term 1 LEADER]: Becoming Leader. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Running, Role: LEADER
I20260925 13:56:10.260517 19978 raft_consensus.cc:515] T 5a7907a50912430bbeeac44d2ebd47de P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:10.261466 19943 consensus_queue.cc:237] T 8bd480f8e7a3487eb6b74a7ac7544b0d P f790493ba7ad403993b678f2df2b55be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:10.265023 19978 leader_election.cc:290] T 5a7907a50912430bbeeac44d2ebd47de P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:10.266203 19942 tablet_bootstrap.cc:492] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:10.266099 19696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a7907a50912430bbeeac44d2ebd47de" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:10.266904 19696 raft_consensus.cc:3060] T 5a7907a50912430bbeeac44d2ebd47de P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:10.272655 19942 tablet_bootstrap.cc:654] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:10.259881 19278 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_ad1b0393fb8344a49339732eba99a331"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:10.278039 19696 raft_consensus.cc:2468] T 5a7907a50912430bbeeac44d2ebd47de P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 1.
I20260925 13:56:10.279904 19976 raft_consensus.cc:2749] T f0cdc99cde2f4dabb819e3e171c5d8d9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:10.279742 19773 leader_election.cc:304] T 5a7907a50912430bbeeac44d2ebd47de P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:10.281020 19978 raft_consensus.cc:2804] T 5a7907a50912430bbeeac44d2ebd47de P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:10.281636 19978 raft_consensus.cc:697] T 5a7907a50912430bbeeac44d2ebd47de P f790493ba7ad403993b678f2df2b55be [term 1 LEADER]: Becoming Leader. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Running, Role: LEADER
I20260925 13:56:10.282748 19978 consensus_queue.cc:237] T 5a7907a50912430bbeeac44d2ebd47de P f790493ba7ad403993b678f2df2b55be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:10.287895 19673 tablet_service.cc:1511] Processing CreateTablet for tablet b0dffbfd867040fd99e0fdc77e732f74 (DEFAULT_TABLE table=default.Table_bb731091526c4e0d95e68171b5b70a60 [id=80deba2b504744738a9fca818e546a7c]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 13:56:10.288666 19277 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f0cdc99cde2f4dabb819e3e171c5d8d9 consensus state has no leader
I20260925 13:56:10.288983 19673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0dffbfd867040fd99e0fdc77e732f74. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:10.278466 19278 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ad1b0393fb8344a49339732eba99a331 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:10.295658 19819 tablet_service.cc:1511] Processing CreateTablet for tablet b0dffbfd867040fd99e0fdc77e732f74 (DEFAULT_TABLE table=default.Table_bb731091526c4e0d95e68171b5b70a60 [id=80deba2b504744738a9fca818e546a7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:10.296723 19819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0dffbfd867040fd99e0fdc77e732f74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:10.300853 19271 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_aef7d8ec17544ce79239c2bf81006b3b" } new_table_name: "default.Table_8ac29bbd5011471a9b60f334fc78a3d3" modify_external_catalogs: true
I20260925 13:56:10.313714 19938 tablet_bootstrap.cc:492] T b0dffbfd867040fd99e0fdc77e732f74 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:10.318969 19938 tablet_bootstrap.cc:654] T b0dffbfd867040fd99e0fdc77e732f74 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:10.326200 19672 tablet_service.cc:1511] Processing CreateTablet for tablet f51c5ad86ce64891a6393c1dc8d414e5 (DEFAULT_TABLE table=default.Table_824449a97a864e3dbbf06e2ad678edc5 [id=af708d3ba4c54ba19c24eca862bbddc1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:10.327692 19672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f51c5ad86ce64891a6393c1dc8d414e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:10.338569 19942 tablet_bootstrap.cc:492] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:10.338948 19942 ts_tablet_manager.cc:1403] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.073s user 0.011s sys 0.000s
I20260925 13:56:10.341486 19942 raft_consensus.cc:359] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:10.342137 19942 raft_consensus.cc:385] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:10.342530 19942 raft_consensus.cc:740] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:10.347680 19277 catalog_manager.cc:5838] T 8bd480f8e7a3487eb6b74a7ac7544b0d P f790493ba7ad403993b678f2df2b55be reported cstate change: term changed from 0 to 1, leader changed from <none> to f790493ba7ad403993b678f2df2b55be (127.18.182.195). New cstate: current_term: 1 leader_uuid: "f790493ba7ad403993b678f2df2b55be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:10.349563 19938 tablet_bootstrap.cc:492] T b0dffbfd867040fd99e0fdc77e732f74 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:10.349898 19959 ts_tablet_manager.cc:1929] T ce8fe54ef2a84dbcbec5d866c37a13d5 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:10.349933 19938 ts_tablet_manager.cc:1403] T b0dffbfd867040fd99e0fdc77e732f74 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.036s user 0.014s sys 0.003s
I20260925 13:56:10.350095 19277 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 8bd480f8e7a3487eb6b74a7ac7544b0d (table default.Table_1c876062216b4a3cbfab9086131b4bb0 [id=3fd25e8e73f946b8ba379c2876e1386e]). Expected version 2 got 0
I20260925 13:56:10.350334 19959 log.cc:1199] T ce8fe54ef2a84dbcbec5d866c37a13d5 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/ce8fe54ef2a84dbcbec5d866c37a13d5
I20260925 13:56:10.351109 19277 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet f0cdc99cde2f4dabb819e3e171c5d8d9 (table default.Table_5d6ee0b4200c486ca1fdec8349603746 [id=7e7ee975d4bb457eb3cc4632e12f3cd5]). Expected version 1 got 0
I20260925 13:56:10.351302 19959 ts_tablet_manager.cc:1950] T ce8fe54ef2a84dbcbec5d866c37a13d5 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:10.352350 19938 raft_consensus.cc:359] T b0dffbfd867040fd99e0fdc77e732f74 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:10.352891 19938 raft_consensus.cc:385] T b0dffbfd867040fd99e0fdc77e732f74 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:10.353124 19938 raft_consensus.cc:740] T b0dffbfd867040fd99e0fdc77e732f74 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:10.353205 19277 catalog_manager.cc:5838] T 5a7907a50912430bbeeac44d2ebd47de P f790493ba7ad403993b678f2df2b55be reported cstate change: term changed from 0 to 1, leader changed from <none> to f790493ba7ad403993b678f2df2b55be (127.18.182.195). New cstate: current_term: 1 leader_uuid: "f790493ba7ad403993b678f2df2b55be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: HEALTHY } } }
I20260925 13:56:10.354615 19277 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 491b8b5c0f51408b882adee10ce80728 (table default.Table_7bd4f946a6a4438692e33e986b637e42 [id=dadfbd02a0a640949b1dc46c01db7bc8]). Expected version 1 got 0
I20260925 13:56:10.354415 19938 consensus_queue.cc:260] T b0dffbfd867040fd99e0fdc77e732f74 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:10.356194 19938 ts_tablet_manager.cc:1434] T b0dffbfd867040fd99e0fdc77e732f74 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
I20260925 13:56:10.356763 19819 tablet_service.cc:1511] Processing CreateTablet for tablet f51c5ad86ce64891a6393c1dc8d414e5 (DEFAULT_TABLE table=default.Table_824449a97a864e3dbbf06e2ad678edc5 [id=af708d3ba4c54ba19c24eca862bbddc1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:10.358340 19819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f51c5ad86ce64891a6393c1dc8d414e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:10.367630 19535 tablet_service.cc:1558] Processing DeleteTablet for tablet b8200d0044434788adc1f7ec29aefa59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:09 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:10.369570 19534 tablet_service.cc:1511] Processing CreateTablet for tablet f51c5ad86ce64891a6393c1dc8d414e5 (DEFAULT_TABLE table=default.Table_824449a97a864e3dbbf06e2ad678edc5 [id=af708d3ba4c54ba19c24eca862bbddc1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:10.370558 19939 raft_consensus.cc:359] T b8200d0044434788adc1f7ec29aefa59 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:10.378981 19938 tablet_bootstrap.cc:492] T f51c5ad86ce64891a6393c1dc8d414e5 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:10.379995 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a7907a50912430bbeeac44d2ebd47de" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:10.381768 19955 tablet_replica.cc:333] T ce8fe54ef2a84dbcbec5d866c37a13d5 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:10.382475 19955 raft_consensus.cc:2243] T ce8fe54ef2a84dbcbec5d866c37a13d5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:10.382929 19955 raft_consensus.cc:2272] T ce8fe54ef2a84dbcbec5d866c37a13d5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:10.384783 19955 ts_tablet_manager.cc:1916] T ce8fe54ef2a84dbcbec5d866c37a13d5 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:10.385337 19938 tablet_bootstrap.cc:654] T f51c5ad86ce64891a6393c1dc8d414e5 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:10.370579 19534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f51c5ad86ce64891a6393c1dc8d414e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:10.371263 19939 raft_consensus.cc:385] T b8200d0044434788adc1f7ec29aefa59 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:10.395596 19939 raft_consensus.cc:740] T b8200d0044434788adc1f7ec29aefa59 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:10.396102 19938 tablet_bootstrap.cc:492] T f51c5ad86ce64891a6393c1dc8d414e5 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:10.396463 19938 ts_tablet_manager.cc:1403] T f51c5ad86ce64891a6393c1dc8d414e5 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.018s user 0.012s sys 0.002s
I20260925 13:56:10.396412 19939 consensus_queue.cc:260] T b8200d0044434788adc1f7ec29aefa59 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:10.398936 19938 raft_consensus.cc:359] T f51c5ad86ce64891a6393c1dc8d414e5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:10.399617 19938 raft_consensus.cc:385] T f51c5ad86ce64891a6393c1dc8d414e5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:10.399868 19938 raft_consensus.cc:740] T f51c5ad86ce64891a6393c1dc8d414e5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:10.400564 19938 consensus_queue.cc:260] T f51c5ad86ce64891a6393c1dc8d414e5 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:10.422817 19531 tablet_service.cc:1511] Processing CreateTablet for tablet 6b52ed579b19479686247d3ecba9489a (DEFAULT_TABLE table=default.Table_c232f8cfe77b43e6ad45e9979d8e3364 [id=d804d75fe5004ea4a1dc3245acb39ae7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:10.423257 19938 ts_tablet_manager.cc:1434] T f51c5ad86ce64891a6393c1dc8d414e5 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.027s user 0.006s sys 0.000s
I20260925 13:56:10.426914 19531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b52ed579b19479686247d3ecba9489a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:10.434531 19276 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_9cdf64e560854f80a1c733b4d896f285"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:10.436390 19276 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9cdf64e560854f80a1c733b4d896f285 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:10.439935 19273 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_731e8683c6714e22a725eae1b86ff4ec"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:10.440866 19280 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_5a29923b85f84545beefc8120b5b29fd" } modify_external_catalogs: true
I20260925 13:56:10.441604 19280 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_5a29923b85f84545beefc8120b5b29fd" } modify_external_catalogs: true
W20260925 13:56:10.441954 19273 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_731e8683c6714e22a725eae1b86ff4ec in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:10.442523 19955 ts_tablet_manager.cc:1929] T ce8fe54ef2a84dbcbec5d866c37a13d5 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:10.442888 19955 log.cc:1199] T ce8fe54ef2a84dbcbec5d866c37a13d5 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/ce8fe54ef2a84dbcbec5d866c37a13d5
I20260925 13:56:10.443606 19955 ts_tablet_manager.cc:1950] T ce8fe54ef2a84dbcbec5d866c37a13d5 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:10.449960 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet ce8fe54ef2a84dbcbec5d866c37a13d5 (table default.Table_b998047d4cac4412a6d2872638486667 [id=b6f3c8cdd85b4a6f87c491267f6cddab]) successfully deleted
W20260925 13:56:10.451527 19773 leader_election.cc:343] T 5a7907a50912430bbeeac44d2ebd47de P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:10.453120 19939 ts_tablet_manager.cc:1434] T b8200d0044434788adc1f7ec29aefa59 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.223s user 0.008s sys 0.002s
I20260925 13:56:10.454044 19939 tablet_bootstrap.cc:492] T c0f05269ea6c43a98d2137eae7a9a401 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
W20260925 13:56:10.457262 19264 catalog_manager.cc:5043] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): delete failed for tablet b8200d0044434788adc1f7ec29aefa59 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b8200d0044434788adc1f7ec29aefa59 already in progress: creating tablet
I20260925 13:56:10.465718 19819 tablet_service.cc:1511] Processing CreateTablet for tablet 6b52ed579b19479686247d3ecba9489a (DEFAULT_TABLE table=default.Table_c232f8cfe77b43e6ad45e9979d8e3364 [id=d804d75fe5004ea4a1dc3245acb39ae7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:10.466802 19819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b52ed579b19479686247d3ecba9489a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:10.343312 19942 consensus_queue.cc:260] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:10.473985 19942 ts_tablet_manager.cc:1434] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.135s user 0.003s sys 0.003s
I20260925 13:56:10.474967 19942 tablet_bootstrap.cc:492] T c0f05269ea6c43a98d2137eae7a9a401 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:10.475740 19959 tablet_replica.cc:333] T b8200d0044434788adc1f7ec29aefa59 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:10.476513 19959 raft_consensus.cc:2243] T b8200d0044434788adc1f7ec29aefa59 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:10.476953 19959 raft_consensus.cc:2272] T b8200d0044434788adc1f7ec29aefa59 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:10.478483 19272 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet b0dffbfd867040fd99e0fdc77e732f74
I20260925 13:56:10.488131 19818 tablet_service.cc:1558] Processing DeleteTablet for tablet b0dffbfd867040fd99e0fdc77e732f74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:10 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:10.490082 19674 tablet_service.cc:1558] Processing DeleteTablet for tablet b0dffbfd867040fd99e0fdc77e732f74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:10 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:10.491417 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet ce8fe54ef2a84dbcbec5d866c37a13d5 (table default.Table_b998047d4cac4412a6d2872638486667 [id=b6f3c8cdd85b4a6f87c491267f6cddab]) successfully deleted
I20260925 13:56:10.492547 19939 tablet_bootstrap.cc:654] T c0f05269ea6c43a98d2137eae7a9a401 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
W20260925 13:56:10.498342 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet b0dffbfd867040fd99e0fdc77e732f74 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b0dffbfd867040fd99e0fdc77e732f74 already in progress: creating tablet
I20260925 13:56:10.500943 19274 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_1c876062216b4a3cbfab9086131b4bb0" } new_table_name: "default.Table_f7190a6e9e55451c8177b52fbaa2a61d" modify_external_catalogs: true
I20260925 13:56:10.501758 19676 tablet_service.cc:1511] Processing CreateTablet for tablet 6b52ed579b19479686247d3ecba9489a (DEFAULT_TABLE table=default.Table_c232f8cfe77b43e6ad45e9979d8e3364 [id=d804d75fe5004ea4a1dc3245acb39ae7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:10.502800 19676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b52ed579b19479686247d3ecba9489a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:10.512828 19275 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_31807711126b487380c586fcf1939670"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:10.514889 19275 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_31807711126b487380c586fcf1939670 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 13:56:10.518550 19271 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ff3d09046b854fdaab2f95feff589ac1 has no consensus state
I20260925 13:56:10.522377 19532 tablet_service.cc:1558] Processing DeleteTablet for tablet b0dffbfd867040fd99e0fdc77e732f74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:10 UTC) from {username='slave'} at 127.0.0.1:40018
W20260925 13:56:10.524506 19264 catalog_manager.cc:5043] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): delete failed for tablet b0dffbfd867040fd99e0fdc77e732f74 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b0dffbfd867040fd99e0fdc77e732f74 already in progress: creating tablet
I20260925 13:56:10.525902 19942 tablet_bootstrap.cc:654] T c0f05269ea6c43a98d2137eae7a9a401 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:10.551381 19959 ts_tablet_manager.cc:1916] T b8200d0044434788adc1f7ec29aefa59 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:10.554289 19939 tablet_bootstrap.cc:492] T c0f05269ea6c43a98d2137eae7a9a401 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:10.555400 19942 tablet_bootstrap.cc:492] T c0f05269ea6c43a98d2137eae7a9a401 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:10.555873 19942 ts_tablet_manager.cc:1403] T c0f05269ea6c43a98d2137eae7a9a401 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.081s user 0.007s sys 0.003s
I20260925 13:56:10.558382 19942 raft_consensus.cc:359] T c0f05269ea6c43a98d2137eae7a9a401 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:10.561012 19939 ts_tablet_manager.cc:1403] T c0f05269ea6c43a98d2137eae7a9a401 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.107s user 0.026s sys 0.007s
I20260925 13:56:10.561216 19938 tablet_bootstrap.cc:492] T 6b52ed579b19479686247d3ecba9489a P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:10.563676 19939 raft_consensus.cc:359] T c0f05269ea6c43a98d2137eae7a9a401 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:10.571630 19272 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet f0cdc99cde2f4dabb819e3e171c5d8d9 (table default.Table_5d6ee0b4200c486ca1fdec8349603746 [id=7e7ee975d4bb457eb3cc4632e12f3cd5]). Expected version 1 got 0
I20260925 13:56:10.575073 19938 tablet_bootstrap.cc:654] T 6b52ed579b19479686247d3ecba9489a P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:10.581578 19272 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet f0cdc99cde2f4dabb819e3e171c5d8d9 (table default.Table_5d6ee0b4200c486ca1fdec8349603746 [id=7e7ee975d4bb457eb3cc4632e12f3cd5]). Expected version 1 got 0
I20260925 13:56:10.582778 19532 tablet_service.cc:1558] Processing DeleteTablet for tablet b8200d0044434788adc1f7ec29aefa59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:09 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:10.584823 19955 tablet_replica.cc:333] T b8200d0044434788adc1f7ec29aefa59 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:10.585528 19955 raft_consensus.cc:2243] T b8200d0044434788adc1f7ec29aefa59 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:10.585978 19955 raft_consensus.cc:2272] T b8200d0044434788adc1f7ec29aefa59 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:10.578729 19942 raft_consensus.cc:385] T c0f05269ea6c43a98d2137eae7a9a401 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:10.587057 19942 raft_consensus.cc:740] T c0f05269ea6c43a98d2137eae7a9a401 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:10.588153 19955 ts_tablet_manager.cc:1916] T b8200d0044434788adc1f7ec29aefa59 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:10.591921 19942 consensus_queue.cc:260] T c0f05269ea6c43a98d2137eae7a9a401 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:10.603456 19939 raft_consensus.cc:385] T c0f05269ea6c43a98d2137eae7a9a401 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:10.603762 19939 raft_consensus.cc:740] T c0f05269ea6c43a98d2137eae7a9a401 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:10.604473 19939 consensus_queue.cc:260] T c0f05269ea6c43a98d2137eae7a9a401 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:10.609723 19939 ts_tablet_manager.cc:1434] T c0f05269ea6c43a98d2137eae7a9a401 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.048s user 0.005s sys 0.000s
I20260925 13:56:10.616024 19939 tablet_bootstrap.cc:492] T 491b8b5c0f51408b882adee10ce80728 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:10.643791 19280 catalog_manager.cc:6128] Not sending DeleteTablet requests; no consensus state for tablet df671987dfcc4adcb64fe267d03bb054
I20260925 13:56:10.644488 19938 tablet_bootstrap.cc:492] T 6b52ed579b19479686247d3ecba9489a P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:10.644855 19938 ts_tablet_manager.cc:1403] T 6b52ed579b19479686247d3ecba9489a P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.084s user 0.010s sys 0.000s
I20260925 13:56:10.650360 19959 ts_tablet_manager.cc:1929] T b8200d0044434788adc1f7ec29aefa59 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:10.651559 19942 ts_tablet_manager.cc:1434] T c0f05269ea6c43a98d2137eae7a9a401 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.095s user 0.006s sys 0.000s
I20260925 13:56:10.654845 19955 ts_tablet_manager.cc:1929] T b8200d0044434788adc1f7ec29aefa59 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:10.657428 19942 tablet_bootstrap.cc:492] T b0dffbfd867040fd99e0fdc77e732f74 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:10.670733 19696 raft_consensus.cc:1275] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Refusing update from remote peer f790493ba7ad403993b678f2df2b55be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:56:10.672371 19955 log.cc:1199] T b8200d0044434788adc1f7ec29aefa59 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/b8200d0044434788adc1f7ec29aefa59
I20260925 13:56:10.668020 19555 raft_consensus.cc:3060] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:10.663338 19942 tablet_bootstrap.cc:654] T b0dffbfd867040fd99e0fdc77e732f74 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:10.681133 19555 raft_consensus.cc:1275] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Refusing update from remote peer f790493ba7ad403993b678f2df2b55be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:56:10.657790 19939 tablet_bootstrap.cc:654] T 491b8b5c0f51408b882adee10ce80728 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:10.682129 19955 ts_tablet_manager.cc:1950] T b8200d0044434788adc1f7ec29aefa59 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:10.687885 19942 tablet_bootstrap.cc:492] T b0dffbfd867040fd99e0fdc77e732f74 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:10.688001 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet b8200d0044434788adc1f7ec29aefa59 (table default.Table_cb55c90b6da345868b11789d1177730d [id=36a1da175eaf48548cdd40e1203d422a]) successfully deleted
I20260925 13:56:10.688413 19942 ts_tablet_manager.cc:1403] T b0dffbfd867040fd99e0fdc77e732f74 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.031s user 0.014s sys 0.002s
I20260925 13:56:10.689981 19275 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_e9dc3916a32d4c6e8d9517c521a20220" } modify_external_catalogs: true
I20260925 13:56:10.690781 19275 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_e9dc3916a32d4c6e8d9517c521a20220" } modify_external_catalogs: true
I20260925 13:56:10.690788 19942 raft_consensus.cc:359] T b0dffbfd867040fd99e0fdc77e732f74 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:10.691432 19942 raft_consensus.cc:385] T b0dffbfd867040fd99e0fdc77e732f74 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:10.691800 19942 raft_consensus.cc:740] T b0dffbfd867040fd99e0fdc77e732f74 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:10.692529 19942 consensus_queue.cc:260] T b0dffbfd867040fd99e0fdc77e732f74 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:10.695559 19959 log.cc:1199] T b8200d0044434788adc1f7ec29aefa59 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/b8200d0044434788adc1f7ec29aefa59
I20260925 13:56:10.696003 19277 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet f51c5ad86ce64891a6393c1dc8d414e5 (table default.Table_824449a97a864e3dbbf06e2ad678edc5 [id=af708d3ba4c54ba19c24eca862bbddc1]). Expected version 1 got 0
I20260925 13:56:10.696278 19977 raft_consensus.cc:493] T c0f05269ea6c43a98d2137eae7a9a401 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:10.696376 19959 ts_tablet_manager.cc:1950] T b8200d0044434788adc1f7ec29aefa59 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:10.696710 19977 raft_consensus.cc:515] T c0f05269ea6c43a98d2137eae7a9a401 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:10.698589 19977 leader_election.cc:290] T c0f05269ea6c43a98d2137eae7a9a401 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:10.698807 19959 tablet_replica.cc:333] T b0dffbfd867040fd99e0fdc77e732f74 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:10.647114 19938 raft_consensus.cc:359] T 6b52ed579b19479686247d3ecba9489a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:10.699523 19938 raft_consensus.cc:385] T 6b52ed579b19479686247d3ecba9489a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:10.699862 19938 raft_consensus.cc:740] T 6b52ed579b19479686247d3ecba9489a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:10.700673 19938 consensus_queue.cc:260] T 6b52ed579b19479686247d3ecba9489a P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:10.702416 19938 ts_tablet_manager.cc:1434] T 6b52ed579b19479686247d3ecba9489a P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.057s user 0.006s sys 0.000s
I20260925 13:56:10.672716 19978 consensus_queue.cc:1048] T 8bd480f8e7a3487eb6b74a7ac7544b0d P f790493ba7ad403993b678f2df2b55be [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:10.704420 19959 raft_consensus.cc:2243] T b0dffbfd867040fd99e0fdc77e732f74 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:10.704859 19959 raft_consensus.cc:2272] T b0dffbfd867040fd99e0fdc77e732f74 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:10.707911 19959 ts_tablet_manager.cc:1916] T b0dffbfd867040fd99e0fdc77e732f74 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:10.707855 19696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f05269ea6c43a98d2137eae7a9a401" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:10.708423 19696 raft_consensus.cc:2468] T c0f05269ea6c43a98d2137eae7a9a401 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 0.
I20260925 13:56:10.709247 19943 consensus_queue.cc:1048] T 8bd480f8e7a3487eb6b74a7ac7544b0d P f790493ba7ad403993b678f2df2b55be [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:10.720086 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet b8200d0044434788adc1f7ec29aefa59 (table default.Table_cb55c90b6da345868b11789d1177730d [id=36a1da175eaf48548cdd40e1203d422a]) successfully deleted
I20260925 13:56:10.722791 19773 leader_election.cc:304] T c0f05269ea6c43a98d2137eae7a9a401 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:10.725394 19978 raft_consensus.cc:2804] T c0f05269ea6c43a98d2137eae7a9a401 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:10.725697 19978 raft_consensus.cc:493] T c0f05269ea6c43a98d2137eae7a9a401 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:10.725581 19274 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_ad1b0393fb8344a49339732eba99a331" } new_table_name: "default.Table_733f29f6f9d44bcdb82c0adb9fed84aa" modify_external_catalogs: true
I20260925 13:56:10.725991 19978 raft_consensus.cc:3060] T c0f05269ea6c43a98d2137eae7a9a401 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:10.744695 19978 raft_consensus.cc:515] T c0f05269ea6c43a98d2137eae7a9a401 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:10.745678 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f05269ea6c43a98d2137eae7a9a401" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:10.746320 19555 raft_consensus.cc:2468] T c0f05269ea6c43a98d2137eae7a9a401 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 0.
I20260925 13:56:10.752643 19942 ts_tablet_manager.cc:1434] T b0dffbfd867040fd99e0fdc77e732f74 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.064s user 0.004s sys 0.000s
I20260925 13:56:10.754869 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f05269ea6c43a98d2137eae7a9a401" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:10.755537 19555 raft_consensus.cc:3060] T c0f05269ea6c43a98d2137eae7a9a401 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:10.757289 19280 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_31da32681ca34ed7a777c23510ef59ce"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:10.759243 19280 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_31da32681ca34ed7a777c23510ef59ce in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:10.761750 19939 tablet_bootstrap.cc:492] T 491b8b5c0f51408b882adee10ce80728 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:10.762149 19939 ts_tablet_manager.cc:1403] T 491b8b5c0f51408b882adee10ce80728 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.146s user 0.015s sys 0.008s
I20260925 13:56:10.762439 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f05269ea6c43a98d2137eae7a9a401" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:10.762945 19978 leader_election.cc:290] T c0f05269ea6c43a98d2137eae7a9a401 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:10.763080 19695 raft_consensus.cc:3060] T c0f05269ea6c43a98d2137eae7a9a401 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:10.764870 19939 raft_consensus.cc:359] T 491b8b5c0f51408b882adee10ce80728 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:10.765502 19939 raft_consensus.cc:385] T 491b8b5c0f51408b882adee10ce80728 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:10.765758 19939 raft_consensus.cc:740] T 491b8b5c0f51408b882adee10ce80728 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:10.766598 19942 tablet_bootstrap.cc:492] T 491b8b5c0f51408b882adee10ce80728 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:10.769512 19695 raft_consensus.cc:2468] T c0f05269ea6c43a98d2137eae7a9a401 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 1.
I20260925 13:56:10.770686 19773 leader_election.cc:304] T c0f05269ea6c43a98d2137eae7a9a401 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:10.771657 19978 raft_consensus.cc:2804] T c0f05269ea6c43a98d2137eae7a9a401 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:10.773265 19978 raft_consensus.cc:697] T c0f05269ea6c43a98d2137eae7a9a401 P f790493ba7ad403993b678f2df2b55be [term 1 LEADER]: Becoming Leader. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Running, Role: LEADER
I20260925 13:56:10.774384 19978 consensus_queue.cc:237] T c0f05269ea6c43a98d2137eae7a9a401 P f790493ba7ad403993b678f2df2b55be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:10.780839 19555 raft_consensus.cc:2468] T c0f05269ea6c43a98d2137eae7a9a401 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 1.
I20260925 13:56:10.783565 19942 tablet_bootstrap.cc:654] T 491b8b5c0f51408b882adee10ce80728 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:10.786692 19959 ts_tablet_manager.cc:1929] T b0dffbfd867040fd99e0fdc77e732f74 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:10.787631 19993 mvcc.cc:204] Tried to move back new op lower bound from 7333251361442250752 to 7333251359800459264. Current Snapshot: MvccSnapshot[applied={T|T < 7333251361442250752}]
I20260925 13:56:10.793666 19959 log.cc:1199] T b0dffbfd867040fd99e0fdc77e732f74 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/b0dffbfd867040fd99e0fdc77e732f74
I20260925 13:56:10.794531 19959 ts_tablet_manager.cc:1950] T b0dffbfd867040fd99e0fdc77e732f74 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:10.783643 19939 consensus_queue.cc:260] T 491b8b5c0f51408b882adee10ce80728 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:10.797156 19939 ts_tablet_manager.cc:1434] T 491b8b5c0f51408b882adee10ce80728 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.035s user 0.006s sys 0.001s
I20260925 13:56:10.798624 19939 tablet_bootstrap.cc:492] T 5a7907a50912430bbeeac44d2ebd47de P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:10.805420 19273 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_4f3bc32149e141ef897cf43378f9020c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:10.807051 19273 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4f3bc32149e141ef897cf43378f9020c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:10.807577 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet b0dffbfd867040fd99e0fdc77e732f74 (table default.Table_bb731091526c4e0d95e68171b5b70a60 [id=80deba2b504744738a9fca818e546a7c]) successfully deleted
I20260925 13:56:10.809046 19939 tablet_bootstrap.cc:654] T 5a7907a50912430bbeeac44d2ebd47de P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
W20260925 13:56:10.816001 19274 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d9455b98734147fb9e2d47c69513424a has no consensus state
I20260925 13:56:10.820118 20000 tablet.cc:1751] T 8bd480f8e7a3487eb6b74a7ac7544b0d P f790493ba7ad403993b678f2df2b55be: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260925 13:56:10.826249 19939 tablet_bootstrap.cc:492] T 5a7907a50912430bbeeac44d2ebd47de P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:10.826700 19939 ts_tablet_manager.cc:1403] T 5a7907a50912430bbeeac44d2ebd47de P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.028s user 0.007s sys 0.004s
I20260925 13:56:10.828936 19939 raft_consensus.cc:359] T 5a7907a50912430bbeeac44d2ebd47de P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:10.831022 19939 raft_consensus.cc:385] T 5a7907a50912430bbeeac44d2ebd47de P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:10.831311 19939 raft_consensus.cc:740] T 5a7907a50912430bbeeac44d2ebd47de P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:10.832068 19939 consensus_queue.cc:260] T 5a7907a50912430bbeeac44d2ebd47de P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:10.834306 19280 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_fa74b192993b433681028dd598b6a56b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:10.836701 19280 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fa74b192993b433681028dd598b6a56b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:10.839427 19942 tablet_bootstrap.cc:492] T 491b8b5c0f51408b882adee10ce80728 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:10.839850 19942 ts_tablet_manager.cc:1403] T 491b8b5c0f51408b882adee10ce80728 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.073s user 0.007s sys 0.003s
I20260925 13:56:10.839955 19939 ts_tablet_manager.cc:1434] T 5a7907a50912430bbeeac44d2ebd47de P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.013s user 0.005s sys 0.000s
I20260925 13:56:10.842268 19942 raft_consensus.cc:359] T 491b8b5c0f51408b882adee10ce80728 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:10.843247 19939 tablet_bootstrap.cc:492] T b0dffbfd867040fd99e0fdc77e732f74 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:10.843523 19942 raft_consensus.cc:385] T 491b8b5c0f51408b882adee10ce80728 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:10.852809 19942 raft_consensus.cc:740] T 491b8b5c0f51408b882adee10ce80728 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:10.853652 19942 consensus_queue.cc:260] T 491b8b5c0f51408b882adee10ce80728 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:10.858349 19942 ts_tablet_manager.cc:1434] T 491b8b5c0f51408b882adee10ce80728 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.018s user 0.004s sys 0.001s
I20260925 13:56:10.860109 19939 tablet_bootstrap.cc:654] T b0dffbfd867040fd99e0fdc77e732f74 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:10.861817 19942 tablet_bootstrap.cc:492] T b8200d0044434788adc1f7ec29aefa59 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:10.879685 19939 tablet_bootstrap.cc:492] T b0dffbfd867040fd99e0fdc77e732f74 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:10.880185 19939 ts_tablet_manager.cc:1403] T b0dffbfd867040fd99e0fdc77e732f74 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.037s user 0.015s sys 0.000s
I20260925 13:56:10.881217 19942 tablet_bootstrap.cc:654] T b8200d0044434788adc1f7ec29aefa59 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:10.882812 19939 raft_consensus.cc:359] T b0dffbfd867040fd99e0fdc77e732f74 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:10.883536 19939 raft_consensus.cc:385] T b0dffbfd867040fd99e0fdc77e732f74 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:10.883831 19939 raft_consensus.cc:740] T b0dffbfd867040fd99e0fdc77e732f74 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:10.884585 19939 consensus_queue.cc:260] T b0dffbfd867040fd99e0fdc77e732f74 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:10.891822 19532 tablet_service.cc:1511] Processing CreateTablet for tablet 3b10ecb0e6064d42a2521c4c4e91c028 (DEFAULT_TABLE table=default.Table_f9ae699facd148ca8bae38d481d136df [id=7a20968885384e23a7a65f63337a67d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:10.893148 19532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b10ecb0e6064d42a2521c4c4e91c028. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:10.900123 19942 tablet_bootstrap.cc:492] T b8200d0044434788adc1f7ec29aefa59 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:10.900707 19942 ts_tablet_manager.cc:1403] T b8200d0044434788adc1f7ec29aefa59 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.039s user 0.012s sys 0.001s
I20260925 13:56:10.893164 19939 ts_tablet_manager.cc:1434] T b0dffbfd867040fd99e0fdc77e732f74 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.013s user 0.001s sys 0.003s
I20260925 13:56:10.903717 19939 tablet_bootstrap.cc:492] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:10.905507 19942 raft_consensus.cc:359] T b8200d0044434788adc1f7ec29aefa59 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:10.906126 19942 raft_consensus.cc:385] T b8200d0044434788adc1f7ec29aefa59 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:10.906366 19942 raft_consensus.cc:740] T b8200d0044434788adc1f7ec29aefa59 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:10.909050 19943 raft_consensus.cc:493] T 491b8b5c0f51408b882adee10ce80728 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:10.909493 19943 raft_consensus.cc:515] T 491b8b5c0f51408b882adee10ce80728 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:10.911382 19943 leader_election.cc:290] T 491b8b5c0f51408b882adee10ce80728 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:10.912562 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "491b8b5c0f51408b882adee10ce80728" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:10.913139 19555 raft_consensus.cc:2468] T 491b8b5c0f51408b882adee10ce80728 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 0.
I20260925 13:56:10.914073 19696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "491b8b5c0f51408b882adee10ce80728" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:10.914600 19696 raft_consensus.cc:2468] T 491b8b5c0f51408b882adee10ce80728 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 0.
I20260925 13:56:10.916003 19773 leader_election.cc:304] T 491b8b5c0f51408b882adee10ce80728 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:10.916594 19676 tablet_service.cc:1511] Processing CreateTablet for tablet 3b10ecb0e6064d42a2521c4c4e91c028 (DEFAULT_TABLE table=default.Table_f9ae699facd148ca8bae38d481d136df [id=7a20968885384e23a7a65f63337a67d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:10.917352 19943 raft_consensus.cc:2804] T 491b8b5c0f51408b882adee10ce80728 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:10.917650 19943 raft_consensus.cc:493] T 491b8b5c0f51408b882adee10ce80728 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:10.917917 19676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b10ecb0e6064d42a2521c4c4e91c028. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:10.917924 19943 raft_consensus.cc:3060] T 491b8b5c0f51408b882adee10ce80728 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:10.924108 19943 raft_consensus.cc:515] T 491b8b5c0f51408b882adee10ce80728 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:10.926599 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "491b8b5c0f51408b882adee10ce80728" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:10.927095 19555 raft_consensus.cc:3060] T 491b8b5c0f51408b882adee10ce80728 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:10.906968 19942 consensus_queue.cc:260] T b8200d0044434788adc1f7ec29aefa59 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:10.867774 19274 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_31da32681ca34ed7a777c23510ef59ce" } modify_external_catalogs: true
I20260925 13:56:10.928064 19274 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_31da32681ca34ed7a777c23510ef59ce" } modify_external_catalogs: true
I20260925 13:56:10.930634 19942 ts_tablet_manager.cc:1434] T b8200d0044434788adc1f7ec29aefa59 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.030s user 0.006s sys 0.000s
I20260925 13:56:10.931720 19942 tablet_bootstrap.cc:492] T f51c5ad86ce64891a6393c1dc8d414e5 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:10.868992 19273 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_733f29f6f9d44bcdb82c0adb9fed84aa" } new_table_name: "default.Table_c1c831f6c04f4135801dcafbaf7a0843" modify_external_catalogs: true
I20260925 13:56:10.886849 19277 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_ab754389c42247ccbed64c8f7c539d90"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:10.888811 19276 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet c0f05269ea6c43a98d2137eae7a9a401 (table default.Table_72b4fdbfbca745a286dca8c191dc2286 [id=41786d7cdac8440ca86768cac5b28f49]). Expected version 1 got 0
I20260925 13:56:10.940706 19943 leader_election.cc:290] T 491b8b5c0f51408b882adee10ce80728 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:10.941293 19279 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet c0f05269ea6c43a98d2137eae7a9a401 (table default.Table_72b4fdbfbca745a286dca8c191dc2286 [id=41786d7cdac8440ca86768cac5b28f49]). Expected version 1 got 0
I20260925 13:56:10.943066 19555 raft_consensus.cc:2468] T 491b8b5c0f51408b882adee10ce80728 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 1.
I20260925 13:56:10.948057 19272 catalog_manager.cc:5838] T c0f05269ea6c43a98d2137eae7a9a401 P f790493ba7ad403993b678f2df2b55be reported cstate change: term changed from 0 to 1, leader changed from <none> to f790493ba7ad403993b678f2df2b55be (127.18.182.195). New cstate: current_term: 1 leader_uuid: "f790493ba7ad403993b678f2df2b55be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: HEALTHY } } }
I20260925 13:56:10.949733 19942 tablet_bootstrap.cc:654] T f51c5ad86ce64891a6393c1dc8d414e5 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:10.949752 19272 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet c0f05269ea6c43a98d2137eae7a9a401 (table default.Table_72b4fdbfbca745a286dca8c191dc2286 [id=41786d7cdac8440ca86768cac5b28f49]). Expected version 1 got 0
I20260925 13:56:10.954054 19773 leader_election.cc:304] T 491b8b5c0f51408b882adee10ce80728 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:10.955399 19943 raft_consensus.cc:2804] T 491b8b5c0f51408b882adee10ce80728 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:10.955865 19943 raft_consensus.cc:697] T 491b8b5c0f51408b882adee10ce80728 P f790493ba7ad403993b678f2df2b55be [term 1 LEADER]: Becoming Leader. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Running, Role: LEADER
I20260925 13:56:10.956650 19943 consensus_queue.cc:237] T 491b8b5c0f51408b882adee10ce80728 P f790493ba7ad403993b678f2df2b55be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
W20260925 13:56:10.960613 19277 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ab754389c42247ccbed64c8f7c539d90 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:10.955838 19696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "491b8b5c0f51408b882adee10ce80728" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:10.962499 19696 raft_consensus.cc:3060] T 491b8b5c0f51408b882adee10ce80728 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:10.970666 19696 raft_consensus.cc:2468] T 491b8b5c0f51408b882adee10ce80728 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 1.
I20260925 13:56:10.979472 19819 tablet_service.cc:1511] Processing CreateTablet for tablet 3b10ecb0e6064d42a2521c4c4e91c028 (DEFAULT_TABLE table=default.Table_f9ae699facd148ca8bae38d481d136df [id=7a20968885384e23a7a65f63337a67d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:10.980927 19819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b10ecb0e6064d42a2521c4c4e91c028. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:10.987622 19531 tablet_service.cc:1511] Processing CreateTablet for tablet 293b017a83cc464a9f023a0923310796 (DEFAULT_TABLE table=default.Table_3c6823adc2894e7e8048a056415f6a73 [id=d2bf5d31983a47d897e93ace83ecf026]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:10.988796 19531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 293b017a83cc464a9f023a0923310796. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:11.004128 19275 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 5a7907a50912430bbeeac44d2ebd47de
I20260925 13:56:11.009114 19534 tablet_service.cc:1558] Processing DeleteTablet for tablet 5a7907a50912430bbeeac44d2ebd47de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:10 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:11.009824 19955 tablet_replica.cc:333] T 5a7907a50912430bbeeac44d2ebd47de P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:11.010576 19955 raft_consensus.cc:2243] T 5a7907a50912430bbeeac44d2ebd47de P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:11.011021 19955 raft_consensus.cc:2272] T 5a7907a50912430bbeeac44d2ebd47de P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:11.011857 19818 tablet_service.cc:1558] Processing DeleteTablet for tablet 5a7907a50912430bbeeac44d2ebd47de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:10 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:11.013186 19955 ts_tablet_manager.cc:1916] T 5a7907a50912430bbeeac44d2ebd47de P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:11.013706 19959 tablet_replica.cc:333] T 5a7907a50912430bbeeac44d2ebd47de P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:11.014441 19959 raft_consensus.cc:2243] T 5a7907a50912430bbeeac44d2ebd47de P f790493ba7ad403993b678f2df2b55be [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:11.015180 19959 pending_rounds.cc:70] T 5a7907a50912430bbeeac44d2ebd47de P f790493ba7ad403993b678f2df2b55be: Trying to abort 1 pending ops.
I20260925 13:56:11.015503 19959 pending_rounds.cc:77] T 5a7907a50912430bbeeac44d2ebd47de P f790493ba7ad403993b678f2df2b55be: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251359893016576 op_type: NO_OP noop_request { }
I20260925 13:56:11.015892 19959 raft_consensus.cc:2889] T 5a7907a50912430bbeeac44d2ebd47de P f790493ba7ad403993b678f2df2b55be [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:56:11.015996 19274 catalog_manager.cc:6128] Not sending DeleteTablet requests; no consensus state for tablet 484f47f521c547b1b63e9a6958222d37
I20260925 13:56:11.016275 19959 raft_consensus.cc:2272] T 5a7907a50912430bbeeac44d2ebd47de P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:11.019016 19938 tablet_bootstrap.cc:492] T 3b10ecb0e6064d42a2521c4c4e91c028 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:11.019804 19275 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_7bd4f946a6a4438692e33e986b637e42" } new_table_name: "default.Table_81fefa73a0904acb9b300de03fb48254" modify_external_catalogs: true
I20260925 13:56:11.020821 19959 ts_tablet_manager.cc:1916] T 5a7907a50912430bbeeac44d2ebd47de P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:11.024521 19938 tablet_bootstrap.cc:654] T 3b10ecb0e6064d42a2521c4c4e91c028 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:11.024142 19279 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_7bebf65cf469407ca242743b60c27282"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:11.025733 19279 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7bebf65cf469407ca242743b60c27282 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:11.031895 20001 tablet.cc:1751] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 7c1b52895a14482fb54a6758f5054f8f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260925 13:56:11.032624 19674 tablet_service.cc:1558] Processing DeleteTablet for tablet 5a7907a50912430bbeeac44d2ebd47de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:10 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:11.032820 19672 tablet_service.cc:1511] Processing CreateTablet for tablet 293b017a83cc464a9f023a0923310796 (DEFAULT_TABLE table=default.Table_3c6823adc2894e7e8048a056415f6a73 [id=d2bf5d31983a47d897e93ace83ecf026]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:11.034636 19969 raft_consensus.cc:493] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:11.035235 19969 raft_consensus.cc:515] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:11.053694 19939 tablet_bootstrap.cc:654] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:11.054705 19955 ts_tablet_manager.cc:1929] T 5a7907a50912430bbeeac44d2ebd47de P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:11.055244 19938 tablet_bootstrap.cc:492] T 3b10ecb0e6064d42a2521c4c4e91c028 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:11.056890 19938 ts_tablet_manager.cc:1403] T 3b10ecb0e6064d42a2521c4c4e91c028 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.038s user 0.012s sys 0.000s
I20260925 13:56:11.059062 19276 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_04d053f331f243ce8e98beb7c00d4159"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:11.060060 19672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 293b017a83cc464a9f023a0923310796. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:11.060292 19955 log.cc:1199] T 5a7907a50912430bbeeac44d2ebd47de P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/5a7907a50912430bbeeac44d2ebd47de
I20260925 13:56:11.061148 19955 ts_tablet_manager.cc:1950] T 5a7907a50912430bbeeac44d2ebd47de P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
W20260925 13:56:11.061100 19276 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_04d053f331f243ce8e98beb7c00d4159 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:11.063014 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet 5a7907a50912430bbeeac44d2ebd47de (table default.Table_e9dc3916a32d4c6e8d9517c521a20220 [id=00c39e12d41b4d27b6cd28a850c76fe4]) successfully deleted
I20260925 13:56:11.059120 19959 ts_tablet_manager.cc:1929] T 5a7907a50912430bbeeac44d2ebd47de P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:11.059768 19938 raft_consensus.cc:359] T 3b10ecb0e6064d42a2521c4c4e91c028 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:11.066707 19959 log.cc:1199] T 5a7907a50912430bbeeac44d2ebd47de P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/5a7907a50912430bbeeac44d2ebd47de
I20260925 13:56:11.066761 19938 raft_consensus.cc:385] T 3b10ecb0e6064d42a2521c4c4e91c028 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:11.067165 19938 raft_consensus.cc:740] T 3b10ecb0e6064d42a2521c4c4e91c028 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:11.071381 20012 tablet.cc:1751] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 8f578869bc084a5d8a9a69f143954eb3: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260925 13:56:11.072690 19938 consensus_queue.cc:260] T 3b10ecb0e6064d42a2521c4c4e91c028 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:11.080042 19938 ts_tablet_manager.cc:1434] T 3b10ecb0e6064d42a2521c4c4e91c028 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.022s user 0.005s sys 0.000s
I20260925 13:56:11.081020 19969 leader_election.cc:290] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:11.067512 19959 ts_tablet_manager.cc:1950] T 5a7907a50912430bbeeac44d2ebd47de P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
W20260925 13:56:11.108938 19265 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 491b8b5c0f51408b882adee10ce80728 (table default.Table_7bd4f946a6a4438692e33e986b637e42 [id=dadfbd02a0a640949b1dc46c01db7bc8], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 491b8b5c0f51408b882adee10ce80728 consensus state has no leader
I20260925 13:56:11.110018 19532 tablet_service.cc:1511] Processing CreateTablet for tablet ff3d09046b854fdaab2f95feff589ac1 (DEFAULT_TABLE table=default.Table_8ac29bbd5011471a9b60f334fc78a3d3 [id=689d277ceae34fc0bb1f2ecd43295722]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:11.111470 19532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff3d09046b854fdaab2f95feff589ac1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:11.117190 19676 tablet_service.cc:1511] Processing CreateTablet for tablet ff3d09046b854fdaab2f95feff589ac1 (DEFAULT_TABLE table=default.Table_8ac29bbd5011471a9b60f334fc78a3d3 [id=689d277ceae34fc0bb1f2ecd43295722]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:11.118465 19676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff3d09046b854fdaab2f95feff589ac1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:11.119623 20011 tablet_replica.cc:333] T 5a7907a50912430bbeeac44d2ebd47de P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:11.120718 20011 raft_consensus.cc:2243] T 5a7907a50912430bbeeac44d2ebd47de P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:11.121313 20011 raft_consensus.cc:2272] T 5a7907a50912430bbeeac44d2ebd47de P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:11.137101 19942 tablet_bootstrap.cc:492] T f51c5ad86ce64891a6393c1dc8d414e5 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:11.137486 19942 ts_tablet_manager.cc:1403] T f51c5ad86ce64891a6393c1dc8d414e5 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.206s user 0.007s sys 0.007s
I20260925 13:56:11.139701 19942 raft_consensus.cc:359] T f51c5ad86ce64891a6393c1dc8d414e5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:11.140292 19942 raft_consensus.cc:385] T f51c5ad86ce64891a6393c1dc8d414e5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:11.140523 19942 raft_consensus.cc:740] T f51c5ad86ce64891a6393c1dc8d414e5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:11.141098 19942 consensus_queue.cc:260] T f51c5ad86ce64891a6393c1dc8d414e5 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:11.142781 19942 ts_tablet_manager.cc:1434] T f51c5ad86ce64891a6393c1dc8d414e5 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260925 13:56:11.143909 19942 tablet_bootstrap.cc:492] T 6b52ed579b19479686247d3ecba9489a P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:11.144783 19274 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet 8bd480f8e7a3487eb6b74a7ac7544b0d (table default.Table_f7190a6e9e55451c8177b52fbaa2a61d [id=3fd25e8e73f946b8ba379c2876e1386e]). Expected version 3 got 0
I20260925 13:56:11.145702 19274 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet 491b8b5c0f51408b882adee10ce80728 (table default.Table_7bd4f946a6a4438692e33e986b637e42 [id=dadfbd02a0a640949b1dc46c01db7bc8]). Expected version 1 got 0
I20260925 13:56:11.149014 19942 tablet_bootstrap.cc:654] T 6b52ed579b19479686247d3ecba9489a P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:11.161319 19819 tablet_service.cc:1511] Processing CreateTablet for tablet 293b017a83cc464a9f023a0923310796 (DEFAULT_TABLE table=default.Table_3c6823adc2894e7e8048a056415f6a73 [id=d2bf5d31983a47d897e93ace83ecf026]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:11.161783 20011 ts_tablet_manager.cc:1916] T 5a7907a50912430bbeeac44d2ebd47de P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:11.162285 19819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 293b017a83cc464a9f023a0923310796. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:11.164193 19818 tablet_service.cc:1511] Processing CreateTablet for tablet ff3d09046b854fdaab2f95feff589ac1 (DEFAULT_TABLE table=default.Table_8ac29bbd5011471a9b60f334fc78a3d3 [id=689d277ceae34fc0bb1f2ecd43295722]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:11.165376 19818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff3d09046b854fdaab2f95feff589ac1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:11.175611 19939 tablet_bootstrap.cc:492] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:11.176056 19939 ts_tablet_manager.cc:1403] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.273s user 0.010s sys 0.000s
I20260925 13:56:11.178551 19939 raft_consensus.cc:359] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:11.179131 19939 raft_consensus.cc:385] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260925 13:56:11.185873 19273 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d9455b98734147fb9e2d47c69513424a has no consensus state
I20260925 13:56:11.190863 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 5a7907a50912430bbeeac44d2ebd47de (table default.Table_e9dc3916a32d4c6e8d9517c521a20220 [id=00c39e12d41b4d27b6cd28a850c76fe4]) successfully deleted
I20260925 13:56:11.193658 19939 raft_consensus.cc:740] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:11.194278 19939 consensus_queue.cc:260] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:11.195719 19938 tablet_bootstrap.cc:492] T 293b017a83cc464a9f023a0923310796 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:11.202415 19938 tablet_bootstrap.cc:654] T 293b017a83cc464a9f023a0923310796 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:11.230707 19939 ts_tablet_manager.cc:1434] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.054s user 0.005s sys 0.000s
I20260925 13:56:11.243848 19276 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_f7190a6e9e55451c8177b52fbaa2a61d" } modify_external_catalogs: true
I20260925 13:56:11.244385 19276 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_f7190a6e9e55451c8177b52fbaa2a61d" } modify_external_catalogs: true
I20260925 13:56:11.248934 19939 tablet_bootstrap.cc:492] T f51c5ad86ce64891a6393c1dc8d414e5 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:11.250097 19279 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_f9ae699facd148ca8bae38d481d136df" } modify_external_catalogs: true
I20260925 13:56:11.250619 19279 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_f9ae699facd148ca8bae38d481d136df" } modify_external_catalogs: true
I20260925 13:56:11.263175 19938 tablet_bootstrap.cc:492] T 293b017a83cc464a9f023a0923310796 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:11.263664 19938 ts_tablet_manager.cc:1403] T 293b017a83cc464a9f023a0923310796 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.068s user 0.014s sys 0.000s
I20260925 13:56:11.266855 19938 raft_consensus.cc:359] T 293b017a83cc464a9f023a0923310796 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:11.267706 19938 raft_consensus.cc:385] T 293b017a83cc464a9f023a0923310796 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:11.268013 19938 raft_consensus.cc:740] T 293b017a83cc464a9f023a0923310796 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:11.270653 19942 tablet_bootstrap.cc:492] T 6b52ed579b19479686247d3ecba9489a P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:11.274880 19969 raft_consensus.cc:493] T b0dffbfd867040fd99e0fdc77e732f74 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:11.275296 19969 raft_consensus.cc:515] T b0dffbfd867040fd99e0fdc77e732f74 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:11.277132 19969 leader_election.cc:290] T b0dffbfd867040fd99e0fdc77e732f74 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:11.278472 19840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0cdc99cde2f4dabb819e3e171c5d8d9" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:11.279211 19840 raft_consensus.cc:2468] T f0cdc99cde2f4dabb819e3e171c5d8d9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:11.281065 19939 tablet_bootstrap.cc:654] T f51c5ad86ce64891a6393c1dc8d414e5 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:11.282394 19278 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet c0f05269ea6c43a98d2137eae7a9a401 (table default.Table_72b4fdbfbca745a286dca8c191dc2286 [id=41786d7cdac8440ca86768cac5b28f49]). Expected version 1 got 0
I20260925 13:56:11.283077 19278 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 8bd480f8e7a3487eb6b74a7ac7544b0d (table default.Table_f7190a6e9e55451c8177b52fbaa2a61d [id=3fd25e8e73f946b8ba379c2876e1386e]). Expected version 3 got 0
I20260925 13:56:11.283622 19938 consensus_queue.cc:260] T 293b017a83cc464a9f023a0923310796 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:11.288995 19819 tablet_service.cc:1511] Processing CreateTablet for tablet b0a293e0094a4d9e937af6454e1ea16c (DEFAULT_TABLE table=default.Table_4f3bc32149e141ef897cf43378f9020c [id=b673ac6542c5403fbf0fad16c4917dfd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:11.289945 19819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0a293e0094a4d9e937af6454e1ea16c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:11.290076 19531 tablet_service.cc:1511] Processing CreateTablet for tablet b0a293e0094a4d9e937af6454e1ea16c (DEFAULT_TABLE table=default.Table_4f3bc32149e141ef897cf43378f9020c [id=b673ac6542c5403fbf0fad16c4917dfd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:11.293185 19531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0a293e0094a4d9e937af6454e1ea16c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:11.293288 19976 raft_consensus.cc:493] T 6b52ed579b19479686247d3ecba9489a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:11.293680 19976 raft_consensus.cc:515] T 6b52ed579b19479686247d3ecba9489a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:11.295580 19976 leader_election.cc:290] T 6b52ed579b19479686247d3ecba9489a P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:11.296734 19532 tablet_service.cc:1511] Processing CreateTablet for tablet f4429db544534f758883f2fae184ea6d (DEFAULT_TABLE table=default.Table_731e8683c6714e22a725eae1b86ff4ec [id=e441f4e1a35e400da9cd488ac1d4a754]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:11.297787 19532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4429db544534f758883f2fae184ea6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:11.300400 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b52ed579b19479686247d3ecba9489a" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:11.300879 19818 tablet_service.cc:1511] Processing CreateTablet for tablet f4429db544534f758883f2fae184ea6d (DEFAULT_TABLE table=default.Table_731e8683c6714e22a725eae1b86ff4ec [id=e441f4e1a35e400da9cd488ac1d4a754]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:11.301100 19820 tablet_service.cc:1511] Processing CreateTablet for tablet 28ac2912b94a4928967efb7a731c42b9 (DEFAULT_TABLE table=default.Table_31807711126b487380c586fcf1939670 [id=56afa81b40064727a7facfe3ccb22571]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:11.301946 19818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4429db544534f758883f2fae184ea6d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:11.302217 19773 leader_election.cc:343] T 6b52ed579b19479686247d3ecba9489a P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:11.302881 19820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28ac2912b94a4928967efb7a731c42b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:11.304389 19534 tablet_service.cc:1511] Processing CreateTablet for tablet 28ac2912b94a4928967efb7a731c42b9 (DEFAULT_TABLE table=default.Table_31807711126b487380c586fcf1939670 [id=56afa81b40064727a7facfe3ccb22571]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:11.305444 19534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28ac2912b94a4928967efb7a731c42b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:11.305588 19631 leader_election.cc:304] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
W20260925 13:56:11.306677 19631 leader_election.cc:343] T b0dffbfd867040fd99e0fdc77e732f74 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: b0dffbfd867040fd99e0fdc77e732f74
I20260925 13:56:11.314252 19278 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 491b8b5c0f51408b882adee10ce80728 (table default.Table_7bd4f946a6a4438692e33e986b637e42 [id=dadfbd02a0a640949b1dc46c01db7bc8]). Expected version 1 got 0
I20260925 13:56:11.314971 19272 catalog_manager.cc:6391] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: default.Table_f7190a6e9e55451c8177b52fbaa2a61d [id=3fd25e8e73f946b8ba379c2876e1386e] alter complete (version 3)
W20260925 13:56:11.315819 19275 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 491b8b5c0f51408b882adee10ce80728 consensus state has no leader
I20260925 13:56:11.319239 19279 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 3b10ecb0e6064d42a2521c4c4e91c028
I20260925 13:56:11.321238 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b52ed579b19479686247d3ecba9489a" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:11.321933 19969 raft_consensus.cc:2804] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:11.322942 19817 tablet_service.cc:1558] Processing DeleteTablet for tablet 3b10ecb0e6064d42a2521c4c4e91c028 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:11 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:11.323827 19959 tablet_replica.cc:333] T 3b10ecb0e6064d42a2521c4c4e91c028 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:11.324741 19959 raft_consensus.cc:2243] T 3b10ecb0e6064d42a2521c4c4e91c028 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:11.325264 19959 raft_consensus.cc:2272] T 3b10ecb0e6064d42a2521c4c4e91c028 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:11.327718 19959 ts_tablet_manager.cc:1916] T 3b10ecb0e6064d42a2521c4c4e91c028 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:11.328286 19942 ts_tablet_manager.cc:1403] T 6b52ed579b19479686247d3ecba9489a P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.185s user 0.013s sys 0.000s
I20260925 13:56:11.339800 19535 tablet_service.cc:1558] Processing DeleteTablet for tablet 3b10ecb0e6064d42a2521c4c4e91c028 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:11 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:11.340250 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0cdc99cde2f4dabb819e3e171c5d8d9" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:11.341053 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0dffbfd867040fd99e0fdc77e732f74" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:11.341964 19555 raft_consensus.cc:2468] T b0dffbfd867040fd99e0fdc77e732f74 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:11.346304 19629 leader_election.cc:304] T b0dffbfd867040fd99e0fdc77e732f74 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3; no voters: f790493ba7ad403993b678f2df2b55be
I20260925 13:56:11.348969 19672 tablet_service.cc:1511] Processing CreateTablet for tablet b0a293e0094a4d9e937af6454e1ea16c (DEFAULT_TABLE table=default.Table_4f3bc32149e141ef897cf43378f9020c [id=b673ac6542c5403fbf0fad16c4917dfd]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 13:56:11.351775 19264 catalog_manager.cc:5043] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): delete failed for tablet 3b10ecb0e6064d42a2521c4c4e91c028 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3b10ecb0e6064d42a2521c4c4e91c028 already in progress: creating tablet
I20260925 13:56:11.353979 19533 tablet_service.cc:1511] Processing CreateTablet for tablet d9455b98734147fb9e2d47c69513424a (DEFAULT_TABLE table=default.Table_c1c831f6c04f4135801dcafbaf7a0843 [id=018679db079c42d2b70631ecb8f241a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:11.322306 19969 raft_consensus.cc:493] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:11.355062 19533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9455b98734147fb9e2d47c69513424a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:11.356483 19530 tablet_service.cc:1558] Processing DeleteTablet for tablet b0dffbfd867040fd99e0fdc77e732f74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:10 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:11.356897 19535 tablet_service.cc:1511] Processing CreateTablet for tablet 51a4c21e06ca4b5b971cf8f04ffa52a9 (DEFAULT_TABLE table=default.Table_9cdf64e560854f80a1c733b4d896f285 [id=fe0f78d1c6e44f589aede852f77f88fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:11.357357 19529 tablet_service.cc:1558] Processing DeleteTablet for tablet 5a7907a50912430bbeeac44d2ebd47de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:10 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:11.358099 19535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51a4c21e06ca4b5b971cf8f04ffa52a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:11.358193 19955 tablet_replica.cc:333] T b0dffbfd867040fd99e0fdc77e732f74 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:11.359041 19955 raft_consensus.cc:2243] T b0dffbfd867040fd99e0fdc77e732f74 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:11.359593 19955 raft_consensus.cc:2272] T b0dffbfd867040fd99e0fdc77e732f74 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:11.362778 19955 ts_tablet_manager.cc:1916] T b0dffbfd867040fd99e0fdc77e732f74 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 13:56:11.363701 19773 leader_election.cc:343] T 6b52ed579b19479686247d3ecba9489a P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:11.363925 19817 tablet_service.cc:1511] Processing CreateTablet for tablet 51a4c21e06ca4b5b971cf8f04ffa52a9 (DEFAULT_TABLE table=default.Table_9cdf64e560854f80a1c733b4d896f285 [id=fe0f78d1c6e44f589aede852f77f88fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:11.364092 19773 leader_election.cc:304] T 6b52ed579b19479686247d3ecba9489a P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:11.350498 19674 tablet_service.cc:1511] Processing CreateTablet for tablet f4429db544534f758883f2fae184ea6d (DEFAULT_TABLE table=default.Table_731e8683c6714e22a725eae1b86ff4ec [id=e441f4e1a35e400da9cd488ac1d4a754]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:11.355823 19673 tablet_service.cc:1558] Processing DeleteTablet for tablet 3b10ecb0e6064d42a2521c4c4e91c028 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:11 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:11.365849 19978 raft_consensus.cc:2749] T 6b52ed579b19479686247d3ecba9489a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:11.366104 19817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51a4c21e06ca4b5b971cf8f04ffa52a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:11.341018 19554 raft_consensus.cc:2468] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:11.369060 19815 tablet_service.cc:1511] Processing CreateTablet for tablet d9455b98734147fb9e2d47c69513424a (DEFAULT_TABLE table=default.Table_c1c831f6c04f4135801dcafbaf7a0843 [id=018679db079c42d2b70631ecb8f241a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:11.370426 19815 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9455b98734147fb9e2d47c69513424a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:11.360070 19976 raft_consensus.cc:493] T f51c5ad86ce64891a6393c1dc8d414e5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:11.377074 19938 ts_tablet_manager.cc:1434] T 293b017a83cc464a9f023a0923310796 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.113s user 0.006s sys 0.001s
I20260925 13:56:11.381279 19938 tablet_bootstrap.cc:492] T ff3d09046b854fdaab2f95feff589ac1 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:11.373498 19976 raft_consensus.cc:515] T f51c5ad86ce64891a6393c1dc8d414e5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:11.383599 19976 leader_election.cc:290] T f51c5ad86ce64891a6393c1dc8d414e5 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:11.386394 19938 tablet_bootstrap.cc:654] T ff3d09046b854fdaab2f95feff589ac1 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:11.393784 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f51c5ad86ce64891a6393c1dc8d414e5" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
W20260925 13:56:11.395274 19773 leader_election.cc:343] T f51c5ad86ce64891a6393c1dc8d414e5 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:11.403244 19279 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_8a18eca530ed42b7808ab612d9bad5e7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:11.403612 19275 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_edddbd4b770d44a3907e4d08fba4c0c7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:11.405371 19275 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_edddbd4b770d44a3907e4d08fba4c0c7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 13:56:11.406030 19279 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8a18eca530ed42b7808ab612d9bad5e7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:11.408201 19273 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_7bebf65cf469407ca242743b60c27282" } new_table_name: "default.Table_abc00b1f4c4c455e8086a46c395f2656" modify_external_catalogs: true
I20260925 13:56:11.376009 19969 raft_consensus.cc:3060] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:11.411592 19955 ts_tablet_manager.cc:1929] T b0dffbfd867040fd99e0fdc77e732f74 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:11.412012 19955 log.cc:1199] T b0dffbfd867040fd99e0fdc77e732f74 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/b0dffbfd867040fd99e0fdc77e732f74
I20260925 13:56:11.412812 19955 ts_tablet_manager.cc:1950] T b0dffbfd867040fd99e0fdc77e732f74 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:11.419777 19274 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet f0cdc99cde2f4dabb819e3e171c5d8d9 (table default.Table_5d6ee0b4200c486ca1fdec8349603746 [id=7e7ee975d4bb457eb3cc4632e12f3cd5]). Expected version 1 got 0
I20260925 13:56:11.420298 19942 raft_consensus.cc:359] T 6b52ed579b19479686247d3ecba9489a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:11.420958 19942 raft_consensus.cc:385] T 6b52ed579b19479686247d3ecba9489a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:11.421206 19942 raft_consensus.cc:740] T 6b52ed579b19479686247d3ecba9489a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:11.422252 20024 raft_consensus.cc:2804] T b0dffbfd867040fd99e0fdc77e732f74 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:11.421583 19272 catalog_manager.cc:5838] T 491b8b5c0f51408b882adee10ce80728 P f790493ba7ad403993b678f2df2b55be reported cstate change: term changed from 0 to 1, leader changed from <none> to f790493ba7ad403993b678f2df2b55be (127.18.182.195). New cstate: current_term: 1 leader_uuid: "f790493ba7ad403993b678f2df2b55be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:11.422590 20024 raft_consensus.cc:493] T b0dffbfd867040fd99e0fdc77e732f74 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:11.423040 19272 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 491b8b5c0f51408b882adee10ce80728 (table default.Table_81fefa73a0904acb9b300de03fb48254 [id=dadfbd02a0a640949b1dc46c01db7bc8]). Expected version 2 got 0
I20260925 13:56:11.423238 20024 raft_consensus.cc:3060] T b0dffbfd867040fd99e0fdc77e732f74 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:11.424149 19276 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 8bd480f8e7a3487eb6b74a7ac7544b0d
I20260925 13:56:11.425467 19938 tablet_bootstrap.cc:492] T ff3d09046b854fdaab2f95feff589ac1 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:11.425856 19938 ts_tablet_manager.cc:1403] T ff3d09046b854fdaab2f95feff589ac1 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.045s user 0.008s sys 0.000s
I20260925 13:56:11.428185 19673 tablet_service.cc:1511] Processing CreateTablet for tablet 28ac2912b94a4928967efb7a731c42b9 (DEFAULT_TABLE table=default.Table_31807711126b487380c586fcf1939670 [id=56afa81b40064727a7facfe3ccb22571]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:11.433058 19942 consensus_queue.cc:260] T 6b52ed579b19479686247d3ecba9489a P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:11.434669 19629 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.193:38269, user_credentials={real_user=slave}} blocked reactor thread for 89360.6us
I20260925 13:56:11.436553 19819 tablet_service.cc:1558] Processing DeleteTablet for tablet 8bd480f8e7a3487eb6b74a7ac7544b0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:11 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:11.438333 19672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0a293e0094a4d9e937af6454e1ea16c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:11.439152 19673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28ac2912b94a4928967efb7a731c42b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:11.440721 19531 tablet_service.cc:1558] Processing DeleteTablet for tablet 8bd480f8e7a3487eb6b74a7ac7544b0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:11 UTC) from {username='slave'} at 127.0.0.1:40018
W20260925 13:56:11.443768 19273 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 18a6d1c4eb7948149e709a380da50d93 has no consensus state
I20260925 13:56:11.439289 19938 raft_consensus.cc:359] T ff3d09046b854fdaab2f95feff589ac1 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:11.444582 19938 raft_consensus.cc:385] T ff3d09046b854fdaab2f95feff589ac1 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:11.444856 19938 raft_consensus.cc:740] T ff3d09046b854fdaab2f95feff589ac1 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:11.446475 19938 consensus_queue.cc:260] T ff3d09046b854fdaab2f95feff589ac1 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:11.446969 19275 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_d2952c2050c5481ab85394e93890ce9b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:11.447553 19674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4429db544534f758883f2fae184ea6d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:11.448843 19275 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d2952c2050c5481ab85394e93890ce9b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:11.452735 19675 tablet_service.cc:1511] Processing CreateTablet for tablet 51a4c21e06ca4b5b971cf8f04ffa52a9 (DEFAULT_TABLE table=default.Table_9cdf64e560854f80a1c733b4d896f285 [id=fe0f78d1c6e44f589aede852f77f88fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:11.453743 19675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51a4c21e06ca4b5b971cf8f04ffa52a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:11.455286 19955 tablet_replica.cc:333] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:11.456072 19955 raft_consensus.cc:2243] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:11.456760 19955 raft_consensus.cc:2272] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:11.462167 19938 ts_tablet_manager.cc:1434] T ff3d09046b854fdaab2f95feff589ac1 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.036s user 0.006s sys 0.000s
I20260925 13:56:11.479318 19959 ts_tablet_manager.cc:1929] T 3b10ecb0e6064d42a2521c4c4e91c028 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:11.486250 19273 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_06cdecf307e549bf908eb64143c31996"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:11.488400 19273 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_06cdecf307e549bf908eb64143c31996 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:11.489138 19959 log.cc:1199] T 3b10ecb0e6064d42a2521c4c4e91c028 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/3b10ecb0e6064d42a2521c4c4e91c028
I20260925 13:56:11.490031 19959 ts_tablet_manager.cc:1950] T 3b10ecb0e6064d42a2521c4c4e91c028 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:11.495963 19938 tablet_bootstrap.cc:492] T b0a293e0094a4d9e937af6454e1ea16c P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:11.501420 19939 tablet_bootstrap.cc:492] T f51c5ad86ce64891a6393c1dc8d414e5 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:11.501883 19939 ts_tablet_manager.cc:1403] T f51c5ad86ce64891a6393c1dc8d414e5 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.253s user 0.019s sys 0.003s
I20260925 13:56:11.503054 19670 tablet_service.cc:1558] Processing DeleteTablet for tablet 8bd480f8e7a3487eb6b74a7ac7544b0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:11 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:11.504580 19939 raft_consensus.cc:359] T f51c5ad86ce64891a6393c1dc8d414e5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:11.505074 19671 tablet_service.cc:1511] Processing CreateTablet for tablet d9455b98734147fb9e2d47c69513424a (DEFAULT_TABLE table=default.Table_c1c831f6c04f4135801dcafbaf7a0843 [id=018679db079c42d2b70631ecb8f241a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:11.505379 19939 raft_consensus.cc:385] T f51c5ad86ce64891a6393c1dc8d414e5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:11.505702 19939 raft_consensus.cc:740] T f51c5ad86ce64891a6393c1dc8d414e5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:11.506467 19939 consensus_queue.cc:260] T f51c5ad86ce64891a6393c1dc8d414e5 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:11.508939 19554 raft_consensus.cc:1275] T 491b8b5c0f51408b882adee10ce80728 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Refusing update from remote peer f790493ba7ad403993b678f2df2b55be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:56:11.484550 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet b0dffbfd867040fd99e0fdc77e732f74 (table default.Table_bb731091526c4e0d95e68171b5b70a60 [id=80deba2b504744738a9fca818e546a7c]) successfully deleted
W20260925 13:56:11.512055 19264 catalog_manager.cc:5030] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): delete failed for tablet 5a7907a50912430bbeeac44d2ebd47de because the tablet was not found. No further retry: Not found: Tablet not found: 5a7907a50912430bbeeac44d2ebd47de
I20260925 13:56:11.516129 19955 ts_tablet_manager.cc:1916] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:11.518852 19976 consensus_queue.cc:1048] T 491b8b5c0f51408b882adee10ce80728 P f790493ba7ad403993b678f2df2b55be [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:11.521848 19671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9455b98734147fb9e2d47c69513424a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:11.523871 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 3b10ecb0e6064d42a2521c4c4e91c028 (table default.Table_f9ae699facd148ca8bae38d481d136df [id=7a20968885384e23a7a65f63337a67d3]) successfully deleted
I20260925 13:56:11.536597 19676 tablet_service.cc:1558] Processing DeleteTablet for tablet b0dffbfd867040fd99e0fdc77e732f74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:10 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:11.541774 19959 tablet_replica.cc:333] T 8bd480f8e7a3487eb6b74a7ac7544b0d P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:11.544763 19275 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_31807711126b487380c586fcf1939670" } new_table_name: "default.Table_52f57b4f58b44f28989ffd9a4afe2eac" modify_external_catalogs: true
I20260925 13:56:11.551455 19942 ts_tablet_manager.cc:1434] T 6b52ed579b19479686247d3ecba9489a P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.223s user 0.009s sys 0.010s
I20260925 13:56:11.552440 19273 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_8ac29bbd5011471a9b60f334fc78a3d3" } modify_external_catalogs: true
I20260925 13:56:11.552959 19273 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_8ac29bbd5011471a9b60f334fc78a3d3" } modify_external_catalogs: true
I20260925 13:56:11.560096 19942 tablet_bootstrap.cc:492] T 3b10ecb0e6064d42a2521c4c4e91c028 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:11.561323 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f51c5ad86ce64891a6393c1dc8d414e5" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:11.562004 19696 raft_consensus.cc:1275] T 491b8b5c0f51408b882adee10ce80728 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Refusing update from remote peer f790493ba7ad403993b678f2df2b55be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:56:11.562031 19695 raft_consensus.cc:2468] T f51c5ad86ce64891a6393c1dc8d414e5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 0.
I20260925 13:56:11.563818 19976 consensus_queue.cc:1048] T 491b8b5c0f51408b882adee10ce80728 P f790493ba7ad403993b678f2df2b55be [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:11.564261 19773 leader_election.cc:304] T f51c5ad86ce64891a6393c1dc8d414e5 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters: 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:11.547646 19959 raft_consensus.cc:2243] T 8bd480f8e7a3487eb6b74a7ac7544b0d P f790493ba7ad403993b678f2df2b55be [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:11.565326 19978 raft_consensus.cc:2804] T f51c5ad86ce64891a6393c1dc8d414e5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:11.565704 19978 raft_consensus.cc:493] T f51c5ad86ce64891a6393c1dc8d414e5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:11.566021 19978 raft_consensus.cc:3060] T f51c5ad86ce64891a6393c1dc8d414e5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:11.566322 19959 raft_consensus.cc:2272] T 8bd480f8e7a3487eb6b74a7ac7544b0d P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:11.574193 19978 raft_consensus.cc:515] T f51c5ad86ce64891a6393c1dc8d414e5 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:11.577028 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f51c5ad86ce64891a6393c1dc8d414e5" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:11.577605 19555 raft_consensus.cc:3060] T f51c5ad86ce64891a6393c1dc8d414e5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:11.579236 19942 tablet_bootstrap.cc:654] T 3b10ecb0e6064d42a2521c4c4e91c028 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:11.584721 19959 ts_tablet_manager.cc:1916] T 8bd480f8e7a3487eb6b74a7ac7544b0d P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:11.584781 19555 raft_consensus.cc:2468] T f51c5ad86ce64891a6393c1dc8d414e5 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 1.
I20260925 13:56:11.586194 19773 leader_election.cc:304] T f51c5ad86ce64891a6393c1dc8d414e5 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:11.587124 19976 raft_consensus.cc:2804] T f51c5ad86ce64891a6393c1dc8d414e5 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:11.588768 19676 tablet_service.cc:1558] Processing DeleteTablet for tablet b8200d0044434788adc1f7ec29aefa59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:09 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:11.598132 19938 tablet_bootstrap.cc:654] T b0a293e0094a4d9e937af6454e1ea16c P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:11.600549 19978 leader_election.cc:290] T f51c5ad86ce64891a6393c1dc8d414e5 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:11.601804 19978 raft_consensus.cc:493] T ff3d09046b854fdaab2f95feff589ac1 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:11.602784 19978 raft_consensus.cc:515] T ff3d09046b854fdaab2f95feff589ac1 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:11.604544 19976 raft_consensus.cc:697] T f51c5ad86ce64891a6393c1dc8d414e5 P f790493ba7ad403993b678f2df2b55be [term 1 LEADER]: Becoming Leader. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Running, Role: LEADER
I20260925 13:56:11.605473 19976 consensus_queue.cc:237] T f51c5ad86ce64891a6393c1dc8d414e5 P f790493ba7ad403993b678f2df2b55be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:11.606225 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3d09046b854fdaab2f95feff589ac1" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
W20260925 13:56:11.607748 19773 leader_election.cc:343] T ff3d09046b854fdaab2f95feff589ac1 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:11.612526 19939 ts_tablet_manager.cc:1434] T f51c5ad86ce64891a6393c1dc8d414e5 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.110s user 0.008s sys 0.000s
I20260925 13:56:11.553308 20024 raft_consensus.cc:515] T b0dffbfd867040fd99e0fdc77e732f74 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:11.613412 19939 tablet_bootstrap.cc:492] T 6b52ed579b19479686247d3ecba9489a P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:11.636472 19815 tablet_service.cc:1511] Processing CreateTablet for tablet 1513c0048bd244028d3dfa8d118bc14e (DEFAULT_TABLE table=default.Table_fa74b192993b433681028dd598b6a56b [id=d81cf07e5f9a49d4bbf5aceda3da5a25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:11.636773 19278 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet f51c5ad86ce64891a6393c1dc8d414e5 (table default.Table_824449a97a864e3dbbf06e2ad678edc5 [id=af708d3ba4c54ba19c24eca862bbddc1]). Expected version 1 got 0
I20260925 13:56:11.637776 19815 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1513c0048bd244028d3dfa8d118bc14e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:11.642611 19978 leader_election.cc:290] T ff3d09046b854fdaab2f95feff589ac1 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
W20260925 13:56:11.645707 19275 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 28ac2912b94a4928967efb7a731c42b9 consensus state has no leader
I20260925 13:56:11.651235 19535 tablet_service.cc:1511] Processing CreateTablet for tablet 1513c0048bd244028d3dfa8d118bc14e (DEFAULT_TABLE table=default.Table_fa74b192993b433681028dd598b6a56b [id=d81cf07e5f9a49d4bbf5aceda3da5a25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:11.652015 19275 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_edddbd4b770d44a3907e4d08fba4c0c7" } modify_external_catalogs: true
I20260925 13:56:11.652560 19535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1513c0048bd244028d3dfa8d118bc14e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:11.652410 19279 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet 491b8b5c0f51408b882adee10ce80728 (table default.Table_81fefa73a0904acb9b300de03fb48254 [id=dadfbd02a0a640949b1dc46c01db7bc8]). Expected version 2 got 0
I20260925 13:56:11.652640 19275 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_edddbd4b770d44a3907e4d08fba4c0c7" } modify_external_catalogs: true
I20260925 13:56:11.658735 19817 tablet_service.cc:1511] Processing CreateTablet for tablet c1f15b9de572479aae6c2e644da3392a (DEFAULT_TABLE table=default.Table_ab754389c42247ccbed64c8f7c539d90 [id=9cb2dc12a91e4de087414b492cd0dcfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:11.660663 19817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1f15b9de572479aae6c2e644da3392a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:11.662659 19818 tablet_service.cc:1511] Processing CreateTablet for tablet 00b500182bb04a72b80cfa9103e85fa9 (DEFAULT_TABLE table=default.Table_04d053f331f243ce8e98beb7c00d4159 [id=745c9e0d4f2c4ce58c3a201c34f1f1e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:11.663259 19939 tablet_bootstrap.cc:654] T 6b52ed579b19479686247d3ecba9489a P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:11.663888 19818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00b500182bb04a72b80cfa9103e85fa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:11.668129 19533 tablet_service.cc:1511] Processing CreateTablet for tablet c1f15b9de572479aae6c2e644da3392a (DEFAULT_TABLE table=default.Table_ab754389c42247ccbed64c8f7c539d90 [id=9cb2dc12a91e4de087414b492cd0dcfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:11.669334 19533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1f15b9de572479aae6c2e644da3392a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:11.672298 19534 tablet_service.cc:1511] Processing CreateTablet for tablet 00b500182bb04a72b80cfa9103e85fa9 (DEFAULT_TABLE table=default.Table_04d053f331f243ce8e98beb7c00d4159 [id=745c9e0d4f2c4ce58c3a201c34f1f1e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:11.673523 19534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00b500182bb04a72b80cfa9103e85fa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:11.675957 19532 tablet_service.cc:1511] Processing CreateTablet for tablet 18a6d1c4eb7948149e709a380da50d93 (DEFAULT_TABLE table=default.Table_abc00b1f4c4c455e8086a46c395f2656 [id=6c0ec9465714408da93ff6e47827eb48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:11.677345 19532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18a6d1c4eb7948149e709a380da50d93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:11.684590 19273 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet ff3d09046b854fdaab2f95feff589ac1
I20260925 13:56:11.688750 19820 tablet_service.cc:1558] Processing DeleteTablet for tablet ff3d09046b854fdaab2f95feff589ac1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:11 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:11.694906 19938 tablet_bootstrap.cc:492] T b0a293e0094a4d9e937af6454e1ea16c P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:11.695536 19938 ts_tablet_manager.cc:1403] T b0a293e0094a4d9e937af6454e1ea16c P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.200s user 0.014s sys 0.005s
I20260925 13:56:11.702322 19938 raft_consensus.cc:359] T b0a293e0094a4d9e937af6454e1ea16c P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:11.704870 19531 tablet_service.cc:1558] Processing DeleteTablet for tablet ff3d09046b854fdaab2f95feff589ac1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:11 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:11.707468 19273 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_9cdf64e560854f80a1c733b4d896f285" } new_table_name: "default.Table_9786cd5857464d5f9008e6e630c41899" modify_external_catalogs: true
I20260925 13:56:11.712292 19938 raft_consensus.cc:385] T b0a293e0094a4d9e937af6454e1ea16c P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:11.712642 19938 raft_consensus.cc:740] T b0a293e0094a4d9e937af6454e1ea16c P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:11.720120 19938 consensus_queue.cc:260] T b0a293e0094a4d9e937af6454e1ea16c P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:11.725757 19959 ts_tablet_manager.cc:1929] T 8bd480f8e7a3487eb6b74a7ac7544b0d P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:56:11.726151 19959 log.cc:1199] T 8bd480f8e7a3487eb6b74a7ac7544b0d P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/8bd480f8e7a3487eb6b74a7ac7544b0d
I20260925 13:56:11.726873 19959 ts_tablet_manager.cc:1950] T 8bd480f8e7a3487eb6b74a7ac7544b0d P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:11.731092 19938 ts_tablet_manager.cc:1434] T b0a293e0094a4d9e937af6454e1ea16c P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.031s user 0.004s sys 0.002s
I20260925 13:56:11.731966 19938 tablet_bootstrap.cc:492] T 28ac2912b94a4928967efb7a731c42b9 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:11.739497 19942 tablet_bootstrap.cc:492] T 3b10ecb0e6064d42a2521c4c4e91c028 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:11.742193 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 8bd480f8e7a3487eb6b74a7ac7544b0d (table default.Table_f7190a6e9e55451c8177b52fbaa2a61d [id=3fd25e8e73f946b8ba379c2876e1386e]) successfully deleted
I20260925 13:56:11.764658 20032 consensus_queue.cc:1048] T c0f05269ea6c43a98d2137eae7a9a401 P f790493ba7ad403993b678f2df2b55be [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:11.755285 20011 ts_tablet_manager.cc:1929] T 5a7907a50912430bbeeac44d2ebd47de P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:11.781061 20024 leader_election.cc:290] T b0dffbfd867040fd99e0fdc77e732f74 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:11.793613 20027 raft_consensus.cc:493] T b8200d0044434788adc1f7ec29aefa59 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:11.794112 19942 ts_tablet_manager.cc:1403] T 3b10ecb0e6064d42a2521c4c4e91c028 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.234s user 0.013s sys 0.010s
I20260925 13:56:11.794076 20027 raft_consensus.cc:515] T b8200d0044434788adc1f7ec29aefa59 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:11.794230 19676 tablet_service.cc:1511] Processing CreateTablet for tablet c1f15b9de572479aae6c2e644da3392a (DEFAULT_TABLE table=default.Table_ab754389c42247ccbed64c8f7c539d90 [id=9cb2dc12a91e4de087414b492cd0dcfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:11.796409 19959 tablet_replica.cc:333] T ff3d09046b854fdaab2f95feff589ac1 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:11.797333 19279 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_6a1a1e9c963f4428a5028a5c9d9384a9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:11.797622 19278 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_8290acbcb9d84a48aad169c54251bc19"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:11.799433 19279 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6a1a1e9c963f4428a5028a5c9d9384a9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 13:56:11.799787 19278 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8290acbcb9d84a48aad169c54251bc19 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:11.803638 19939 tablet_bootstrap.cc:492] T 6b52ed579b19479686247d3ecba9489a P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:11.804014 19939 ts_tablet_manager.cc:1403] T 6b52ed579b19479686247d3ecba9489a P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.191s user 0.024s sys 0.013s
I20260925 13:56:11.805387 19959 raft_consensus.cc:2243] T ff3d09046b854fdaab2f95feff589ac1 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:11.805867 19959 raft_consensus.cc:2272] T ff3d09046b854fdaab2f95feff589ac1 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:11.810145 19938 tablet_bootstrap.cc:654] T 28ac2912b94a4928967efb7a731c42b9 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:11.811029 19939 raft_consensus.cc:359] T 6b52ed579b19479686247d3ecba9489a P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:11.795521 19676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1f15b9de572479aae6c2e644da3392a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:11.811673 19939 raft_consensus.cc:385] T 6b52ed579b19479686247d3ecba9489a P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:11.817953 20027 leader_election.cc:290] T b8200d0044434788adc1f7ec29aefa59 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:11.818930 19959 ts_tablet_manager.cc:1916] T ff3d09046b854fdaab2f95feff589ac1 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:11.819410 19673 tablet_service.cc:1511] Processing CreateTablet for tablet 00b500182bb04a72b80cfa9103e85fa9 (DEFAULT_TABLE table=default.Table_04d053f331f243ce8e98beb7c00d4159 [id=745c9e0d4f2c4ce58c3a201c34f1f1e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:11.779856 20011 log.cc:1199] T 5a7907a50912430bbeeac44d2ebd47de P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/5a7907a50912430bbeeac44d2ebd47de
I20260925 13:56:11.812009 19939 raft_consensus.cc:740] T 6b52ed579b19479686247d3ecba9489a P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:11.823887 19939 consensus_queue.cc:260] T 6b52ed579b19479686247d3ecba9489a P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:11.824790 19629 leader_election.cc:343] T b0dffbfd867040fd99e0fdc77e732f74 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Not found: Tablet not found: b0dffbfd867040fd99e0fdc77e732f74
W20260925 13:56:11.825889 19629 leader_election.cc:343] T b8200d0044434788adc1f7ec29aefa59 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Not found: Tablet not found: b8200d0044434788adc1f7ec29aefa59
I20260925 13:56:11.829813 20011 ts_tablet_manager.cc:1950] T 5a7907a50912430bbeeac44d2ebd47de P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:11.833348 19696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f51c5ad86ce64891a6393c1dc8d414e5" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:11.833863 19696 raft_consensus.cc:3060] T f51c5ad86ce64891a6393c1dc8d414e5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:11.835130 19939 ts_tablet_manager.cc:1434] T 6b52ed579b19479686247d3ecba9489a P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.031s user 0.006s sys 0.000s
I20260925 13:56:11.839185 19939 tablet_bootstrap.cc:492] T 3b10ecb0e6064d42a2521c4c4e91c028 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:11.845603 19939 tablet_bootstrap.cc:654] T 3b10ecb0e6064d42a2521c4c4e91c028 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:11.846695 19554 raft_consensus.cc:1275] T c0f05269ea6c43a98d2137eae7a9a401 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Refusing update from remote peer f790493ba7ad403993b678f2df2b55be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:56:11.848495 20032 consensus_queue.cc:1048] T c0f05269ea6c43a98d2137eae7a9a401 P f790493ba7ad403993b678f2df2b55be [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 13:56:11.856724 19265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.254:40825, user_credentials={real_user=slave}} blocked reactor thread for 64283.9us
I20260925 13:56:11.857725 19694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3d09046b854fdaab2f95feff589ac1" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:11.862365 19942 raft_consensus.cc:359] T 3b10ecb0e6064d42a2521c4c4e91c028 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:11.862684 19773 leader_election.cc:343] T ff3d09046b854fdaab2f95feff589ac1 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:11.863121 19773 leader_election.cc:304] T ff3d09046b854fdaab2f95feff589ac1 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
W20260925 13:56:11.863777 19773 raft_consensus.cc:2686] T ff3d09046b854fdaab2f95feff589ac1 P f790493ba7ad403993b678f2df2b55be: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260925 13:56:11.876149 19938 tablet_bootstrap.cc:492] T 28ac2912b94a4928967efb7a731c42b9 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:11.876582 19938 ts_tablet_manager.cc:1403] T 28ac2912b94a4928967efb7a731c42b9 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.145s user 0.016s sys 0.002s
I20260925 13:56:11.885846 19959 ts_tablet_manager.cc:1929] T ff3d09046b854fdaab2f95feff589ac1 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:11.879065 19938 raft_consensus.cc:359] T 28ac2912b94a4928967efb7a731c42b9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:11.886281 19959 log.cc:1199] T ff3d09046b854fdaab2f95feff589ac1 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/ff3d09046b854fdaab2f95feff589ac1
I20260925 13:56:11.886451 19938 raft_consensus.cc:385] T 28ac2912b94a4928967efb7a731c42b9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:11.886811 19938 raft_consensus.cc:740] T 28ac2912b94a4928967efb7a731c42b9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:11.886991 19959 ts_tablet_manager.cc:1950] T ff3d09046b854fdaab2f95feff589ac1 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
W20260925 13:56:11.865234 19631 leader_election.cc:343] T b0dffbfd867040fd99e0fdc77e732f74 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: b0dffbfd867040fd99e0fdc77e732f74
I20260925 13:56:11.875438 19673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00b500182bb04a72b80cfa9103e85fa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:11.888270 19275 catalog_manager.cc:6128] Not sending DeleteTablet requests; no consensus state for tablet ea23ad7945d0405a8ba02340e56e50ae
I20260925 13:56:11.895509 19817 tablet_service.cc:1511] Processing CreateTablet for tablet 18a6d1c4eb7948149e709a380da50d93 (DEFAULT_TABLE table=default.Table_abc00b1f4c4c455e8086a46c395f2656 [id=6c0ec9465714408da93ff6e47827eb48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:11.896665 19817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18a6d1c4eb7948149e709a380da50d93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:11.897639 19275 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_3b49fb893abb48ac8646283360e8e681"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:11.899735 19275 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3b49fb893abb48ac8646283360e8e681 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:11.905236 19939 tablet_bootstrap.cc:492] T 3b10ecb0e6064d42a2521c4c4e91c028 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:11.905627 19939 ts_tablet_manager.cc:1403] T 3b10ecb0e6064d42a2521c4c4e91c028 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.067s user 0.006s sys 0.003s
I20260925 13:56:11.889514 19942 raft_consensus.cc:385] T 3b10ecb0e6064d42a2521c4c4e91c028 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:11.908457 19942 raft_consensus.cc:740] T 3b10ecb0e6064d42a2521c4c4e91c028 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:11.908962 19939 raft_consensus.cc:359] T 3b10ecb0e6064d42a2521c4c4e91c028 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:11.909566 19939 raft_consensus.cc:385] T 3b10ecb0e6064d42a2521c4c4e91c028 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:11.909809 19939 raft_consensus.cc:740] T 3b10ecb0e6064d42a2521c4c4e91c028 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:11.910406 19939 consensus_queue.cc:260] T 3b10ecb0e6064d42a2521c4c4e91c028 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:11.912349 20044 tablet.cc:1751] T 491b8b5c0f51408b882adee10ce80728 P f790493ba7ad403993b678f2df2b55be: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 13:56:11.912895 19939 ts_tablet_manager.cc:1434] T 3b10ecb0e6064d42a2521c4c4e91c028 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260925 13:56:11.913677 19939 tablet_bootstrap.cc:492] T ff3d09046b854fdaab2f95feff589ac1 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:11.865721 19671 tablet_service.cc:1558] Processing DeleteTablet for tablet ff3d09046b854fdaab2f95feff589ac1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:11 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:11.917565 19938 consensus_queue.cc:260] T 28ac2912b94a4928967efb7a731c42b9 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:11.917747 19276 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_f1d49a17b4c9418a9872e05be0d39488"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:11.919405 19276 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f1d49a17b4c9418a9872e05be0d39488 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:11.903735 20045 tablet.cc:1751] T 491b8b5c0f51408b882adee10ce80728 P 8f578869bc084a5d8a9a69f143954eb3: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 13:56:11.923626 19280 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_3c6823adc2894e7e8048a056415f6a73" } new_table_name: "default.Table_da4fc82971544f509fcd31edfea882da" modify_external_catalogs: true
I20260925 13:56:11.927168 19938 ts_tablet_manager.cc:1434] T 28ac2912b94a4928967efb7a731c42b9 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.050s user 0.005s sys 0.001s
I20260925 13:56:11.927814 19277 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_c1c831f6c04f4135801dcafbaf7a0843" } modify_external_catalogs: true
I20260925 13:56:11.928501 19939 tablet_bootstrap.cc:654] T ff3d09046b854fdaab2f95feff589ac1 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:11.928367 19277 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_c1c831f6c04f4135801dcafbaf7a0843" } modify_external_catalogs: true
I20260925 13:56:11.932577 19938 tablet_bootstrap.cc:492] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:11.946161 19938 tablet_bootstrap.cc:654] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:11.917182 20011 tablet_replica.cc:333] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:11.916185 19631 leader_election.cc:304] T b0dffbfd867040fd99e0fdc77e732f74 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
W20260925 13:56:11.950563 19631 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 89086.9us
W20260925 13:56:11.951292 19631 leader_election.cc:343] T b8200d0044434788adc1f7ec29aefa59 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: b8200d0044434788adc1f7ec29aefa59
I20260925 13:56:11.951715 19631 leader_election.cc:304] T b8200d0044434788adc1f7ec29aefa59 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:11.950691 20027 raft_consensus.cc:2749] T b0dffbfd867040fd99e0fdc77e732f74 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260925 13:56:11.952364 20044 tablet.cc:1751] T c0f05269ea6c43a98d2137eae7a9a401 P f790493ba7ad403993b678f2df2b55be: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:11.955807 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet ff3d09046b854fdaab2f95feff589ac1 (table default.Table_8ac29bbd5011471a9b60f334fc78a3d3 [id=689d277ceae34fc0bb1f2ecd43295722]) successfully deleted
I20260925 13:56:11.950551 19675 tablet_service.cc:1511] Processing CreateTablet for tablet 1513c0048bd244028d3dfa8d118bc14e (DEFAULT_TABLE table=default.Table_fa74b192993b433681028dd598b6a56b [id=d81cf07e5f9a49d4bbf5aceda3da5a25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:11.957280 20024 raft_consensus.cc:2749] T b8200d0044434788adc1f7ec29aefa59 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:11.958432 19675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1513c0048bd244028d3dfa8d118bc14e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:11.959718 20011 raft_consensus.cc:2243] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:11.960310 20011 raft_consensus.cc:2272] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:11.960988 19938 tablet_bootstrap.cc:492] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:11.961342 19938 ts_tablet_manager.cc:1403] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.029s user 0.007s sys 0.000s
I20260925 13:56:11.966264 19696 raft_consensus.cc:2468] T f51c5ad86ce64891a6393c1dc8d414e5 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 1.
I20260925 13:56:11.966632 19938 raft_consensus.cc:359] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:11.967229 19938 raft_consensus.cc:385] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:11.967510 19938 raft_consensus.cc:740] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:11.914942 20035 tablet.cc:1751] T c0f05269ea6c43a98d2137eae7a9a401 P 8f578869bc084a5d8a9a69f143954eb3: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:11.980459 19938 consensus_queue.cc:260] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:11.980935 20011 ts_tablet_manager.cc:1916] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:11.986704 19938 ts_tablet_manager.cc:1434] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.025s user 0.007s sys 0.000s
I20260925 13:56:11.988190 20032 raft_consensus.cc:493] T 293b017a83cc464a9f023a0923310796 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:11.988597 20032 raft_consensus.cc:515] T 293b017a83cc464a9f023a0923310796 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:11.990902 20032 leader_election.cc:290] T 293b017a83cc464a9f023a0923310796 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:11.992951 19938 tablet_bootstrap.cc:492] T f4429db544534f758883f2fae184ea6d P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:11.992765 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293b017a83cc464a9f023a0923310796" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
W20260925 13:56:11.994095 19773 leader_election.cc:343] T 293b017a83cc464a9f023a0923310796 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:11.994459 19696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293b017a83cc464a9f023a0923310796" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:11.995792 19773 leader_election.cc:343] T 293b017a83cc464a9f023a0923310796 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:11.996183 19773 leader_election.cc:304] T 293b017a83cc464a9f023a0923310796 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:11.997555 20032 raft_consensus.cc:2749] T 293b017a83cc464a9f023a0923310796 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:12.004833 19939 tablet_bootstrap.cc:492] T ff3d09046b854fdaab2f95feff589ac1 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:12.005220 19939 ts_tablet_manager.cc:1403] T ff3d09046b854fdaab2f95feff589ac1 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.092s user 0.008s sys 0.017s
W20260925 13:56:12.005949 19969 consensus_meta.cc:287] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f: Time spent flushing consensus metadata: real 0.582s user 0.008s sys 0.007s
I20260925 13:56:12.007962 19938 tablet_bootstrap.cc:654] T f4429db544534f758883f2fae184ea6d P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:12.007553 19939 raft_consensus.cc:359] T ff3d09046b854fdaab2f95feff589ac1 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:12.008376 19939 raft_consensus.cc:385] T ff3d09046b854fdaab2f95feff589ac1 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:12.008641 19939 raft_consensus.cc:740] T ff3d09046b854fdaab2f95feff589ac1 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:12.009292 19939 consensus_queue.cc:260] T ff3d09046b854fdaab2f95feff589ac1 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:12.011480 19939 ts_tablet_manager.cc:1434] T ff3d09046b854fdaab2f95feff589ac1 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.006s user 0.007s sys 0.001s
I20260925 13:56:12.014379 19939 tablet_bootstrap.cc:492] T 293b017a83cc464a9f023a0923310796 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:12.020792 19938 tablet_bootstrap.cc:492] T f4429db544534f758883f2fae184ea6d P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:12.021229 19938 ts_tablet_manager.cc:1403] T f4429db544534f758883f2fae184ea6d P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.028s user 0.010s sys 0.000s
I20260925 13:56:12.023299 19938 raft_consensus.cc:359] T f4429db544534f758883f2fae184ea6d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:12.023912 19938 raft_consensus.cc:385] T f4429db544534f758883f2fae184ea6d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:12.024220 19938 raft_consensus.cc:740] T f4429db544534f758883f2fae184ea6d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
W20260925 13:56:12.024149 19273 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 51a4c21e06ca4b5b971cf8f04ffa52a9 consensus state has no leader
I20260925 13:56:12.029589 19938 consensus_queue.cc:260] T f4429db544534f758883f2fae184ea6d P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:12.031972 19939 tablet_bootstrap.cc:654] T 293b017a83cc464a9f023a0923310796 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:12.032936 19938 ts_tablet_manager.cc:1434] T f4429db544534f758883f2fae184ea6d P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20260925 13:56:12.033756 19938 tablet_bootstrap.cc:492] T d9455b98734147fb9e2d47c69513424a P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:12.042235 19938 tablet_bootstrap.cc:654] T d9455b98734147fb9e2d47c69513424a P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:12.044538 19939 tablet_bootstrap.cc:492] T 293b017a83cc464a9f023a0923310796 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:12.044924 19939 ts_tablet_manager.cc:1403] T 293b017a83cc464a9f023a0923310796 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.031s user 0.011s sys 0.001s
I20260925 13:56:12.047192 19939 raft_consensus.cc:359] T 293b017a83cc464a9f023a0923310796 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:12.047945 19939 raft_consensus.cc:385] T 293b017a83cc464a9f023a0923310796 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:12.048215 19939 raft_consensus.cc:740] T 293b017a83cc464a9f023a0923310796 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:12.048810 19939 consensus_queue.cc:260] T 293b017a83cc464a9f023a0923310796 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:12.055058 19272 catalog_manager.cc:5838] T f51c5ad86ce64891a6393c1dc8d414e5 P f790493ba7ad403993b678f2df2b55be reported cstate change: term changed from 0 to 1, leader changed from <none> to f790493ba7ad403993b678f2df2b55be (127.18.182.195). New cstate: current_term: 1 leader_uuid: "f790493ba7ad403993b678f2df2b55be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: HEALTHY } } }
I20260925 13:56:12.057281 19985 raft_consensus.cc:493] T ff3d09046b854fdaab2f95feff589ac1 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:12.057752 19985 raft_consensus.cc:515] T ff3d09046b854fdaab2f95feff589ac1 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:11.942867 19942 consensus_queue.cc:260] T 3b10ecb0e6064d42a2521c4c4e91c028 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:12.067090 19938 tablet_bootstrap.cc:492] T d9455b98734147fb9e2d47c69513424a P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:12.067564 19938 ts_tablet_manager.cc:1403] T d9455b98734147fb9e2d47c69513424a P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.034s user 0.012s sys 0.000s
I20260925 13:56:12.070420 19985 leader_election.cc:290] T ff3d09046b854fdaab2f95feff589ac1 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:12.070329 19938 raft_consensus.cc:359] T d9455b98734147fb9e2d47c69513424a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:12.071053 19938 raft_consensus.cc:385] T d9455b98734147fb9e2d47c69513424a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:12.071363 19938 raft_consensus.cc:740] T d9455b98734147fb9e2d47c69513424a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:12.072193 19938 consensus_queue.cc:260] T d9455b98734147fb9e2d47c69513424a P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:12.074651 19938 ts_tablet_manager.cc:1434] T d9455b98734147fb9e2d47c69513424a P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260925 13:56:12.075495 19938 tablet_bootstrap.cc:492] T 1513c0048bd244028d3dfa8d118bc14e P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:12.080497 19938 tablet_bootstrap.cc:654] T 1513c0048bd244028d3dfa8d118bc14e P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:12.081699 19939 ts_tablet_manager.cc:1434] T 293b017a83cc464a9f023a0923310796 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.037s user 0.006s sys 0.000s
I20260925 13:56:12.082602 19939 tablet_bootstrap.cc:492] T 28ac2912b94a4928967efb7a731c42b9 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:12.057855 19969 raft_consensus.cc:515] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:12.085263 19839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0cdc99cde2f4dabb819e3e171c5d8d9" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:12.085839 19839 raft_consensus.cc:3060] T f0cdc99cde2f4dabb819e3e171c5d8d9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:12.104537 19942 ts_tablet_manager.cc:1434] T 3b10ecb0e6064d42a2521c4c4e91c028 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.310s user 0.007s sys 0.000s
I20260925 13:56:12.106092 19942 tablet_bootstrap.cc:492] T 293b017a83cc464a9f023a0923310796 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:12.106158 19939 tablet_bootstrap.cc:654] T 28ac2912b94a4928967efb7a731c42b9 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:12.104831 19696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff3d09046b854fdaab2f95feff589ac1" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:12.109978 19488 leader_election.cc:343] T ff3d09046b854fdaab2f95feff589ac1 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:12.121759 19490 leader_election.cc:343] T ff3d09046b854fdaab2f95feff589ac1 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: ff3d09046b854fdaab2f95feff589ac1
I20260925 13:56:12.122365 19490 leader_election.cc:304] T ff3d09046b854fdaab2f95feff589ac1 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3; no voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:12.123602 19985 raft_consensus.cc:2749] T ff3d09046b854fdaab2f95feff589ac1 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:12.124656 19272 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet f51c5ad86ce64891a6393c1dc8d414e5 (table default.Table_824449a97a864e3dbbf06e2ad678edc5 [id=af708d3ba4c54ba19c24eca862bbddc1]). Expected version 1 got 0
I20260925 13:56:12.127547 19839 raft_consensus.cc:2468] T f0cdc99cde2f4dabb819e3e171c5d8d9 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
I20260925 13:56:12.129055 19939 tablet_bootstrap.cc:492] T 28ac2912b94a4928967efb7a731c42b9 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:12.129467 19939 ts_tablet_manager.cc:1403] T 28ac2912b94a4928967efb7a731c42b9 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.047s user 0.007s sys 0.004s
I20260925 13:56:12.130219 20051 tablet.cc:1751] T c0f05269ea6c43a98d2137eae7a9a401 P 7c1b52895a14482fb54a6758f5054f8f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:12.132004 19939 raft_consensus.cc:359] T 28ac2912b94a4928967efb7a731c42b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:12.132655 19939 raft_consensus.cc:385] T 28ac2912b94a4928967efb7a731c42b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:12.132920 19939 raft_consensus.cc:740] T 28ac2912b94a4928967efb7a731c42b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:12.134255 19939 consensus_queue.cc:260] T 28ac2912b94a4928967efb7a731c42b9 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
W20260925 13:56:12.134589 19265 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 51a4c21e06ca4b5b971cf8f04ffa52a9 (table default.Table_9786cd5857464d5f9008e6e630c41899 [id=fe0f78d1c6e44f589aede852f77f88fc], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 51a4c21e06ca4b5b971cf8f04ffa52a9 consensus state has no leader
I20260925 13:56:12.139050 19939 ts_tablet_manager.cc:1434] T 28ac2912b94a4928967efb7a731c42b9 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.009s user 0.002s sys 0.004s
I20260925 13:56:12.152096 19939 tablet_bootstrap.cc:492] T b0a293e0094a4d9e937af6454e1ea16c P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:12.157485 19939 tablet_bootstrap.cc:654] T b0a293e0094a4d9e937af6454e1ea16c P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:12.169829 19969 leader_election.cc:290] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:12.170451 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0cdc99cde2f4dabb819e3e171c5d8d9" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:12.171043 19555 raft_consensus.cc:3060] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:12.178283 19555 raft_consensus.cc:2468] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
I20260925 13:56:12.179371 19629 leader_election.cc:304] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3; no voters:
I20260925 13:56:12.180886 19939 tablet_bootstrap.cc:492] T b0a293e0094a4d9e937af6454e1ea16c P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:12.181258 19939 ts_tablet_manager.cc:1403] T b0a293e0094a4d9e937af6454e1ea16c P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.042s user 0.005s sys 0.004s
I20260925 13:56:12.183450 19939 raft_consensus.cc:359] T b0a293e0094a4d9e937af6454e1ea16c P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:12.184055 19939 raft_consensus.cc:385] T b0a293e0094a4d9e937af6454e1ea16c P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:12.184307 19939 raft_consensus.cc:740] T b0a293e0094a4d9e937af6454e1ea16c P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:12.184892 19939 consensus_queue.cc:260] T b0a293e0094a4d9e937af6454e1ea16c P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:12.189628 19939 ts_tablet_manager.cc:1434] T b0a293e0094a4d9e937af6454e1ea16c P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.008s user 0.006s sys 0.001s
I20260925 13:56:12.194837 19939 tablet_bootstrap.cc:492] T f4429db544534f758883f2fae184ea6d P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:12.196132 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 5a7907a50912430bbeeac44d2ebd47de (table default.Table_e9dc3916a32d4c6e8d9517c521a20220 [id=00c39e12d41b4d27b6cd28a850c76fe4]) successfully deleted
I20260925 13:56:12.209707 19939 tablet_bootstrap.cc:654] T f4429db544534f758883f2fae184ea6d P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:12.210345 20024 raft_consensus.cc:2804] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:12.210927 20024 raft_consensus.cc:697] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Becoming Leader. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Running, Role: LEADER
I20260925 13:56:12.211997 20024 consensus_queue.cc:237] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:12.223788 19985 raft_consensus.cc:493] T 293b017a83cc464a9f023a0923310796 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:12.224205 19985 raft_consensus.cc:515] T 293b017a83cc464a9f023a0923310796 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:12.225975 19985 leader_election.cc:290] T 293b017a83cc464a9f023a0923310796 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:12.227830 19839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293b017a83cc464a9f023a0923310796" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:12.228386 19839 raft_consensus.cc:2468] T 293b017a83cc464a9f023a0923310796 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:12.229483 19490 leader_election.cc:304] T 293b017a83cc464a9f023a0923310796 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:12.230134 19985 raft_consensus.cc:2804] T 293b017a83cc464a9f023a0923310796 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:12.230438 19985 raft_consensus.cc:493] T 293b017a83cc464a9f023a0923310796 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:12.230720 19985 raft_consensus.cc:3060] T 293b017a83cc464a9f023a0923310796 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
W20260925 13:56:12.230808 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 3b10ecb0e6064d42a2521c4c4e91c028 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3b10ecb0e6064d42a2521c4c4e91c028 already in progress: creating tablet
I20260925 13:56:12.236958 19939 tablet_bootstrap.cc:492] T f4429db544534f758883f2fae184ea6d P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:12.237321 19939 ts_tablet_manager.cc:1403] T f4429db544534f758883f2fae184ea6d P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.047s user 0.009s sys 0.000s
I20260925 13:56:12.239966 19939 raft_consensus.cc:359] T f4429db544534f758883f2fae184ea6d P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:12.240693 19939 raft_consensus.cc:385] T f4429db544534f758883f2fae184ea6d P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:12.240962 19939 raft_consensus.cc:740] T f4429db544534f758883f2fae184ea6d P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:12.245208 19985 raft_consensus.cc:515] T 293b017a83cc464a9f023a0923310796 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:12.247907 19939 consensus_queue.cc:260] T f4429db544534f758883f2fae184ea6d P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:12.251327 20068 raft_consensus.cc:493] T 3b10ecb0e6064d42a2521c4c4e91c028 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:12.253209 20068 raft_consensus.cc:515] T 3b10ecb0e6064d42a2521c4c4e91c028 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:12.257946 20068 leader_election.cc:290] T 3b10ecb0e6064d42a2521c4c4e91c028 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
W20260925 13:56:12.259815 19490 leader_election.cc:343] T 3b10ecb0e6064d42a2521c4c4e91c028 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 3b10ecb0e6064d42a2521c4c4e91c028
I20260925 13:56:12.266062 19839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293b017a83cc464a9f023a0923310796" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:12.266633 19839 raft_consensus.cc:3060] T 293b017a83cc464a9f023a0923310796 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:12.273543 19839 raft_consensus.cc:2468] T 293b017a83cc464a9f023a0923310796 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:12.274690 19490 leader_election.cc:304] T 293b017a83cc464a9f023a0923310796 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:12.275496 20068 raft_consensus.cc:2804] T 293b017a83cc464a9f023a0923310796 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:12.275611 19985 leader_election.cc:290] T 293b017a83cc464a9f023a0923310796 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:12.275981 20068 raft_consensus.cc:697] T 293b017a83cc464a9f023a0923310796 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Running, Role: LEADER
I20260925 13:56:12.276868 20068 consensus_queue.cc:237] T 293b017a83cc464a9f023a0923310796 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:12.282644 19939 ts_tablet_manager.cc:1434] T f4429db544534f758883f2fae184ea6d P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.045s user 0.011s sys 0.000s
W20260925 13:56:12.283501 19266 consensus_peers.cc:597] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 -> Peer fc3953a2ba984a658630c423ec9b9728 (127.18.182.252:36181): Couldn't send request to peer fc3953a2ba984a658630c423ec9b9728. Status: Network error: Client connection negotiation failed: client connection to 127.18.182.252:36181: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260925 13:56:12.285866 19939 tablet_bootstrap.cc:492] T d9455b98734147fb9e2d47c69513424a P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:12.291633 19939 tablet_bootstrap.cc:654] T d9455b98734147fb9e2d47c69513424a P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
W20260925 13:56:12.297289 19280 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 293b017a83cc464a9f023a0923310796 consensus state has no leader
I20260925 13:56:12.308670 19694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b10ecb0e6064d42a2521c4c4e91c028" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:12.309478 19694 raft_consensus.cc:2468] T 3b10ecb0e6064d42a2521c4c4e91c028 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:12.320173 19488 leader_election.cc:304] T 3b10ecb0e6064d42a2521c4c4e91c028 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3; no voters: f790493ba7ad403993b678f2df2b55be
I20260925 13:56:12.321214 20068 raft_consensus.cc:2804] T 3b10ecb0e6064d42a2521c4c4e91c028 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:12.321537 20068 raft_consensus.cc:493] T 3b10ecb0e6064d42a2521c4c4e91c028 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:12.321828 20068 raft_consensus.cc:3060] T 3b10ecb0e6064d42a2521c4c4e91c028 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:12.322286 19693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293b017a83cc464a9f023a0923310796" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
W20260925 13:56:12.324330 19488 leader_election.cc:343] T 293b017a83cc464a9f023a0923310796 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:12.325350 19696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293b017a83cc464a9f023a0923310796" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:12.326578 19488 leader_election.cc:343] T 293b017a83cc464a9f023a0923310796 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:12.322863 19969 raft_consensus.cc:493] T 6b52ed579b19479686247d3ecba9489a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:12.327811 19969 raft_consensus.cc:515] T 6b52ed579b19479686247d3ecba9489a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:12.328676 20068 raft_consensus.cc:515] T 3b10ecb0e6064d42a2521c4c4e91c028 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:12.330464 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b52ed579b19479686247d3ecba9489a" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:12.330992 19555 raft_consensus.cc:2468] T 6b52ed579b19479686247d3ecba9489a P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:12.331342 19839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b52ed579b19479686247d3ecba9489a" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:12.332118 19839 raft_consensus.cc:2468] T 6b52ed579b19479686247d3ecba9489a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:12.331045 20068 leader_election.cc:290] T 3b10ecb0e6064d42a2521c4c4e91c028 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
W20260925 13:56:12.334229 19490 leader_election.cc:343] T 3b10ecb0e6064d42a2521c4c4e91c028 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 3b10ecb0e6064d42a2521c4c4e91c028
I20260925 13:56:12.334437 19629 leader_election.cc:304] T 6b52ed579b19479686247d3ecba9489a P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3; no voters:
I20260925 13:56:12.336525 20024 raft_consensus.cc:2804] T 6b52ed579b19479686247d3ecba9489a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:12.336966 20024 raft_consensus.cc:493] T 6b52ed579b19479686247d3ecba9489a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:12.337352 20024 raft_consensus.cc:3060] T 6b52ed579b19479686247d3ecba9489a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:12.343086 19277 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet d9455b98734147fb9e2d47c69513424a
I20260925 13:56:12.348171 19696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b10ecb0e6064d42a2521c4c4e91c028" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:12.349095 19696 raft_consensus.cc:3060] T 3b10ecb0e6064d42a2521c4c4e91c028 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:12.353864 19939 tablet_bootstrap.cc:492] T d9455b98734147fb9e2d47c69513424a P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:12.354274 19939 ts_tablet_manager.cc:1403] T d9455b98734147fb9e2d47c69513424a P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.071s user 0.003s sys 0.020s
I20260925 13:56:12.356498 19939 raft_consensus.cc:359] T d9455b98734147fb9e2d47c69513424a P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:12.357081 19939 raft_consensus.cc:385] T d9455b98734147fb9e2d47c69513424a P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:12.357342 19939 raft_consensus.cc:740] T d9455b98734147fb9e2d47c69513424a P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:12.358049 19939 consensus_queue.cc:260] T d9455b98734147fb9e2d47c69513424a P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:12.362056 19277 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_d2952c2050c5481ab85394e93890ce9b" } modify_external_catalogs: true
I20260925 13:56:12.362712 19277 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_d2952c2050c5481ab85394e93890ce9b" } modify_external_catalogs: true
I20260925 13:56:12.363634 19532 tablet_service.cc:1558] Processing DeleteTablet for tablet d9455b98734147fb9e2d47c69513424a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:11 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:12.363698 19939 ts_tablet_manager.cc:1434] T d9455b98734147fb9e2d47c69513424a P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.009s user 0.003s sys 0.004s
I20260925 13:56:12.366012 20068 raft_consensus.cc:493] T b0a293e0094a4d9e937af6454e1ea16c P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:12.366518 20068 raft_consensus.cc:515] T b0a293e0094a4d9e937af6454e1ea16c P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:12.369369 20068 leader_election.cc:290] T b0a293e0094a4d9e937af6454e1ea16c P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:12.369997 19693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0a293e0094a4d9e937af6454e1ea16c" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:12.371449 19488 leader_election.cc:343] T b0a293e0094a4d9e937af6454e1ea16c P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:12.371919 19939 tablet_bootstrap.cc:492] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:12.372256 19840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0a293e0094a4d9e937af6454e1ea16c" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:12.372808 19840 raft_consensus.cc:2468] T b0a293e0094a4d9e937af6454e1ea16c P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:12.373284 19938 tablet_bootstrap.cc:492] T 1513c0048bd244028d3dfa8d118bc14e P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:12.373651 19938 ts_tablet_manager.cc:1403] T 1513c0048bd244028d3dfa8d118bc14e P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.298s user 0.067s sys 0.094s
I20260925 13:56:12.374022 19490 leader_election.cc:304] T b0a293e0094a4d9e937af6454e1ea16c P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:12.374836 20068 raft_consensus.cc:2804] T b0a293e0094a4d9e937af6454e1ea16c P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:12.375173 20068 raft_consensus.cc:493] T b0a293e0094a4d9e937af6454e1ea16c P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:12.375326 19817 tablet_service.cc:1558] Processing DeleteTablet for tablet d9455b98734147fb9e2d47c69513424a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:11 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:12.375491 20068 raft_consensus.cc:3060] T b0a293e0094a4d9e937af6454e1ea16c P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:12.376046 19959 tablet_replica.cc:333] T d9455b98734147fb9e2d47c69513424a P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:12.376747 19959 raft_consensus.cc:2243] T d9455b98734147fb9e2d47c69513424a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:12.377270 19959 raft_consensus.cc:2272] T d9455b98734147fb9e2d47c69513424a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:12.379381 19959 ts_tablet_manager.cc:1916] T d9455b98734147fb9e2d47c69513424a P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:12.381842 20068 raft_consensus.cc:515] T b0a293e0094a4d9e937af6454e1ea16c P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:12.382851 19939 tablet_bootstrap.cc:654] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:12.384508 19840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0a293e0094a4d9e937af6454e1ea16c" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:12.385025 19840 raft_consensus.cc:3060] T b0a293e0094a4d9e937af6454e1ea16c P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:12.387449 20068 leader_election.cc:290] T b0a293e0094a4d9e937af6454e1ea16c P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:12.395735 19840 raft_consensus.cc:2468] T b0a293e0094a4d9e937af6454e1ea16c P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:12.398500 19693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0a293e0094a4d9e937af6454e1ea16c" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
W20260925 13:56:12.401696 20055 log.cc:927] Time spent T 491b8b5c0f51408b882adee10ce80728 P 7c1b52895a14482fb54a6758f5054f8f: Append to log took a long time: real 0.078s user 0.000s sys 0.000s
I20260925 13:56:12.401788 19938 raft_consensus.cc:359] T 1513c0048bd244028d3dfa8d118bc14e P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:12.402398 19938 raft_consensus.cc:385] T 1513c0048bd244028d3dfa8d118bc14e P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:12.402638 19938 raft_consensus.cc:740] T 1513c0048bd244028d3dfa8d118bc14e P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:12.403055 19676 tablet_service.cc:1511] Processing CreateTablet for tablet 18a6d1c4eb7948149e709a380da50d93 (DEFAULT_TABLE table=default.Table_abc00b1f4c4c455e8086a46c395f2656 [id=6c0ec9465714408da93ff6e47827eb48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:12.404707 19676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18a6d1c4eb7948149e709a380da50d93. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:12.403985 19488 leader_election.cc:343] T b0a293e0094a4d9e937af6454e1ea16c P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:12.408577 20066 tablet.cc:1751] T 491b8b5c0f51408b882adee10ce80728 P 7c1b52895a14482fb54a6758f5054f8f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 13:56:12.413400 19274 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet 491b8b5c0f51408b882adee10ce80728 (table default.Table_81fefa73a0904acb9b300de03fb48254 [id=dadfbd02a0a640949b1dc46c01db7bc8]). Expected version 2 got 0
I20260925 13:56:12.414297 19274 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet f51c5ad86ce64891a6393c1dc8d414e5 (table default.Table_824449a97a864e3dbbf06e2ad678edc5 [id=af708d3ba4c54ba19c24eca862bbddc1]). Expected version 1 got 0
I20260925 13:56:12.417738 19278 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 491b8b5c0f51408b882adee10ce80728 (table default.Table_81fefa73a0904acb9b300de03fb48254 [id=dadfbd02a0a640949b1dc46c01db7bc8]). Expected version 2 got 0
I20260925 13:56:12.403236 19938 consensus_queue.cc:260] T 1513c0048bd244028d3dfa8d118bc14e P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:12.430413 19938 ts_tablet_manager.cc:1434] T 1513c0048bd244028d3dfa8d118bc14e P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.057s user 0.005s sys 0.000s
I20260925 13:56:12.431262 19938 tablet_bootstrap.cc:492] T c1f15b9de572479aae6c2e644da3392a P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:12.435458 19959 ts_tablet_manager.cc:1929] T d9455b98734147fb9e2d47c69513424a P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:12.437830 19959 log.cc:1199] T d9455b98734147fb9e2d47c69513424a P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/d9455b98734147fb9e2d47c69513424a
I20260925 13:56:12.438442 19959 ts_tablet_manager.cc:1950] T d9455b98734147fb9e2d47c69513424a P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:12.454890 19278 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_38f7207982be43189299b739031aa942"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:12.456642 19278 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_38f7207982be43189299b739031aa942 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:12.458668 19938 tablet_bootstrap.cc:654] T c1f15b9de572479aae6c2e644da3392a P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:12.401257 19673 tablet_service.cc:1558] Processing DeleteTablet for tablet d9455b98734147fb9e2d47c69513424a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:11 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:12.401432 19969 leader_election.cc:290] T 6b52ed579b19479686247d3ecba9489a P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:12.466495 19490 leader_election.cc:304] T b0a293e0094a4d9e937af6454e1ea16c P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:12.473469 20068 raft_consensus.cc:2804] T b0a293e0094a4d9e937af6454e1ea16c P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:12.473913 20068 raft_consensus.cc:697] T b0a293e0094a4d9e937af6454e1ea16c P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Running, Role: LEADER
I20260925 13:56:12.474658 20068 consensus_queue.cc:237] T b0a293e0094a4d9e937af6454e1ea16c P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:12.482779 19942 tablet_bootstrap.cc:654] T 293b017a83cc464a9f023a0923310796 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:12.482756 19955 ts_tablet_manager.cc:1929] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:56:12.483220 19955 log.cc:1199] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/8bd480f8e7a3487eb6b74a7ac7544b0d
I20260925 13:56:12.484555 19955 ts_tablet_manager.cc:1950] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:12.484848 20024 raft_consensus.cc:515] T 6b52ed579b19479686247d3ecba9489a P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:12.485930 19938 tablet_bootstrap.cc:492] T c1f15b9de572479aae6c2e644da3392a P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:12.486333 19938 ts_tablet_manager.cc:1403] T c1f15b9de572479aae6c2e644da3392a P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.055s user 0.009s sys 0.000s
I20260925 13:56:12.487538 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b52ed579b19479686247d3ecba9489a" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:12.488065 19555 raft_consensus.cc:3060] T 6b52ed579b19479686247d3ecba9489a P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:12.488747 19938 raft_consensus.cc:359] T c1f15b9de572479aae6c2e644da3392a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:12.489339 19938 raft_consensus.cc:385] T c1f15b9de572479aae6c2e644da3392a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:12.489585 19938 raft_consensus.cc:740] T c1f15b9de572479aae6c2e644da3392a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:12.489508 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 29) took 1049 ms. Trace:
I20260925 13:56:12.490448 19955 rpcz_store.cc:276] 0925 13:56:11.440454 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:11.440558 (+ 104us) service_pool.cc:224] Handling call
0925 13:56:11.455086 (+ 14528us) ts_tablet_manager.cc:1151] Deleting tablet 8bd480f8e7a3487eb6b74a7ac7544b0d
0925 13:56:11.455139 (+ 53us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:11.725693 (+270554us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:12.466767 (+741074us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:12.489447 (+ 22680us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":13955}
I20260925 13:56:12.498494 19696 raft_consensus.cc:2468] T 3b10ecb0e6064d42a2521c4c4e91c028 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:12.494496 19555 raft_consensus.cc:2468] T 6b52ed579b19479686247d3ecba9489a P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
I20260925 13:56:12.499673 19488 leader_election.cc:304] T 3b10ecb0e6064d42a2521c4c4e91c028 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3; no voters: f790493ba7ad403993b678f2df2b55be
I20260925 13:56:12.501811 20030 raft_consensus.cc:493] T f4429db544534f758883f2fae184ea6d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:12.502188 20030 raft_consensus.cc:515] T f4429db544534f758883f2fae184ea6d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:12.504032 20030 leader_election.cc:290] T f4429db544534f758883f2fae184ea6d P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:12.504419 19629 leader_election.cc:304] T 6b52ed579b19479686247d3ecba9489a P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3; no voters:
I20260925 13:56:12.505972 20068 raft_consensus.cc:2804] T 3b10ecb0e6064d42a2521c4c4e91c028 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:12.506541 20068 raft_consensus.cc:697] T 3b10ecb0e6064d42a2521c4c4e91c028 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Running, Role: LEADER
I20260925 13:56:12.502429 19955 tablet_replica.cc:333] T ff3d09046b854fdaab2f95feff589ac1 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:12.510236 19955 raft_consensus.cc:2243] T ff3d09046b854fdaab2f95feff589ac1 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:12.511169 19955 raft_consensus.cc:2272] T ff3d09046b854fdaab2f95feff589ac1 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:12.500358 19938 consensus_queue.cc:260] T c1f15b9de572479aae6c2e644da3392a P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:12.513047 20068 consensus_queue.cc:237] T 3b10ecb0e6064d42a2521c4c4e91c028 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:12.515690 19955 ts_tablet_manager.cc:1916] T ff3d09046b854fdaab2f95feff589ac1 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:12.517655 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4429db544534f758883f2fae184ea6d" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:12.518200 19555 raft_consensus.cc:2468] T f4429db544534f758883f2fae184ea6d P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 0.
I20260925 13:56:12.519394 19773 leader_election.cc:304] T f4429db544534f758883f2fae184ea6d P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:12.522111 19976 raft_consensus.cc:2804] T f4429db544534f758883f2fae184ea6d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:12.522516 19976 raft_consensus.cc:493] T f4429db544534f758883f2fae184ea6d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:12.522899 19976 raft_consensus.cc:3060] T f4429db544534f758883f2fae184ea6d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:12.463228 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet d9455b98734147fb9e2d47c69513424a (table default.Table_c1c831f6c04f4135801dcafbaf7a0843 [id=018679db079c42d2b70631ecb8f241a5]) successfully deleted
W20260925 13:56:12.528752 19266 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 65570us
I20260925 13:56:12.534981 19276 catalog_manager.cc:5838] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194). New cstate: current_term: 1 leader_uuid: "7c1b52895a14482fb54a6758f5054f8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: HEALTHY } } }
I20260925 13:56:12.536620 19276 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet f0cdc99cde2f4dabb819e3e171c5d8d9 (table default.Table_5d6ee0b4200c486ca1fdec8349603746 [id=7e7ee975d4bb457eb3cc4632e12f3cd5]). Expected version 1 got 0
I20260925 13:56:12.537118 19976 raft_consensus.cc:515] T f4429db544534f758883f2fae184ea6d P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:12.512723 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet 8bd480f8e7a3487eb6b74a7ac7544b0d (table default.Table_f7190a6e9e55451c8177b52fbaa2a61d [id=3fd25e8e73f946b8ba379c2876e1386e]) successfully deleted
I20260925 13:56:12.538947 19976 leader_election.cc:290] T f4429db544534f758883f2fae184ea6d P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:12.509617 19275 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_4559d216bb6041bdba82905a01b2412a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:12.540702 19969 raft_consensus.cc:2804] T 6b52ed579b19479686247d3ecba9489a P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:12.540896 19694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4429db544534f758883f2fae184ea6d" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:12.541400 19969 raft_consensus.cc:697] T 6b52ed579b19479686247d3ecba9489a P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Becoming Leader. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Running, Role: LEADER
I20260925 13:56:12.542165 19969 consensus_queue.cc:237] T 6b52ed579b19479686247d3ecba9489a P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:12.543664 19773 leader_election.cc:343] T f4429db544534f758883f2fae184ea6d P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:12.546162 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4429db544534f758883f2fae184ea6d" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:12.547365 19773 leader_election.cc:343] T f4429db544534f758883f2fae184ea6d P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:12.548182 20024 leader_election.cc:290] T 6b52ed579b19479686247d3ecba9489a P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:12.551535 19938 ts_tablet_manager.cc:1434] T c1f15b9de572479aae6c2e644da3392a P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.065s user 0.006s sys 0.001s
I20260925 13:56:12.551815 19840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b52ed579b19479686247d3ecba9489a" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:12.552395 19840 raft_consensus.cc:3060] T 6b52ed579b19479686247d3ecba9489a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:12.559751 19955 ts_tablet_manager.cc:1929] T ff3d09046b854fdaab2f95feff589ac1 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:12.559708 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4429db544534f758883f2fae184ea6d" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:12.560541 19555 raft_consensus.cc:3060] T f4429db544534f758883f2fae184ea6d P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:12.501080 19274 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_6a1a1e9c963f4428a5028a5c9d9384a9" } new_table_name: "default.Table_3a3e48a0641a43a1a783c6ba6374a708" modify_external_catalogs: true
I20260925 13:56:12.568709 20030 consensus_queue.cc:1048] T f51c5ad86ce64891a6393c1dc8d414e5 P f790493ba7ad403993b678f2df2b55be [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:12.573329 20030 consensus_queue.cc:1048] T f51c5ad86ce64891a6393c1dc8d414e5 P f790493ba7ad403993b678f2df2b55be [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:12.576418 20011 ts_tablet_manager.cc:1929] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:56:12.576838 20011 log.cc:1199] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/8bd480f8e7a3487eb6b74a7ac7544b0d
I20260925 13:56:12.577787 20011 ts_tablet_manager.cc:1950] T 8bd480f8e7a3487eb6b74a7ac7544b0d P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:12.560158 19955 log.cc:1199] T ff3d09046b854fdaab2f95feff589ac1 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/ff3d09046b854fdaab2f95feff589ac1
I20260925 13:56:12.583714 19955 ts_tablet_manager.cc:1950] T ff3d09046b854fdaab2f95feff589ac1 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:12.584076 19938 tablet_bootstrap.cc:492] T 00b500182bb04a72b80cfa9103e85fa9 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:12.586396 19163 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskYmJttM/build/tsan/bin/kudu
/tmp/dist-test-taskYmJttM/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.182.252:36181
--webserver_interface=127.18.182.252
--webserver_port=42511
--builtin_ntp_servers=127.18.182.212:43091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.182.254:40825,127.18.182.253:39641,127.18.182.252:36181
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260925 13:56:12.589157 19942 tablet_bootstrap.cc:492] T 293b017a83cc464a9f023a0923310796 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:12.589586 19942 ts_tablet_manager.cc:1403] T 293b017a83cc464a9f023a0923310796 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.484s user 0.011s sys 0.000s
I20260925 13:56:12.599972 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet ff3d09046b854fdaab2f95feff589ac1 (table default.Table_8ac29bbd5011471a9b60f334fc78a3d3 [id=689d277ceae34fc0bb1f2ecd43295722]) successfully deleted
I20260925 13:56:12.601227 19280 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_fa74b192993b433681028dd598b6a56b" } new_table_name: "default.Table_74b6076b22674a31bc58fef71cc9bfb1" modify_external_catalogs: true
I20260925 13:56:12.602058 19955 tablet_replica.cc:333] T d9455b98734147fb9e2d47c69513424a P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:12.602705 19955 raft_consensus.cc:2243] T d9455b98734147fb9e2d47c69513424a P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:12.603171 19955 raft_consensus.cc:2272] T d9455b98734147fb9e2d47c69513424a P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:12.605244 19955 ts_tablet_manager.cc:1916] T d9455b98734147fb9e2d47c69513424a P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 13:56:12.611593 19275 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4559d216bb6041bdba82905a01b2412a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:12.612666 19277 catalog_manager.cc:6128] Not sending DeleteTablet requests; no consensus state for tablet 795203292e8f4c3fa5127eeab8847aa9
I20260925 13:56:12.624998 19273 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_4a3bc798b06b415eab163e79fbf2ee5e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:12.626641 19273 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4a3bc798b06b415eab163e79fbf2ee5e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:12.630537 19279 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_374792fc278744f5ad6b9c1818b7a370"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:12.632120 19279 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_374792fc278744f5ad6b9c1818b7a370 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:12.632145 19942 raft_consensus.cc:359] T 293b017a83cc464a9f023a0923310796 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:12.632774 19942 raft_consensus.cc:385] T 293b017a83cc464a9f023a0923310796 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:12.633024 19942 raft_consensus.cc:740] T 293b017a83cc464a9f023a0923310796 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:12.647794 19271 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_81fefa73a0904acb9b300de03fb48254" } modify_external_catalogs: true
I20260925 13:56:12.648343 19271 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_81fefa73a0904acb9b300de03fb48254" } modify_external_catalogs: true
I20260925 13:56:12.645757 19942 consensus_queue.cc:260] T 293b017a83cc464a9f023a0923310796 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:12.649117 19938 tablet_bootstrap.cc:654] T 00b500182bb04a72b80cfa9103e85fa9 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:12.655299 19942 ts_tablet_manager.cc:1434] T 293b017a83cc464a9f023a0923310796 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.065s user 0.007s sys 0.000s
I20260925 13:56:12.656179 19942 tablet_bootstrap.cc:492] T ff3d09046b854fdaab2f95feff589ac1 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:12.674115 19942 tablet_bootstrap.cc:654] T ff3d09046b854fdaab2f95feff589ac1 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:12.684760 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 25) took 1216 ms. Trace:
I20260925 13:56:12.685300 20011 rpcz_store.cc:276] 0925 13:56:11.468360 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:11.474673 (+ 6313us) service_pool.cc:224] Handling call
0925 13:56:11.876450 (+401777us) ts_tablet_manager.cc:1151] Deleting tablet 8bd480f8e7a3487eb6b74a7ac7544b0d
0925 13:56:11.907971 (+ 31521us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:12.164872 (+256901us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:12.576290 (+411418us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:12.684697 (+108407us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":372653}
I20260925 13:56:12.693887 19555 raft_consensus.cc:2468] T f4429db544534f758883f2fae184ea6d P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 1.
I20260925 13:56:12.724256 20011 tablet_replica.cc:333] T b0dffbfd867040fd99e0fdc77e732f74 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:12.725139 20011 raft_consensus.cc:2243] T b0dffbfd867040fd99e0fdc77e732f74 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:12.725688 20011 raft_consensus.cc:2272] T b0dffbfd867040fd99e0fdc77e732f74 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:12.727559 20011 ts_tablet_manager.cc:1916] T b0dffbfd867040fd99e0fdc77e732f74 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:12.728250 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 8bd480f8e7a3487eb6b74a7ac7544b0d (table default.Table_f7190a6e9e55451c8177b52fbaa2a61d [id=3fd25e8e73f946b8ba379c2876e1386e]) successfully deleted
I20260925 13:56:12.730719 19938 tablet_bootstrap.cc:492] T 00b500182bb04a72b80cfa9103e85fa9 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:12.731101 19938 ts_tablet_manager.cc:1403] T 00b500182bb04a72b80cfa9103e85fa9 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.147s user 0.013s sys 0.002s
I20260925 13:56:12.559307 19840 raft_consensus.cc:2468] T 6b52ed579b19479686247d3ecba9489a P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
I20260925 13:56:12.736138 19938 raft_consensus.cc:359] T 00b500182bb04a72b80cfa9103e85fa9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:12.736869 19938 raft_consensus.cc:385] T 00b500182bb04a72b80cfa9103e85fa9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:12.737149 19938 raft_consensus.cc:740] T 00b500182bb04a72b80cfa9103e85fa9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:12.737859 19938 consensus_queue.cc:260] T 00b500182bb04a72b80cfa9103e85fa9 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:12.748401 19938 ts_tablet_manager.cc:1434] T 00b500182bb04a72b80cfa9103e85fa9 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.017s user 0.007s sys 0.001s
I20260925 13:56:12.752278 19938 tablet_bootstrap.cc:492] T 18a6d1c4eb7948149e709a380da50d93 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:12.754845 19955 ts_tablet_manager.cc:1929] T d9455b98734147fb9e2d47c69513424a P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:12.755236 19955 log.cc:1199] T d9455b98734147fb9e2d47c69513424a P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/d9455b98734147fb9e2d47c69513424a
I20260925 13:56:12.755905 19955 ts_tablet_manager.cc:1950] T d9455b98734147fb9e2d47c69513424a P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:12.757611 19938 tablet_bootstrap.cc:654] T 18a6d1c4eb7948149e709a380da50d93 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:12.761268 19942 tablet_bootstrap.cc:492] T ff3d09046b854fdaab2f95feff589ac1 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:12.761722 19942 ts_tablet_manager.cc:1403] T ff3d09046b854fdaab2f95feff589ac1 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.106s user 0.016s sys 0.002s
I20260925 13:56:12.765136 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet d9455b98734147fb9e2d47c69513424a (table default.Table_c1c831f6c04f4135801dcafbaf7a0843 [id=018679db079c42d2b70631ecb8f241a5]) successfully deleted
W20260925 13:56:12.766708 19274 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 47521fb74f9c40ee999a7f704aa551c9 has no consensus state
I20260925 13:56:12.767901 19942 raft_consensus.cc:359] T ff3d09046b854fdaab2f95feff589ac1 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:12.768520 19942 raft_consensus.cc:385] T ff3d09046b854fdaab2f95feff589ac1 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:12.768800 19942 raft_consensus.cc:740] T ff3d09046b854fdaab2f95feff589ac1 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
W20260925 13:56:12.772794 19280 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1513c0048bd244028d3dfa8d118bc14e consensus state has no leader
I20260925 13:56:12.772796 19942 consensus_queue.cc:260] T ff3d09046b854fdaab2f95feff589ac1 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:12.775068 19942 ts_tablet_manager.cc:1434] T ff3d09046b854fdaab2f95feff589ac1 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.013s user 0.006s sys 0.001s
I20260925 13:56:12.775905 19942 tablet_bootstrap.cc:492] T 28ac2912b94a4928967efb7a731c42b9 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:12.784802 19773 leader_election.cc:304] T f4429db544534f758883f2fae184ea6d P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:12.785890 19976 raft_consensus.cc:2804] T f4429db544534f758883f2fae184ea6d P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:12.786319 19976 raft_consensus.cc:697] T f4429db544534f758883f2fae184ea6d P f790493ba7ad403993b678f2df2b55be [term 1 LEADER]: Becoming Leader. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Running, Role: LEADER
I20260925 13:56:12.787109 19976 consensus_queue.cc:237] T f4429db544534f758883f2fae184ea6d P f790493ba7ad403993b678f2df2b55be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:12.794759 19938 tablet_bootstrap.cc:492] T 18a6d1c4eb7948149e709a380da50d93 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:12.795167 19938 ts_tablet_manager.cc:1403] T 18a6d1c4eb7948149e709a380da50d93 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.043s user 0.005s sys 0.004s
I20260925 13:56:12.797444 19938 raft_consensus.cc:359] T 18a6d1c4eb7948149e709a380da50d93 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:12.798038 19938 raft_consensus.cc:385] T 18a6d1c4eb7948149e709a380da50d93 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:12.798272 19938 raft_consensus.cc:740] T 18a6d1c4eb7948149e709a380da50d93 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:12.803712 19939 tablet_bootstrap.cc:492] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:12.804106 19939 ts_tablet_manager.cc:1403] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.432s user 0.020s sys 0.006s
I20260925 13:56:12.806628 19939 raft_consensus.cc:359] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:12.807229 19939 raft_consensus.cc:385] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:12.807760 19939 raft_consensus.cc:740] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:12.808827 19817 tablet_service.cc:1511] Processing CreateTablet for tablet 47521fb74f9c40ee999a7f704aa551c9 (DEFAULT_TABLE table=default.Table_3a3e48a0641a43a1a783c6ba6374a708 [id=e64fe44a237f48859a288a87a565ee5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:12.809129 19942 tablet_bootstrap.cc:654] T 28ac2912b94a4928967efb7a731c42b9 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:12.809904 19817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47521fb74f9c40ee999a7f704aa551c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:12.809904 19939 consensus_queue.cc:260] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:12.805503 19272 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 28ac2912b94a4928967efb7a731c42b9 (table default.Table_52f57b4f58b44f28989ffd9a4afe2eac [id=56afa81b40064727a7facfe3ccb22571]). Expected version 1 got 0
I20260925 13:56:12.814109 19532 tablet_service.cc:1511] Processing CreateTablet for tablet 47521fb74f9c40ee999a7f704aa551c9 (DEFAULT_TABLE table=default.Table_3a3e48a0641a43a1a783c6ba6374a708 [id=e64fe44a237f48859a288a87a565ee5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:12.814503 19676 tablet_service.cc:1511] Processing CreateTablet for tablet 47521fb74f9c40ee999a7f704aa551c9 (DEFAULT_TABLE table=default.Table_3a3e48a0641a43a1a783c6ba6374a708 [id=e64fe44a237f48859a288a87a565ee5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:12.815580 19532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47521fb74f9c40ee999a7f704aa551c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:12.816114 20032 raft_consensus.cc:493] T 28ac2912b94a4928967efb7a731c42b9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:12.816534 19272 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 51a4c21e06ca4b5b971cf8f04ffa52a9 (table default.Table_9786cd5857464d5f9008e6e630c41899 [id=fe0f78d1c6e44f589aede852f77f88fc]). Expected version 1 got 0
I20260925 13:56:12.816608 20032 raft_consensus.cc:515] T 28ac2912b94a4928967efb7a731c42b9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:12.818488 20032 leader_election.cc:290] T 28ac2912b94a4928967efb7a731c42b9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:12.827593 19278 master_service.cc:960] ReplaceTablet: received request to replace tablet 00b500182bb04a72b80cfa9103e85fa9 from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:12.827711 19938 consensus_queue.cc:260] T 18a6d1c4eb7948149e709a380da50d93 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:12.828104 19673 tablet_service.cc:1511] Processing CreateTablet for tablet 7292d04b411c415394e9d0df85aaf579 (DEFAULT_TABLE table=default.Table_06cdecf307e549bf908eb64143c31996 [id=d2bc135460ab4f2fad4740a281bfc6e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:12.829723 19938 ts_tablet_manager.cc:1434] T 18a6d1c4eb7948149e709a380da50d93 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.034s user 0.003s sys 0.000s
I20260925 13:56:12.830127 19533 tablet_service.cc:1511] Processing CreateTablet for tablet 7292d04b411c415394e9d0df85aaf579 (DEFAULT_TABLE table=default.Table_06cdecf307e549bf908eb64143c31996 [id=d2bc135460ab4f2fad4740a281bfc6e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:12.831188 19533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7292d04b411c415394e9d0df85aaf579. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:12.831058 19818 tablet_service.cc:1511] Processing CreateTablet for tablet 7292d04b411c415394e9d0df85aaf579 (DEFAULT_TABLE table=default.Table_06cdecf307e549bf908eb64143c31996 [id=d2bc135460ab4f2fad4740a281bfc6e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:12.832047 19818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7292d04b411c415394e9d0df85aaf579. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:12.839381 19675 tablet_service.cc:1511] Processing CreateTablet for tablet 980d1264c0ea45f3b2b762509e0b2c77 (DEFAULT_TABLE table=default.Table_8290acbcb9d84a48aad169c54251bc19 [id=90312b1b0c254ff9ad5693f561114a7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:12.840580 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ac2912b94a4928967efb7a731c42b9" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:12.841025 19280 master_service.cc:960] ReplaceTablet: received request to replace tablet 293b017a83cc464a9f023a0923310796 from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:12.842674 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ac2912b94a4928967efb7a731c42b9" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:12.843250 19554 raft_consensus.cc:2468] T 28ac2912b94a4928967efb7a731c42b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 0.
I20260925 13:56:12.844020 19676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47521fb74f9c40ee999a7f704aa551c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:12.844427 19939 ts_tablet_manager.cc:1434] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.040s user 0.005s sys 0.000s
I20260925 13:56:12.847728 19673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7292d04b411c415394e9d0df85aaf579. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:12.848001 19939 tablet_bootstrap.cc:492] T 00b500182bb04a72b80cfa9103e85fa9 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:12.851804 19675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 980d1264c0ea45f3b2b762509e0b2c77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:12.853364 19274 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_52f57b4f58b44f28989ffd9a4afe2eac" } modify_external_catalogs: true
I20260925 13:56:12.853879 19274 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_52f57b4f58b44f28989ffd9a4afe2eac" } modify_external_catalogs: true
I20260925 13:56:12.826699 19277 catalog_manager.cc:5838] T 293b017a83cc464a9f023a0923310796 P 8f578869bc084a5d8a9a69f143954eb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "8f578869bc084a5d8a9a69f143954eb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:12.859686 19535 tablet_service.cc:1511] Processing CreateTablet for tablet 980d1264c0ea45f3b2b762509e0b2c77 (DEFAULT_TABLE table=default.Table_8290acbcb9d84a48aad169c54251bc19 [id=90312b1b0c254ff9ad5693f561114a7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:12.860711 19535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 980d1264c0ea45f3b2b762509e0b2c77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:12.860994 19672 tablet_service.cc:1511] Processing CreateTablet for tablet dc14c37aa393407a87af0d0eb8b88d82 (DEFAULT_TABLE table=default.Table_8a18eca530ed42b7808ab612d9bad5e7 [id=e6f2a872536b4e55b071231020903581]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:12.861969 19672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc14c37aa393407a87af0d0eb8b88d82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:12.879632 19773 leader_election.cc:304] T 28ac2912b94a4928967efb7a731c42b9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:12.880336 20030 raft_consensus.cc:2804] T 28ac2912b94a4928967efb7a731c42b9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:12.880661 20030 raft_consensus.cc:493] T 28ac2912b94a4928967efb7a731c42b9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:12.880954 20030 raft_consensus.cc:3060] T 28ac2912b94a4928967efb7a731c42b9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:12.881027 19277 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet 293b017a83cc464a9f023a0923310796 (table default.Table_da4fc82971544f509fcd31edfea882da [id=d2bf5d31983a47d897e93ace83ecf026]). Expected version 1 got 0
I20260925 13:56:12.882474 19277 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet 28ac2912b94a4928967efb7a731c42b9 (table default.Table_52f57b4f58b44f28989ffd9a4afe2eac [id=56afa81b40064727a7facfe3ccb22571]). Expected version 1 got 0
W20260925 13:56:12.887744 19773 leader_election.cc:343] T 28ac2912b94a4928967efb7a731c42b9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:12.889448 19815 tablet_service.cc:1511] Processing CreateTablet for tablet 980d1264c0ea45f3b2b762509e0b2c77 (DEFAULT_TABLE table=default.Table_8290acbcb9d84a48aad169c54251bc19 [id=90312b1b0c254ff9ad5693f561114a7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:12.890473 19815 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 980d1264c0ea45f3b2b762509e0b2c77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:12.892398 19820 tablet_service.cc:1511] Processing CreateTablet for tablet dc14c37aa393407a87af0d0eb8b88d82 (DEFAULT_TABLE table=default.Table_8a18eca530ed42b7808ab612d9bad5e7 [id=e6f2a872536b4e55b071231020903581]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:12.893335 19820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc14c37aa393407a87af0d0eb8b88d82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:12.895524 19276 catalog_manager.cc:6391] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: default.Table_81fefa73a0904acb9b300de03fb48254 [id=dadfbd02a0a640949b1dc46c01db7bc8] alter complete (version 2)
I20260925 13:56:12.903482 20032 raft_consensus.cc:493] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:12.903795 19939 tablet_bootstrap.cc:654] T 00b500182bb04a72b80cfa9103e85fa9 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:12.903892 20032 raft_consensus.cc:515] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:12.905854 20032 leader_election.cc:290] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:12.903534 19534 tablet_service.cc:1511] Processing CreateTablet for tablet dc14c37aa393407a87af0d0eb8b88d82 (DEFAULT_TABLE table=default.Table_8a18eca530ed42b7808ab612d9bad5e7 [id=e6f2a872536b4e55b071231020903581]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:12.912688 20011 ts_tablet_manager.cc:1929] T b0dffbfd867040fd99e0fdc77e732f74 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:12.913625 20011 log.cc:1199] T b0dffbfd867040fd99e0fdc77e732f74 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/b0dffbfd867040fd99e0fdc77e732f74
I20260925 13:56:12.914357 20011 ts_tablet_manager.cc:1950] T b0dffbfd867040fd99e0fdc77e732f74 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:12.915800 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51a4c21e06ca4b5b971cf8f04ffa52a9" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:12.916420 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 26) took 1447 ms. Trace:
W20260925 13:56:12.917184 19773 leader_election.cc:343] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:12.916785 20011 rpcz_store.cc:276] 0925 13:56:11.502828 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:11.507536 (+ 4708us) service_pool.cc:224] Handling call
0925 13:56:12.724078 (+1216542us) ts_tablet_manager.cc:1151] Deleting tablet b0dffbfd867040fd99e0fdc77e732f74
0925 13:56:12.724135 (+ 57us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:12.877449 (+153314us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:12.912581 (+ 35132us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:12.916361 (+ 3780us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1186920}
I20260925 13:56:12.918785 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51a4c21e06ca4b5b971cf8f04ffa52a9" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:12.919394 19554 raft_consensus.cc:2468] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 0.
I20260925 13:56:12.920507 19278 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 00b500182bb04a72b80cfa9103e85fa9
I20260925 13:56:12.921967 19278 catalog_manager.cc:6934] ReplaceTablet: tablet 00b500182bb04a72b80cfa9103e85fa9 deleted and replaced by tablet f5a0bd0e205f4889be93739dfbd2a816
I20260925 13:56:12.923974 19534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc14c37aa393407a87af0d0eb8b88d82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:12.926864 19674 tablet_service.cc:1558] Processing DeleteTablet for tablet 00b500182bb04a72b80cfa9103e85fa9 with delete_type TABLET_DATA_DELETED (replaced by tablet f5a0bd0e205f4889be93739dfbd2a816) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:12.927522 20032 raft_consensus.cc:493] T 1513c0048bd244028d3dfa8d118bc14e P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:12.927940 20032 raft_consensus.cc:515] T 1513c0048bd244028d3dfa8d118bc14e P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:12.928283 19272 catalog_manager.cc:6391] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: default.Table_72b4fdbfbca745a286dca8c191dc2286 [id=41786d7cdac8440ca86768cac5b28f49] alter complete (version 1)
I20260925 13:56:12.929344 19773 leader_election.cc:304] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:12.929741 20032 leader_election.cc:290] T 1513c0048bd244028d3dfa8d118bc14e P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:12.930182 20032 raft_consensus.cc:2804] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:12.930480 20032 raft_consensus.cc:493] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:12.930765 20032 raft_consensus.cc:3060] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:12.932509 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1513c0048bd244028d3dfa8d118bc14e" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:12.932869 20030 raft_consensus.cc:515] T 28ac2912b94a4928967efb7a731c42b9 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:12.934473 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1513c0048bd244028d3dfa8d118bc14e" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:12.934623 20030 leader_election.cc:290] T 28ac2912b94a4928967efb7a731c42b9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:12.935585 19819 tablet_service.cc:1558] Processing DeleteTablet for tablet 00b500182bb04a72b80cfa9103e85fa9 with delete_type TABLET_DATA_DELETED (replaced by tablet f5a0bd0e205f4889be93739dfbd2a816) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:12.935727 19674 tablet_service.cc:1558] Processing DeleteTablet for tablet 3b10ecb0e6064d42a2521c4c4e91c028 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:11 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:12.936467 19959 tablet_replica.cc:333] T 00b500182bb04a72b80cfa9103e85fa9 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:12.936496 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ac2912b94a4928967efb7a731c42b9" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:12.936934 19554 raft_consensus.cc:3060] T 28ac2912b94a4928967efb7a731c42b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:12.937132 19959 raft_consensus.cc:2243] T 00b500182bb04a72b80cfa9103e85fa9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:12.937526 19959 raft_consensus.cc:2272] T 00b500182bb04a72b80cfa9103e85fa9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:12.940979 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ac2912b94a4928967efb7a731c42b9" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
W20260925 13:56:12.942206 19773 leader_election.cc:343] T 1513c0048bd244028d3dfa8d118bc14e P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:12.942775 19773 leader_election.cc:343] T 28ac2912b94a4928967efb7a731c42b9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:12.943300 19773 leader_election.cc:343] T 1513c0048bd244028d3dfa8d118bc14e P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:12.943531 19959 ts_tablet_manager.cc:1916] T 00b500182bb04a72b80cfa9103e85fa9 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:12.943734 19773 leader_election.cc:304] T 1513c0048bd244028d3dfa8d118bc14e P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:12.947489 20030 raft_consensus.cc:2749] T 1513c0048bd244028d3dfa8d118bc14e P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:12.966172 19554 raft_consensus.cc:2468] T 28ac2912b94a4928967efb7a731c42b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 1.
I20260925 13:56:12.970727 19938 tablet_bootstrap.cc:492] T 980d1264c0ea45f3b2b762509e0b2c77 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:12.976221 19938 tablet_bootstrap.cc:654] T 980d1264c0ea45f3b2b762509e0b2c77 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:12.967919 19533 tablet_service.cc:1558] Processing DeleteTablet for tablet 00b500182bb04a72b80cfa9103e85fa9 with delete_type TABLET_DATA_DELETED (replaced by tablet f5a0bd0e205f4889be93739dfbd2a816) from {username='slave'} at 127.0.0.1:40018
W20260925 13:56:12.978720 19264 catalog_manager.cc:5043] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): delete failed for tablet 00b500182bb04a72b80cfa9103e85fa9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 00b500182bb04a72b80cfa9103e85fa9 already in progress: creating tablet
I20260925 13:56:12.981549 19274 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 28ac2912b94a4928967efb7a731c42b9
I20260925 13:56:12.986934 19819 tablet_service.cc:1558] Processing DeleteTablet for tablet 28ac2912b94a4928967efb7a731c42b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:12 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:12.988132 19275 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_4f3bc32149e141ef897cf43378f9020c" } new_table_name: "default.Table_256401fd998c430db7f494657ef8e75d" modify_external_catalogs: true
I20260925 13:56:12.989192 19674 tablet_service.cc:1558] Processing DeleteTablet for tablet 28ac2912b94a4928967efb7a731c42b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:12 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:12.990090 19533 tablet_service.cc:1558] Processing DeleteTablet for tablet 28ac2912b94a4928967efb7a731c42b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:12 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:12.990828 19955 tablet_replica.cc:333] T 28ac2912b94a4928967efb7a731c42b9 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:12.991732 19278 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_4a3bc798b06b415eab163e79fbf2ee5e" } new_table_name: "default.Table_7cc40850827b4347a71d511dc1aed755" modify_external_catalogs: true
I20260925 13:56:12.992198 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet b0dffbfd867040fd99e0fdc77e732f74 (table default.Table_bb731091526c4e0d95e68171b5b70a60 [id=80deba2b504744738a9fca818e546a7c]) successfully deleted
I20260925 13:56:12.993100 20011 tablet_replica.cc:333] T b8200d0044434788adc1f7ec29aefa59 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:12.995779 19274 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_430d31bd6eb64dfea4d41806e919d5bf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:12.996990 19942 tablet_bootstrap.cc:492] T 28ac2912b94a4928967efb7a731c42b9 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:12.997383 19942 ts_tablet_manager.cc:1403] T 28ac2912b94a4928967efb7a731c42b9 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.222s user 0.003s sys 0.007s
W20260925 13:56:12.997386 19274 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_430d31bd6eb64dfea4d41806e919d5bf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:13.002555 19955 raft_consensus.cc:2243] T 28ac2912b94a4928967efb7a731c42b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:13.004916 19955 raft_consensus.cc:2272] T 28ac2912b94a4928967efb7a731c42b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:13.006256 19280 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 293b017a83cc464a9f023a0923310796
I20260925 13:56:13.006481 19955 ts_tablet_manager.cc:1916] T 28ac2912b94a4928967efb7a731c42b9 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:13.006742 19938 tablet_bootstrap.cc:492] T 980d1264c0ea45f3b2b762509e0b2c77 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:13.007099 19938 ts_tablet_manager.cc:1403] T 980d1264c0ea45f3b2b762509e0b2c77 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.037s user 0.009s sys 0.000s
I20260925 13:56:13.007932 19280 catalog_manager.cc:6934] ReplaceTablet: tablet 293b017a83cc464a9f023a0923310796 deleted and replaced by tablet 15e580a6be094a769a9eebbd4e2bc4e7
I20260925 13:56:13.008417 19674 tablet_service.cc:1558] Processing DeleteTablet for tablet 293b017a83cc464a9f023a0923310796 with delete_type TABLET_DATA_DELETED (replaced by tablet 15e580a6be094a769a9eebbd4e2bc4e7) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:13.009426 19938 raft_consensus.cc:359] T 980d1264c0ea45f3b2b762509e0b2c77 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:13.010031 19938 raft_consensus.cc:385] T 980d1264c0ea45f3b2b762509e0b2c77 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:13.010267 19938 raft_consensus.cc:740] T 980d1264c0ea45f3b2b762509e0b2c77 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:13.010262 20011 raft_consensus.cc:2243] T b8200d0044434788adc1f7ec29aefa59 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:13.010726 20011 raft_consensus.cc:2272] T b8200d0044434788adc1f7ec29aefa59 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:13.010949 19938 consensus_queue.cc:260] T 980d1264c0ea45f3b2b762509e0b2c77 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:13.012660 20011 ts_tablet_manager.cc:1916] T b8200d0044434788adc1f7ec29aefa59 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:13.026923 19942 raft_consensus.cc:359] T 28ac2912b94a4928967efb7a731c42b9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:13.027653 19942 raft_consensus.cc:385] T 28ac2912b94a4928967efb7a731c42b9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:13.027979 19942 raft_consensus.cc:740] T 28ac2912b94a4928967efb7a731c42b9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:13.031175 19820 tablet_service.cc:1558] Processing DeleteTablet for tablet 293b017a83cc464a9f023a0923310796 with delete_type TABLET_DATA_DELETED (replaced by tablet 15e580a6be094a769a9eebbd4e2bc4e7) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:13.034132 19955 ts_tablet_manager.cc:1929] T 28ac2912b94a4928967efb7a731c42b9 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:13.034484 19955 log.cc:1199] T 28ac2912b94a4928967efb7a731c42b9 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/28ac2912b94a4928967efb7a731c42b9
I20260925 13:56:13.035070 19955 ts_tablet_manager.cc:1950] T 28ac2912b94a4928967efb7a731c42b9 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:13.037437 19959 ts_tablet_manager.cc:1929] T 00b500182bb04a72b80cfa9103e85fa9 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:13.038902 19942 consensus_queue.cc:260] T 28ac2912b94a4928967efb7a731c42b9 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:13.041996 19942 ts_tablet_manager.cc:1434] T 28ac2912b94a4928967efb7a731c42b9 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.044s user 0.007s sys 0.000s
I20260925 13:56:13.040138 19959 log.cc:1199] T 00b500182bb04a72b80cfa9103e85fa9 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/00b500182bb04a72b80cfa9103e85fa9
I20260925 13:56:13.042900 19959 ts_tablet_manager.cc:1950] T 00b500182bb04a72b80cfa9103e85fa9 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:13.044102 19535 tablet_service.cc:1558] Processing DeleteTablet for tablet 293b017a83cc464a9f023a0923310796 with delete_type TABLET_DATA_DELETED (replaced by tablet 15e580a6be094a769a9eebbd4e2bc4e7) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:13.044757 19773 leader_election.cc:304] T 28ac2912b94a4928967efb7a731c42b9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:13.045874 20090 raft_consensus.cc:2804] T 28ac2912b94a4928967efb7a731c42b9 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:13.046350 20090 raft_consensus.cc:697] T 28ac2912b94a4928967efb7a731c42b9 P f790493ba7ad403993b678f2df2b55be [term 1 LEADER]: Becoming Leader. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Running, Role: LEADER
I20260925 13:56:13.047153 20090 consensus_queue.cc:237] T 28ac2912b94a4928967efb7a731c42b9 P f790493ba7ad403993b678f2df2b55be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:13.050966 19959 tablet_replica.cc:333] T 28ac2912b94a4928967efb7a731c42b9 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:13.051920 19942 tablet_bootstrap.cc:492] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:13.056489 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 00b500182bb04a72b80cfa9103e85fa9 (table default.Table_04d053f331f243ce8e98beb7c00d4159 [id=745c9e0d4f2c4ce58c3a201c34f1f1e3]) successfully deleted
I20260925 13:56:13.061651 19939 tablet_bootstrap.cc:492] T 00b500182bb04a72b80cfa9103e85fa9 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:13.051545 20032 raft_consensus.cc:515] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:13.063946 19939 ts_tablet_manager.cc:1403] T 00b500182bb04a72b80cfa9103e85fa9 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.216s user 0.015s sys 0.011s
I20260925 13:56:13.064666 20032 leader_election.cc:290] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:13.053164 19938 ts_tablet_manager.cc:1434] T 980d1264c0ea45f3b2b762509e0b2c77 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.046s user 0.006s sys 0.000s
I20260925 13:56:13.065864 19938 tablet_bootstrap.cc:492] T 7292d04b411c415394e9d0df85aaf579 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:13.066349 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51a4c21e06ca4b5b971cf8f04ffa52a9" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
W20260925 13:56:13.067873 19773 leader_election.cc:343] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:13.072129 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet 28ac2912b94a4928967efb7a731c42b9 (table default.Table_52f57b4f58b44f28989ffd9a4afe2eac [id=56afa81b40064727a7facfe3ccb22571]) successfully deleted
I20260925 13:56:13.076164 19942 tablet_bootstrap.cc:654] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:13.068929 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51a4c21e06ca4b5b971cf8f04ffa52a9" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:13.077587 19554 raft_consensus.cc:3060] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
W20260925 13:56:13.087226 19275 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b0a293e0094a4d9e937af6454e1ea16c consensus state has no leader
I20260925 13:56:13.081342 19955 tablet_replica.cc:333] T 293b017a83cc464a9f023a0923310796 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:13.088083 19955 raft_consensus.cc:2243] T 293b017a83cc464a9f023a0923310796 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:13.088796 19955 pending_rounds.cc:70] T 293b017a83cc464a9f023a0923310796 P 8f578869bc084a5d8a9a69f143954eb3: Trying to abort 1 pending ops.
I20260925 13:56:13.090389 19938 tablet_bootstrap.cc:654] T 7292d04b411c415394e9d0df85aaf579 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:13.092211 19279 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_65608937f2cd4986b2588a4831750601"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:13.089053 19955 pending_rounds.cc:77] T 293b017a83cc464a9f023a0923310796 P 8f578869bc084a5d8a9a69f143954eb3: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251368058617856 op_type: NO_OP noop_request { }
I20260925 13:56:13.091583 19554 raft_consensus.cc:2468] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 1.
I20260925 13:56:13.093713 19955 raft_consensus.cc:2889] T 293b017a83cc464a9f023a0923310796 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:56:13.094023 19955 raft_consensus.cc:2272] T 293b017a83cc464a9f023a0923310796 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 13:56:13.094039 19279 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_65608937f2cd4986b2588a4831750601 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:13.094504 19773 leader_election.cc:304] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:13.095654 20032 raft_consensus.cc:2804] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:13.096371 20032 raft_consensus.cc:697] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P f790493ba7ad403993b678f2df2b55be [term 1 LEADER]: Becoming Leader. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Running, Role: LEADER
I20260925 13:56:13.097187 20032 consensus_queue.cc:237] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P f790493ba7ad403993b678f2df2b55be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:13.103411 19955 ts_tablet_manager.cc:1916] T 293b017a83cc464a9f023a0923310796 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:13.104353 19280 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_38131969dbb24c24bc8b3e4f845e48d9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:13.105993 19280 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_38131969dbb24c24bc8b3e4f845e48d9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:13.106170 19969 consensus_queue.cc:1048] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 13:56:13.107836 19959 raft_consensus.cc:2243] T 28ac2912b94a4928967efb7a731c42b9 P f790493ba7ad403993b678f2df2b55be [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:13.116802 19959 pending_rounds.cc:70] T 28ac2912b94a4928967efb7a731c42b9 P f790493ba7ad403993b678f2df2b55be: Trying to abort 1 pending ops.
I20260925 13:56:13.117115 19959 pending_rounds.cc:77] T 28ac2912b94a4928967efb7a731c42b9 P f790493ba7ad403993b678f2df2b55be: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251371214626816 op_type: NO_OP noop_request { }
I20260925 13:56:13.117519 19959 raft_consensus.cc:2889] T 28ac2912b94a4928967efb7a731c42b9 P f790493ba7ad403993b678f2df2b55be [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:56:13.117839 19959 raft_consensus.cc:2272] T 28ac2912b94a4928967efb7a731c42b9 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:13.117596 19275 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_1aa8069da6504812a85380a42ca07583"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:13.119272 19275 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1aa8069da6504812a85380a42ca07583 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:13.124089 19959 ts_tablet_manager.cc:1916] T 28ac2912b94a4928967efb7a731c42b9 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:13.129031 20011 ts_tablet_manager.cc:1929] T b8200d0044434788adc1f7ec29aefa59 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:13.129449 20011 log.cc:1199] T b8200d0044434788adc1f7ec29aefa59 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/b8200d0044434788adc1f7ec29aefa59
I20260925 13:56:13.130102 20011 ts_tablet_manager.cc:1950] T b8200d0044434788adc1f7ec29aefa59 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:13.132464 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 27) took 1625 ms. Trace:
I20260925 13:56:13.132763 20011 rpcz_store.cc:276] 0925 13:56:11.559612 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:11.588577 (+ 28965us) service_pool.cc:224] Handling call
0925 13:56:12.992941 (+1404364us) ts_tablet_manager.cc:1151] Deleting tablet b8200d0044434788adc1f7ec29aefa59
0925 13:56:12.992986 (+ 45us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:13.059343 (+ 66357us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:13.128901 (+ 69558us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:13.132408 (+ 3507us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1403737}
I20260925 13:56:13.134094 20011 tablet_replica.cc:333] T ff3d09046b854fdaab2f95feff589ac1 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:13.134331 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet b8200d0044434788adc1f7ec29aefa59 (table default.Table_cb55c90b6da345868b11789d1177730d [id=36a1da175eaf48548cdd40e1203d422a]) successfully deleted
I20260925 13:56:13.134852 20011 raft_consensus.cc:2243] T ff3d09046b854fdaab2f95feff589ac1 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:13.140928 19955 ts_tablet_manager.cc:1929] T 293b017a83cc464a9f023a0923310796 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:13.141285 19955 log.cc:1199] T 293b017a83cc464a9f023a0923310796 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/293b017a83cc464a9f023a0923310796
I20260925 13:56:13.141654 19938 tablet_bootstrap.cc:492] T 7292d04b411c415394e9d0df85aaf579 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:13.141738 19969 consensus_queue.cc:1048] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:13.142043 19955 ts_tablet_manager.cc:1950] T 293b017a83cc464a9f023a0923310796 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:13.142076 19938 ts_tablet_manager.cc:1403] T 7292d04b411c415394e9d0df85aaf579 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.076s user 0.010s sys 0.000s
I20260925 13:56:13.143250 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet 293b017a83cc464a9f023a0923310796 (table default.Table_da4fc82971544f509fcd31edfea882da [id=d2bf5d31983a47d897e93ace83ecf026]) successfully deleted
I20260925 13:56:13.145138 19938 raft_consensus.cc:359] T 7292d04b411c415394e9d0df85aaf579 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:13.145732 19938 raft_consensus.cc:385] T 7292d04b411c415394e9d0df85aaf579 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:13.145989 19938 raft_consensus.cc:740] T 7292d04b411c415394e9d0df85aaf579 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:13.148144 19942 tablet_bootstrap.cc:492] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:13.148572 19942 ts_tablet_manager.cc:1403] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.097s user 0.010s sys 0.001s
W20260925 13:56:13.156077 19278 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 26644768f516412f8c95d68a76bb99c5 has no consensus state
I20260925 13:56:13.156621 19942 raft_consensus.cc:359] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:13.157284 19942 raft_consensus.cc:385] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:13.157523 19942 raft_consensus.cc:740] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:13.160817 19939 raft_consensus.cc:359] T 00b500182bb04a72b80cfa9103e85fa9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:13.161463 19939 raft_consensus.cc:385] T 00b500182bb04a72b80cfa9103e85fa9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:13.161715 19939 raft_consensus.cc:740] T 00b500182bb04a72b80cfa9103e85fa9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:13.164551 19939 consensus_queue.cc:260] T 00b500182bb04a72b80cfa9103e85fa9 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
W20260925 13:56:13.169175 19267 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet b0a293e0094a4d9e937af6454e1ea16c (table default.Table_256401fd998c430db7f494657ef8e75d [id=b673ac6542c5403fbf0fad16c4917dfd], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet b0a293e0094a4d9e937af6454e1ea16c consensus state has no leader
I20260925 13:56:13.178179 19278 master_service.cc:960] ReplaceTablet: received request to replace tablet f0cdc99cde2f4dabb819e3e171c5d8d9 from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:13.179971 19939 ts_tablet_manager.cc:1434] T 00b500182bb04a72b80cfa9103e85fa9 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.073s user 0.003s sys 0.002s
I20260925 13:56:13.182629 20032 raft_consensus.cc:493] T 18a6d1c4eb7948149e709a380da50d93 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:13.183017 20032 raft_consensus.cc:515] T 18a6d1c4eb7948149e709a380da50d93 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:13.185606 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a6d1c4eb7948149e709a380da50d93" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
W20260925 13:56:13.186944 19773 leader_election.cc:343] T 18a6d1c4eb7948149e709a380da50d93 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:13.187014 19264 catalog_manager.cc:5136] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): alter failed for tablet 293b017a83cc464a9f023a0923310796 (table default.Table_da4fc82971544f509fcd31edfea882da [id=d2bf5d31983a47d897e93ace83ecf026]),no further retry: Not found: Tablet not found: 293b017a83cc464a9f023a0923310796
I20260925 13:56:13.189637 19938 consensus_queue.cc:260] T 7292d04b411c415394e9d0df85aaf579 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:13.191511 19938 ts_tablet_manager.cc:1434] T 7292d04b411c415394e9d0df85aaf579 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.049s user 0.006s sys 0.000s
I20260925 13:56:13.194219 19273 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_da4fc82971544f509fcd31edfea882da" } modify_external_catalogs: true
I20260925 13:56:13.194752 19273 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_da4fc82971544f509fcd31edfea882da" } modify_external_catalogs: true
I20260925 13:56:13.198726 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a6d1c4eb7948149e709a380da50d93" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:13.192886 19942 consensus_queue.cc:260] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
W20260925 13:56:13.200647 19773 leader_election.cc:343] T 18a6d1c4eb7948149e709a380da50d93 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:13.201032 19773 leader_election.cc:304] T 18a6d1c4eb7948149e709a380da50d93 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:13.201884 19976 raft_consensus.cc:2749] T 18a6d1c4eb7948149e709a380da50d93 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:13.202916 20032 leader_election.cc:290] T 18a6d1c4eb7948149e709a380da50d93 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:13.215940 19938 tablet_bootstrap.cc:492] T dc14c37aa393407a87af0d0eb8b88d82 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:13.222448 20103 raft_consensus.cc:493] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:13.223023 19939 tablet_bootstrap.cc:492] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:13.222575 20011 raft_consensus.cc:2272] T ff3d09046b854fdaab2f95feff589ac1 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:13.222954 20103 raft_consensus.cc:515] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:13.230919 19676 tablet_service.cc:1511] Processing CreateTablet for tablet bae8c5b6e1014731901bc3f336104784 (DEFAULT_TABLE table=default.Table_38f7207982be43189299b739031aa942 [id=0736e722eb8c43e9b6a151a7c2880fe3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:13.232002 20103 leader_election.cc:290] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:13.236338 19534 tablet_service.cc:1511] Processing CreateTablet for tablet bae8c5b6e1014731901bc3f336104784 (DEFAULT_TABLE table=default.Table_38f7207982be43189299b739031aa942 [id=0736e722eb8c43e9b6a151a7c2880fe3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:13.237397 19534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bae8c5b6e1014731901bc3f336104784. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:13.237664 19817 tablet_service.cc:1511] Processing CreateTablet for tablet bae8c5b6e1014731901bc3f336104784 (DEFAULT_TABLE table=default.Table_38f7207982be43189299b739031aa942 [id=0736e722eb8c43e9b6a151a7c2880fe3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:13.238910 19817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bae8c5b6e1014731901bc3f336104784. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:13.240298 19840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51a4c21e06ca4b5b971cf8f04ffa52a9" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:13.232517 19676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bae8c5b6e1014731901bc3f336104784. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:13.246495 19675 tablet_service.cc:1511] Processing CreateTablet for tablet b36aebfe984f454ea910e84645c4ac7d (DEFAULT_TABLE table=default.Table_f1d49a17b4c9418a9872e05be0d39488 [id=e2b98c2d1dbf4efea5fa61f182fc2fd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:13.247562 19675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b36aebfe984f454ea910e84645c4ac7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:13.251950 19938 tablet_bootstrap.cc:654] T dc14c37aa393407a87af0d0eb8b88d82 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:13.255841 19820 tablet_service.cc:1511] Processing CreateTablet for tablet b36aebfe984f454ea910e84645c4ac7d (DEFAULT_TABLE table=default.Table_f1d49a17b4c9418a9872e05be0d39488 [id=e2b98c2d1dbf4efea5fa61f182fc2fd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:13.243098 19276 catalog_manager.cc:5838] T 6b52ed579b19479686247d3ecba9489a P 7c1b52895a14482fb54a6758f5054f8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194). New cstate: current_term: 1 leader_uuid: "7c1b52895a14482fb54a6758f5054f8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:13.257690 19535 tablet_service.cc:1511] Processing CreateTablet for tablet b36aebfe984f454ea910e84645c4ac7d (DEFAULT_TABLE table=default.Table_f1d49a17b4c9418a9872e05be0d39488 [id=e2b98c2d1dbf4efea5fa61f182fc2fd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:13.257915 19276 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet f51c5ad86ce64891a6393c1dc8d414e5 (table default.Table_824449a97a864e3dbbf06e2ad678edc5 [id=af708d3ba4c54ba19c24eca862bbddc1]). Expected version 1 got 0
I20260925 13:56:13.258222 19820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b36aebfe984f454ea910e84645c4ac7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:13.258695 19271 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 491b8b5c0f51408b882adee10ce80728
I20260925 13:56:13.258890 19535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b36aebfe984f454ea910e84645c4ac7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:13.254987 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51a4c21e06ca4b5b971cf8f04ffa52a9" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:13.264055 19555 raft_consensus.cc:2393] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7c1b52895a14482fb54a6758f5054f8f in current term 1: Already voted for candidate f790493ba7ad403993b678f2df2b55be in this term.
I20260925 13:56:13.265054 19629 leader_election.cc:304] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
W20260925 13:56:13.266369 19272 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.195:57799 (request call id 12) took 849 ms (client timeout 1000 ms). Trace:
I20260925 13:56:13.268921 19533 tablet_service.cc:1558] Processing DeleteTablet for tablet 491b8b5c0f51408b882adee10ce80728 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:12 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:13.269575 19955 tablet_replica.cc:333] T 491b8b5c0f51408b882adee10ce80728 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:13.270246 19955 raft_consensus.cc:2243] T 491b8b5c0f51408b882adee10ce80728 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:13.270752 19955 raft_consensus.cc:2272] T 491b8b5c0f51408b882adee10ce80728 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 13:56:13.266729 19272 rpcz_store.cc:269] 0925 13:56:12.416940 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:12.419663 (+ 2723us) service_pool.cc:224] Handling call
0925 13:56:13.266301 (+846638us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:12.870353 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:12.870674 (+ 321us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:12.870696 (+ 22us) write_op.cc:435] Acquired schema lock
0925 13:56:12.870708 (+ 12us) tablet.cc:662] Decoding operations
0925 13:56:12.879157 (+ 8449us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:12.879191 (+ 34us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:12.879213 (+ 22us) tablet.cc:685] Acquiring locks for 1 operations
0925 13:56:12.879368 (+ 155us) tablet.cc:701] Row locks acquired
0925 13:56:12.879381 (+ 13us) write_op.cc:260] PREPARE: finished
0925 13:56:12.879488 (+ 107us) write_op.cc:270] Start()
0925 13:56:12.879564 (+ 76us) write_op.cc:276] Timestamp: P: 1790344572879648 usec, L: 0
0925 13:56:12.879579 (+ 15us) op_driver.cc:348] REPLICATION: starting
0925 13:56:12.880025 (+ 446us) log.cc:844] Serialized 355 byte log entry
0925 13:56:12.924023 (+ 43998us) op_driver.cc:464] REPLICATION: finished
0925 13:56:12.926792 (+ 2769us) write_op.cc:301] APPLY: starting
0925 13:56:12.926844 (+ 52us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:12.926945 (+ 101us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:12.927032 (+ 87us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:12.927164 (+ 132us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:12.927285 (+ 121us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 13:56:12.927327 (+ 42us) write_op.cc:312] APPLY: finished
0925 13:56:12.927500 (+ 173us) log.cc:844] Serialized 22 byte log entry
0925 13:56:12.927630 (+ 130us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:12.927681 (+ 51us) write_op.cc:454] Released schema lock
0925 13:56:12.927708 (+ 27us) write_op.cc:341] FINISH: Updating metrics
0925 13:56:12.928869 (+ 1161us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:12.928906 (+ 37us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":7,"child_traces":[["op",{"apply.queue_time_us":2615,"apply.run_cpu_time_us":1130,"apply.run_wall_time_us":2045,"mutex_wait_us":27598,"num_ops":1,"prepare.queue_time_us":12017,"prepare.run_cpu_time_us":1780,"prepare.run_wall_time_us":37560,"raft.queue_time_us":28930,"raft.run_cpu_time_us":10,"raft.run_wall_time_us":10,"replication_time_us":44356}]]}
I20260925 13:56:13.274025 20101 raft_consensus.cc:3060] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:13.273089 19815 tablet_service.cc:1558] Processing DeleteTablet for tablet 491b8b5c0f51408b882adee10ce80728 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:12 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:13.273304 19818 tablet_service.cc:1558] Processing DeleteTablet for tablet d9455b98734147fb9e2d47c69513424a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:11 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:13.273165 19673 tablet_service.cc:1558] Processing DeleteTablet for tablet 491b8b5c0f51408b882adee10ce80728 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:12 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:13.276548 19955 ts_tablet_manager.cc:1916] T 491b8b5c0f51408b882adee10ce80728 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:13.279546 20032 raft_consensus.cc:493] T 7292d04b411c415394e9d0df85aaf579 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:13.280081 20032 raft_consensus.cc:515] T 7292d04b411c415394e9d0df85aaf579 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:13.285212 20032 leader_election.cc:290] T 7292d04b411c415394e9d0df85aaf579 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:13.237473 19942 ts_tablet_manager.cc:1434] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.089s user 0.009s sys 0.000s
I20260925 13:56:13.287022 19942 tablet_bootstrap.cc:492] T d9455b98734147fb9e2d47c69513424a P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:13.289496 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7292d04b411c415394e9d0df85aaf579" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:13.292713 19773 leader_election.cc:343] T 7292d04b411c415394e9d0df85aaf579 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:13.294989 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7292d04b411c415394e9d0df85aaf579" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:13.297040 19959 ts_tablet_manager.cc:1929] T 28ac2912b94a4928967efb7a731c42b9 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:13.297382 19959 log.cc:1199] T 28ac2912b94a4928967efb7a731c42b9 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/28ac2912b94a4928967efb7a731c42b9
W20260925 13:56:13.297559 19773 leader_election.cc:343] T 7292d04b411c415394e9d0df85aaf579 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:13.298981 19773 leader_election.cc:304] T 7292d04b411c415394e9d0df85aaf579 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:13.304399 20090 raft_consensus.cc:2749] T 7292d04b411c415394e9d0df85aaf579 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:13.310874 19955 ts_tablet_manager.cc:1929] T 491b8b5c0f51408b882adee10ce80728 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:56:13.315984 19274 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_9786cd5857464d5f9008e6e630c41899" } new_table_name: "default.Table_3cf69a468c014c279a928211b6ff89a2" modify_external_catalogs: true
I20260925 13:56:13.339046 20090 raft_consensus.cc:493] T c1f15b9de572479aae6c2e644da3392a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:13.339504 20090 raft_consensus.cc:515] T c1f15b9de572479aae6c2e644da3392a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:13.341288 20090 leader_election.cc:290] T c1f15b9de572479aae6c2e644da3392a P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:13.341809 19275 master_service.cc:960] ReplaceTablet: received request to replace tablet 51a4c21e06ca4b5b971cf8f04ffa52a9 from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:13.345541 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f15b9de572479aae6c2e644da3392a" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:13.348280 19955 log.cc:1199] T 491b8b5c0f51408b882adee10ce80728 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/491b8b5c0f51408b882adee10ce80728
I20260925 13:56:13.349145 19955 ts_tablet_manager.cc:1950] T 491b8b5c0f51408b882adee10ce80728 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:13.350457 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet 491b8b5c0f51408b882adee10ce80728 (table default.Table_81fefa73a0904acb9b300de03fb48254 [id=dadfbd02a0a640949b1dc46c01db7bc8]) successfully deleted
I20260925 13:56:13.312367 20011 ts_tablet_manager.cc:1916] T ff3d09046b854fdaab2f95feff589ac1 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:13.314260 20101 raft_consensus.cc:2749] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:13.314340 19673 tablet_service.cc:1511] Processing CreateTablet for tablet 608bed5b04394bfca1776b5d6c502df1 (DEFAULT_TABLE table=default.Table_3b49fb893abb48ac8646283360e8e681 [id=9621507077d04649a58715b250fcef05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:13.354414 19673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 608bed5b04394bfca1776b5d6c502df1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:13.344614 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f15b9de572479aae6c2e644da3392a" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:13.360435 20101 consensus_queue.cc:1048] T 6b52ed579b19479686247d3ecba9489a P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 13:56:13.360532 19773 leader_election.cc:343] T c1f15b9de572479aae6c2e644da3392a P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:13.360852 19279 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_7cc40850827b4347a71d511dc1aed755" } new_table_name: "default.Table_bd52a86b8ff14bdbb06fcf88a60b1e82" modify_external_catalogs: true
I20260925 13:56:13.361723 19535 tablet_service.cc:1511] Processing CreateTablet for tablet 608bed5b04394bfca1776b5d6c502df1 (DEFAULT_TABLE table=default.Table_3b49fb893abb48ac8646283360e8e681 [id=9621507077d04649a58715b250fcef05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:13.362829 19535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 608bed5b04394bfca1776b5d6c502df1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:13.363067 19938 tablet_bootstrap.cc:492] T dc14c37aa393407a87af0d0eb8b88d82 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:13.363471 19938 ts_tablet_manager.cc:1403] T dc14c37aa393407a87af0d0eb8b88d82 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.148s user 0.013s sys 0.000s
I20260925 13:56:13.365747 19938 raft_consensus.cc:359] T dc14c37aa393407a87af0d0eb8b88d82 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:13.366321 19938 raft_consensus.cc:385] T dc14c37aa393407a87af0d0eb8b88d82 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:13.366552 19938 raft_consensus.cc:740] T dc14c37aa393407a87af0d0eb8b88d82 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:13.335229 19820 tablet_service.cc:1511] Processing CreateTablet for tablet 608bed5b04394bfca1776b5d6c502df1 (DEFAULT_TABLE table=default.Table_3b49fb893abb48ac8646283360e8e681 [id=9621507077d04649a58715b250fcef05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:13.371764 19820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 608bed5b04394bfca1776b5d6c502df1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:13.374172 19942 tablet_bootstrap.cc:654] T d9455b98734147fb9e2d47c69513424a P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:13.380610 19939 tablet_bootstrap.cc:654] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:13.298126 19959 ts_tablet_manager.cc:1950] T 28ac2912b94a4928967efb7a731c42b9 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
W20260925 13:56:13.395872 19773 leader_election.cc:343] T c1f15b9de572479aae6c2e644da3392a P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:13.396325 19773 leader_election.cc:304] T c1f15b9de572479aae6c2e644da3392a P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:13.367120 19938 consensus_queue.cc:260] T dc14c37aa393407a87af0d0eb8b88d82 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:13.397030 20032 raft_consensus.cc:2749] T c1f15b9de572479aae6c2e644da3392a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:13.400050 20101 raft_consensus.cc:493] T 293b017a83cc464a9f023a0923310796 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:13.400462 20101 raft_consensus.cc:515] T 293b017a83cc464a9f023a0923310796 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:13.402287 20101 leader_election.cc:290] T 293b017a83cc464a9f023a0923310796 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:13.404130 19959 tablet_replica.cc:333] T 293b017a83cc464a9f023a0923310796 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:13.405769 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 28ac2912b94a4928967efb7a731c42b9 (table default.Table_52f57b4f58b44f28989ffd9a4afe2eac [id=56afa81b40064727a7facfe3ccb22571]) successfully deleted
I20260925 13:56:13.413197 19272 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_d3801e388b7b4b4d95188dc60386cce9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:13.414716 19272 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d3801e388b7b4b4d95188dc60386cce9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:13.415879 19938 ts_tablet_manager.cc:1434] T dc14c37aa393407a87af0d0eb8b88d82 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.052s user 0.006s sys 0.000s
I20260925 13:56:13.416622 19959 raft_consensus.cc:2243] T 293b017a83cc464a9f023a0923310796 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus shutting down.
W20260925 13:56:13.417062 19629 leader_election.cc:343] T 293b017a83cc464a9f023a0923310796 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Not found: Tablet not found: 293b017a83cc464a9f023a0923310796
I20260925 13:56:13.417310 19959 raft_consensus.cc:2272] T 293b017a83cc464a9f023a0923310796 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:13.417675 20100 consensus_queue.cc:1048] T 6b52ed579b19479686247d3ecba9489a P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:13.418861 19959 ts_tablet_manager.cc:1916] T 293b017a83cc464a9f023a0923310796 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:13.421913 19839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293b017a83cc464a9f023a0923310796" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
W20260925 13:56:13.424993 19631 leader_election.cc:343] T 293b017a83cc464a9f023a0923310796 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Illegal state: Tablet not RUNNING: STOPPED
I20260925 13:56:13.425390 19631 leader_election.cc:304] T 293b017a83cc464a9f023a0923310796 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:13.426357 20100 raft_consensus.cc:2749] T 293b017a83cc464a9f023a0923310796 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:13.427268 19942 tablet_bootstrap.cc:492] T d9455b98734147fb9e2d47c69513424a P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:13.427699 19278 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet f0cdc99cde2f4dabb819e3e171c5d8d9
I20260925 13:56:13.427814 19942 ts_tablet_manager.cc:1403] T d9455b98734147fb9e2d47c69513424a P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.141s user 0.011s sys 0.000s
I20260925 13:56:13.429184 19278 catalog_manager.cc:6934] ReplaceTablet: tablet f0cdc99cde2f4dabb819e3e171c5d8d9 deleted and replaced by tablet f514fa52e7524cd3ba07f3f65de532d4
I20260925 13:56:13.430939 19534 tablet_service.cc:1558] Processing DeleteTablet for tablet f0cdc99cde2f4dabb819e3e171c5d8d9 with delete_type TABLET_DATA_DELETED (replaced by tablet f514fa52e7524cd3ba07f3f65de532d4) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:13.431813 19955 tablet_replica.cc:333] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:13.432051 19817 tablet_service.cc:1558] Processing DeleteTablet for tablet f0cdc99cde2f4dabb819e3e171c5d8d9 with delete_type TABLET_DATA_DELETED (replaced by tablet f514fa52e7524cd3ba07f3f65de532d4) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:13.432492 19955 raft_consensus.cc:2243] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:13.433071 19955 raft_consensus.cc:2272] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:13.433534 19676 tablet_service.cc:1558] Processing DeleteTablet for tablet f0cdc99cde2f4dabb819e3e171c5d8d9 with delete_type TABLET_DATA_DELETED (replaced by tablet f514fa52e7524cd3ba07f3f65de532d4) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:13.435635 19955 ts_tablet_manager.cc:1916] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:13.483047 19278 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_824449a97a864e3dbbf06e2ad678edc5" } modify_external_catalogs: true
I20260925 13:56:13.483636 19278 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_824449a97a864e3dbbf06e2ad678edc5" } modify_external_catalogs: true
I20260925 13:56:13.489805 19273 catalog_manager.cc:6128] Not sending DeleteTablet requests; no consensus state for tablet 15e580a6be094a769a9eebbd4e2bc4e7
W20260925 13:56:13.501030 19264 catalog_manager.cc:5136] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): alter failed for tablet 293b017a83cc464a9f023a0923310796 (table default.Table_da4fc82971544f509fcd31edfea882da [id=d2bf5d31983a47d897e93ace83ecf026]),no further retry: Not found: Tablet not found: 293b017a83cc464a9f023a0923310796
I20260925 13:56:13.503024 19534 tablet_service.cc:1558] Processing DeleteTablet for tablet 3b10ecb0e6064d42a2521c4c4e91c028 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:11 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:13.504761 19534 tablet_service.cc:1558] Processing DeleteTablet for tablet ff3d09046b854fdaab2f95feff589ac1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:11 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:13.507057 19534 tablet_service.cc:1558] Processing DeleteTablet for tablet d9455b98734147fb9e2d47c69513424a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:11 UTC) from {username='slave'} at 127.0.0.1:40018
W20260925 13:56:13.508174 19277 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.193:37907 (request call id 22) took 889 ms (client timeout 1000 ms). Trace:
W20260925 13:56:13.513111 19277 rpcz_store.cc:269] 0925 13:56:12.618327 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:12.618426 (+ 99us) service_pool.cc:224] Handling call
0925 13:56:13.508115 (+889689us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:12.908035 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:12.908399 (+ 364us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:12.908423 (+ 24us) write_op.cc:435] Acquired schema lock
0925 13:56:12.908437 (+ 14us) tablet.cc:662] Decoding operations
0925 13:56:12.908677 (+ 240us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:12.908710 (+ 33us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:12.908731 (+ 21us) tablet.cc:685] Acquiring locks for 1 operations
0925 13:56:12.908868 (+ 137us) tablet.cc:701] Row locks acquired
0925 13:56:12.908882 (+ 14us) write_op.cc:260] PREPARE: finished
0925 13:56:12.908997 (+ 115us) write_op.cc:270] Start()
0925 13:56:12.909081 (+ 84us) write_op.cc:276] Timestamp: P: 1790344572908957 usec, L: 0
0925 13:56:12.909100 (+ 19us) op_driver.cc:348] REPLICATION: starting
0925 13:56:12.909532 (+ 432us) log.cc:844] Serialized 535 byte log entry
0925 13:56:12.948339 (+ 38807us) op_driver.cc:464] REPLICATION: finished
0925 13:56:12.949830 (+ 1491us) write_op.cc:301] APPLY: starting
0925 13:56:12.949885 (+ 55us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:12.949988 (+ 103us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:12.950084 (+ 96us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:12.950210 (+ 126us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:12.950329 (+ 119us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 13:56:12.950368 (+ 39us) write_op.cc:312] APPLY: finished
0925 13:56:12.950513 (+ 145us) log.cc:844] Serialized 22 byte log entry
0925 13:56:12.950625 (+ 112us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:12.950684 (+ 59us) write_op.cc:454] Released schema lock
0925 13:56:12.950713 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0925 13:56:12.950935 (+ 222us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:12.950960 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":10,"child_traces":[["op",{"apply.queue_time_us":1352,"apply.run_cpu_time_us":1054,"apply.run_wall_time_us":1054,"num_ops":1,"prepare.queue_time_us":2616,"prepare.run_cpu_time_us":1746,"prepare.run_wall_time_us":1957,"raft.queue_time_us":91,"raft.run_cpu_time_us":10,"raft.run_wall_time_us":11,"replication_time_us":39148}]]}
I20260925 13:56:13.519481 19938 tablet_bootstrap.cc:492] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:13.525024 19939 tablet_bootstrap.cc:492] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:13.525432 19939 ts_tablet_manager.cc:1403] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.345s user 0.009s sys 0.001s
I20260925 13:56:13.529026 19938 tablet_bootstrap.cc:654] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:13.535173 19942 raft_consensus.cc:359] T d9455b98734147fb9e2d47c69513424a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:13.535934 19939 raft_consensus.cc:359] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:13.536291 19942 raft_consensus.cc:385] T d9455b98734147fb9e2d47c69513424a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:13.536530 19939 raft_consensus.cc:385] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:13.536646 19942 raft_consensus.cc:740] T d9455b98734147fb9e2d47c69513424a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:13.536785 19939 raft_consensus.cc:740] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:13.537372 19939 consensus_queue.cc:260] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:13.537585 19942 consensus_queue.cc:260] T d9455b98734147fb9e2d47c69513424a P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:13.549543 19277 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_b194d76d005f44b7a93ec10df4e3c59c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:13.552076 19939 ts_tablet_manager.cc:1434] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.026s user 0.006s sys 0.001s
I20260925 13:56:13.552922 19939 tablet_bootstrap.cc:492] T c1f15b9de572479aae6c2e644da3392a P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
W20260925 13:56:13.557948 19277 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b194d76d005f44b7a93ec10df4e3c59c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:13.575021 19955 ts_tablet_manager.cc:1929] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:13.563889 19280 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_d647554488064f3ea60ad9b397cabe30"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:13.576792 19280 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d647554488064f3ea60ad9b397cabe30 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:13.571578 19275 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 51a4c21e06ca4b5b971cf8f04ffa52a9
I20260925 13:56:13.580660 19275 catalog_manager.cc:6934] ReplaceTablet: tablet 51a4c21e06ca4b5b971cf8f04ffa52a9 deleted and replaced by tablet 2a1f34dba13143b097a38bab3782c404
I20260925 13:56:13.584664 19673 tablet_service.cc:1558] Processing DeleteTablet for tablet 51a4c21e06ca4b5b971cf8f04ffa52a9 with delete_type TABLET_DATA_DELETED (replaced by tablet 2a1f34dba13143b097a38bab3782c404) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:13.596514 20011 ts_tablet_manager.cc:1929] T ff3d09046b854fdaab2f95feff589ac1 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:13.596941 20011 log.cc:1199] T ff3d09046b854fdaab2f95feff589ac1 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/ff3d09046b854fdaab2f95feff589ac1
I20260925 13:56:13.597594 20011 ts_tablet_manager.cc:1950] T ff3d09046b854fdaab2f95feff589ac1 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
W20260925 13:56:13.598094 19279 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 26644768f516412f8c95d68a76bb99c5 has no consensus state
I20260925 13:56:13.610010 19942 ts_tablet_manager.cc:1434] T d9455b98734147fb9e2d47c69513424a P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.182s user 0.006s sys 0.000s
I20260925 13:56:13.613090 19942 tablet_bootstrap.cc:492] T f4429db544534f758883f2fae184ea6d P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:13.557971 19939 tablet_bootstrap.cc:654] T c1f15b9de572479aae6c2e644da3392a P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:13.618801 19955 log.cc:1199] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/f0cdc99cde2f4dabb819e3e171c5d8d9
I20260925 13:56:13.619591 19955 ts_tablet_manager.cc:1950] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:13.583256 19534 tablet_service.cc:1558] Processing DeleteTablet for tablet 51a4c21e06ca4b5b971cf8f04ffa52a9 with delete_type TABLET_DATA_DELETED (replaced by tablet 2a1f34dba13143b097a38bab3782c404) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:13.622457 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet f0cdc99cde2f4dabb819e3e171c5d8d9 (table default.Table_5d6ee0b4200c486ca1fdec8349603746 [id=7e7ee975d4bb457eb3cc4632e12f3cd5]) successfully deleted
I20260925 13:56:13.623570 19955 tablet_replica.cc:333] T 3b10ecb0e6064d42a2521c4c4e91c028 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:13.624468 19955 raft_consensus.cc:2243] T 3b10ecb0e6064d42a2521c4c4e91c028 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:13.625167 19955 pending_rounds.cc:70] T 3b10ecb0e6064d42a2521c4c4e91c028 P 8f578869bc084a5d8a9a69f143954eb3: Trying to abort 1 pending ops.
I20260925 13:56:13.625397 19955 pending_rounds.cc:77] T 3b10ecb0e6064d42a2521c4c4e91c028 P 8f578869bc084a5d8a9a69f143954eb3: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251369048354816 op_type: NO_OP noop_request { }
I20260925 13:56:13.625730 19955 raft_consensus.cc:2889] T 3b10ecb0e6064d42a2521c4c4e91c028 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:56:13.626044 19955 raft_consensus.cc:2272] T 3b10ecb0e6064d42a2521c4c4e91c028 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 13:56:13.580870 19773 consensus_peers.cc:597] T 491b8b5c0f51408b882adee10ce80728 P f790493ba7ad403993b678f2df2b55be -> Peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Couldn't send request to peer 8f578869bc084a5d8a9a69f143954eb3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 491b8b5c0f51408b882adee10ce80728. This is attempt 1: this message will repeat every 5th retry.
W20260925 13:56:13.627560 19773 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.193:38269, user_credentials={real_user=slave}} blocked reactor thread for 47015.5us
I20260925 13:56:13.628718 19820 tablet_service.cc:1558] Processing DeleteTablet for tablet 51a4c21e06ca4b5b971cf8f04ffa52a9 with delete_type TABLET_DATA_DELETED (replaced by tablet 2a1f34dba13143b097a38bab3782c404) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:13.630155 19272 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_02ff478a0f98431cacbe0cb5cfba6e8c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:13.631927 19272 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_02ff478a0f98431cacbe0cb5cfba6e8c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:13.638115 19942 tablet_bootstrap.cc:654] T f4429db544534f758883f2fae184ea6d P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:13.638980 20068 consensus_queue.cc:1048] T b0a293e0094a4d9e937af6454e1ea16c P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:13.643517 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 31) took 1821 ms. Trace:
I20260925 13:56:13.643855 20011 rpcz_store.cc:276] 0925 13:56:11.822650 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:11.849639 (+ 26989us) service_pool.cc:224] Handling call
0925 13:56:13.133940 (+1284301us) ts_tablet_manager.cc:1151] Deleting tablet ff3d09046b854fdaab2f95feff589ac1
0925 13:56:13.133988 (+ 48us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:13.398430 (+264442us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:13.596383 (+197953us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:13.643449 (+ 47066us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1216991}
I20260925 13:56:13.645299 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet ff3d09046b854fdaab2f95feff589ac1 (table default.Table_8ac29bbd5011471a9b60f334fc78a3d3 [id=689d277ceae34fc0bb1f2ecd43295722]) successfully deleted
I20260925 13:56:13.646279 20011 tablet_replica.cc:333] T d9455b98734147fb9e2d47c69513424a P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:13.651327 19271 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 1513c0048bd244028d3dfa8d118bc14e (table default.Table_74b6076b22674a31bc58fef71cc9bfb1 [id=d81cf07e5f9a49d4bbf5aceda3da5a25]). Expected version 1 got 0
I20260925 13:56:13.652410 19271 catalog_manager.cc:5838] T f4429db544534f758883f2fae184ea6d P f790493ba7ad403993b678f2df2b55be reported cstate change: term changed from 0 to 1, leader changed from <none> to f790493ba7ad403993b678f2df2b55be (127.18.182.195). New cstate: current_term: 1 leader_uuid: "f790493ba7ad403993b678f2df2b55be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:13.654268 19271 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 18a6d1c4eb7948149e709a380da50d93 (table default.Table_abc00b1f4c4c455e8086a46c395f2656 [id=6c0ec9465714408da93ff6e47827eb48]). Expected version 1 got 0
I20260925 13:56:13.664505 19939 tablet_bootstrap.cc:492] T c1f15b9de572479aae6c2e644da3392a P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:13.664891 19939 ts_tablet_manager.cc:1403] T c1f15b9de572479aae6c2e644da3392a P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.112s user 0.012s sys 0.003s
I20260925 13:56:13.667081 19939 raft_consensus.cc:359] T c1f15b9de572479aae6c2e644da3392a P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:13.667805 19939 raft_consensus.cc:385] T c1f15b9de572479aae6c2e644da3392a P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:13.668045 19939 raft_consensus.cc:740] T c1f15b9de572479aae6c2e644da3392a P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:13.668226 20011 raft_consensus.cc:2243] T d9455b98734147fb9e2d47c69513424a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:13.668794 20011 raft_consensus.cc:2272] T d9455b98734147fb9e2d47c69513424a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:13.668596 19939 consensus_queue.cc:260] T c1f15b9de572479aae6c2e644da3392a P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:13.671105 20011 ts_tablet_manager.cc:1916] T d9455b98734147fb9e2d47c69513424a P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:13.685204 19279 master_service.cc:960] ReplaceTablet: received request to replace tablet f51c5ad86ce64891a6393c1dc8d414e5 from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:13.689095 19955 ts_tablet_manager.cc:1916] T 3b10ecb0e6064d42a2521c4c4e91c028 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:13.660068 19959 ts_tablet_manager.cc:1929] T 293b017a83cc464a9f023a0923310796 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:13.689875 19959 log.cc:1199] T 293b017a83cc464a9f023a0923310796 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/293b017a83cc464a9f023a0923310796
I20260925 13:56:13.690513 19959 ts_tablet_manager.cc:1950] T 293b017a83cc464a9f023a0923310796 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:13.663486 19938 tablet_bootstrap.cc:492] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:13.691733 19938 ts_tablet_manager.cc:1403] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.172s user 0.010s sys 0.000s
W20260925 13:56:13.674090 19773 consensus_peers.cc:597] T f4429db544534f758883f2fae184ea6d P f790493ba7ad403993b678f2df2b55be -> Peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Couldn't send request to peer 7c1b52895a14482fb54a6758f5054f8f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:13.694837 19938 raft_consensus.cc:359] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:13.695429 19938 raft_consensus.cc:385] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:13.695655 19938 raft_consensus.cc:740] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:13.696300 19938 consensus_queue.cc:260] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:13.705546 19280 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_8290acbcb9d84a48aad169c54251bc19" } modify_external_catalogs: true
I20260925 13:56:13.706048 19280 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_8290acbcb9d84a48aad169c54251bc19" } modify_external_catalogs: true
I20260925 13:56:13.698225 19939 ts_tablet_manager.cc:1434] T c1f15b9de572479aae6c2e644da3392a P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.033s user 0.005s sys 0.000s
I20260925 13:56:13.719383 19939 tablet_bootstrap.cc:492] T 18a6d1c4eb7948149e709a380da50d93 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:13.722198 19959 tablet_replica.cc:333] T 491b8b5c0f51408b882adee10ce80728 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:13.722898 19959 raft_consensus.cc:2243] T 491b8b5c0f51408b882adee10ce80728 P f790493ba7ad403993b678f2df2b55be [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:13.723701 19959 raft_consensus.cc:2272] T 491b8b5c0f51408b882adee10ce80728 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:13.725368 19959 ts_tablet_manager.cc:1916] T 491b8b5c0f51408b882adee10ce80728 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 13:56:13.740000 19488 consensus_peers.cc:597] T b0a293e0094a4d9e937af6454e1ea16c P 8f578869bc084a5d8a9a69f143954eb3 -> Peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Couldn't send request to peer 7c1b52895a14482fb54a6758f5054f8f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:13.740182 19939 tablet_bootstrap.cc:654] T 18a6d1c4eb7948149e709a380da50d93 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:13.741689 19942 tablet_bootstrap.cc:492] T f4429db544534f758883f2fae184ea6d P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:13.742187 19942 ts_tablet_manager.cc:1403] T f4429db544534f758883f2fae184ea6d P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.129s user 0.002s sys 0.008s
I20260925 13:56:13.744727 19938 ts_tablet_manager.cc:1434] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.053s user 0.003s sys 0.002s
I20260925 13:56:13.759003 20032 raft_consensus.cc:493] T 980d1264c0ea45f3b2b762509e0b2c77 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:13.759585 20032 raft_consensus.cc:515] T 980d1264c0ea45f3b2b762509e0b2c77 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:13.761615 20032 leader_election.cc:290] T 980d1264c0ea45f3b2b762509e0b2c77 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:13.763260 19696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "980d1264c0ea45f3b2b762509e0b2c77" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:13.764798 19773 leader_election.cc:343] T 980d1264c0ea45f3b2b762509e0b2c77 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:13.777784 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "980d1264c0ea45f3b2b762509e0b2c77" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
W20260925 13:56:13.779218 19773 leader_election.cc:343] T 980d1264c0ea45f3b2b762509e0b2c77 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:13.779594 19939 tablet_bootstrap.cc:492] T 18a6d1c4eb7948149e709a380da50d93 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:13.779635 19773 leader_election.cc:304] T 980d1264c0ea45f3b2b762509e0b2c77 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:13.779975 19939 ts_tablet_manager.cc:1403] T 18a6d1c4eb7948149e709a380da50d93 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.061s user 0.014s sys 0.002s
I20260925 13:56:13.780303 20032 raft_consensus.cc:2749] T 980d1264c0ea45f3b2b762509e0b2c77 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:13.781075 19938 tablet_bootstrap.cc:492] T b36aebfe984f454ea910e84645c4ac7d P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:13.784117 19942 raft_consensus.cc:359] T f4429db544534f758883f2fae184ea6d P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:13.792979 19939 raft_consensus.cc:359] T 18a6d1c4eb7948149e709a380da50d93 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:13.793606 19939 raft_consensus.cc:385] T 18a6d1c4eb7948149e709a380da50d93 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:13.793730 19942 raft_consensus.cc:385] T f4429db544534f758883f2fae184ea6d P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:13.793877 19939 raft_consensus.cc:740] T 18a6d1c4eb7948149e709a380da50d93 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:13.794100 19942 raft_consensus.cc:740] T f4429db544534f758883f2fae184ea6d P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:13.794577 19939 consensus_queue.cc:260] T 18a6d1c4eb7948149e709a380da50d93 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:13.799929 19939 ts_tablet_manager.cc:1434] T 18a6d1c4eb7948149e709a380da50d93 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20260925 13:56:13.794814 19942 consensus_queue.cc:260] T f4429db544534f758883f2fae184ea6d P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:13.803845 20032 raft_consensus.cc:493] T dc14c37aa393407a87af0d0eb8b88d82 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:13.804296 20032 raft_consensus.cc:515] T dc14c37aa393407a87af0d0eb8b88d82 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:13.806224 20032 leader_election.cc:290] T dc14c37aa393407a87af0d0eb8b88d82 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:13.807832 19279 master_service.cc:960] ReplaceTablet: received request to replace tablet 00b500182bb04a72b80cfa9103e85fa9 from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:13.808319 19939 tablet_bootstrap.cc:492] T 47521fb74f9c40ee999a7f704aa551c9 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:13.811990 19272 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_38131969dbb24c24bc8b3e4f845e48d9" } new_table_name: "default.Table_d0dadd9b31b143fd8dd5adcb7f752c5f" modify_external_catalogs: true
I20260925 13:56:13.815990 19939 tablet_bootstrap.cc:654] T 47521fb74f9c40ee999a7f704aa551c9 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:13.820499 19277 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_03fdc4ff0d534d8c9db1aba08970a310"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:13.822081 19277 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_03fdc4ff0d534d8c9db1aba08970a310 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:13.822785 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc14c37aa393407a87af0d0eb8b88d82" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:13.824054 19696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc14c37aa393407a87af0d0eb8b88d82" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:13.824653 19773 leader_election.cc:343] T dc14c37aa393407a87af0d0eb8b88d82 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:13.825521 19773 leader_election.cc:343] T dc14c37aa393407a87af0d0eb8b88d82 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:13.825979 19773 leader_election.cc:304] T dc14c37aa393407a87af0d0eb8b88d82 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:13.827083 20032 raft_consensus.cc:2749] T dc14c37aa393407a87af0d0eb8b88d82 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:13.831542 19942 ts_tablet_manager.cc:1434] T f4429db544534f758883f2fae184ea6d P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.089s user 0.006s sys 0.000s
I20260925 13:56:13.833503 19280 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 980d1264c0ea45f3b2b762509e0b2c77
I20260925 13:56:13.843709 19820 tablet_service.cc:1558] Processing DeleteTablet for tablet 980d1264c0ea45f3b2b762509e0b2c77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:13 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:13.846837 20117 raft_consensus.cc:493] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:13.847481 19673 tablet_service.cc:1558] Processing DeleteTablet for tablet 980d1264c0ea45f3b2b762509e0b2c77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:13 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:13.847309 20117 raft_consensus.cc:515] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:13.849329 20117 leader_election.cc:290] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:13.851527 19696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1513c0048bd244028d3dfa8d118bc14e" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:13.852355 19840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1513c0048bd244028d3dfa8d118bc14e" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
W20260925 13:56:13.852869 19488 leader_election.cc:343] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:13.852988 19840 raft_consensus.cc:2468] T 1513c0048bd244028d3dfa8d118bc14e P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:13.854278 19490 leader_election.cc:304] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:13.854676 19534 tablet_service.cc:1558] Processing DeleteTablet for tablet 980d1264c0ea45f3b2b762509e0b2c77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:13 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:13.855508 20117 raft_consensus.cc:2804] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:13.855865 20117 raft_consensus.cc:493] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:13.856199 20117 raft_consensus.cc:3060] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
W20260925 13:56:13.856554 19272 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3e6c8b9bbf4549b3a8685f886973ed9c has no consensus state
I20260925 13:56:13.863806 20117 raft_consensus.cc:515] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:13.867443 19938 tablet_bootstrap.cc:654] T b36aebfe984f454ea910e84645c4ac7d P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:13.868073 19840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1513c0048bd244028d3dfa8d118bc14e" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:13.868587 19840 raft_consensus.cc:3060] T 1513c0048bd244028d3dfa8d118bc14e P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:13.868686 19959 ts_tablet_manager.cc:1929] T 491b8b5c0f51408b882adee10ce80728 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:56:13.868994 19959 log.cc:1199] T 491b8b5c0f51408b882adee10ce80728 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/491b8b5c0f51408b882adee10ce80728
I20260925 13:56:13.869653 19959 ts_tablet_manager.cc:1950] T 491b8b5c0f51408b882adee10ce80728 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:13.870450 20100 raft_consensus.cc:493] T 28ac2912b94a4928967efb7a731c42b9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:13.870851 20100 raft_consensus.cc:515] T 28ac2912b94a4928967efb7a731c42b9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:13.872668 20100 leader_election.cc:290] T 28ac2912b94a4928967efb7a731c42b9 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:13.873857 19942 tablet_bootstrap.cc:492] T b0a293e0094a4d9e937af6454e1ea16c P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
W20260925 13:56:13.877202 19631 leader_election.cc:343] T 28ac2912b94a4928967efb7a731c42b9 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 28ac2912b94a4928967efb7a731c42b9
W20260925 13:56:13.877964 19629 leader_election.cc:343] T 28ac2912b94a4928967efb7a731c42b9 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Not found: Tablet not found: 28ac2912b94a4928967efb7a731c42b9
I20260925 13:56:13.878324 19629 leader_election.cc:304] T 28ac2912b94a4928967efb7a731c42b9 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:13.879292 20100 raft_consensus.cc:2749] T 28ac2912b94a4928967efb7a731c42b9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:13.879978 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 491b8b5c0f51408b882adee10ce80728 (table default.Table_81fefa73a0904acb9b300de03fb48254 [id=dadfbd02a0a640949b1dc46c01db7bc8]) successfully deleted
W20260925 13:56:13.880796 19266 catalog_manager.cc:5030] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): delete failed for tablet d9455b98734147fb9e2d47c69513424a because the tablet was not found. No further retry: Not found: Tablet not found: d9455b98734147fb9e2d47c69513424a
I20260925 13:56:13.882807 19840 raft_consensus.cc:2468] T 1513c0048bd244028d3dfa8d118bc14e P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:13.884096 19490 leader_election.cc:304] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:13.884974 20068 raft_consensus.cc:2804] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:13.887840 19959 tablet_replica.cc:333] T f0cdc99cde2f4dabb819e3e171c5d8d9 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:13.888893 19959 raft_consensus.cc:2243] T f0cdc99cde2f4dabb819e3e171c5d8d9 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:13.889472 19959 raft_consensus.cc:2272] T f0cdc99cde2f4dabb819e3e171c5d8d9 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:13.891956 19959 ts_tablet_manager.cc:1916] T f0cdc99cde2f4dabb819e3e171c5d8d9 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:13.893523 19939 tablet_bootstrap.cc:492] T 47521fb74f9c40ee999a7f704aa551c9 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:13.893939 19939 ts_tablet_manager.cc:1403] T 47521fb74f9c40ee999a7f704aa551c9 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.086s user 0.005s sys 0.023s
I20260925 13:56:13.895309 19277 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_86a5b8762a65441aacd09147e7fb78fd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:13.895888 20068 raft_consensus.cc:697] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Running, Role: LEADER
I20260925 13:56:13.896731 20068 consensus_queue.cc:237] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
W20260925 13:56:13.897328 19277 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_86a5b8762a65441aacd09147e7fb78fd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:13.901499 19939 raft_consensus.cc:359] T 47521fb74f9c40ee999a7f704aa551c9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:13.904431 19939 raft_consensus.cc:385] T 47521fb74f9c40ee999a7f704aa551c9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:13.904696 19939 raft_consensus.cc:740] T 47521fb74f9c40ee999a7f704aa551c9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:13.906852 19696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1513c0048bd244028d3dfa8d118bc14e" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
W20260925 13:56:13.908339 19488 leader_election.cc:343] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:13.908906 20117 leader_election.cc:290] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:13.893554 19955 ts_tablet_manager.cc:1929] T 3b10ecb0e6064d42a2521c4c4e91c028 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:13.919842 19955 log.cc:1199] T 3b10ecb0e6064d42a2521c4c4e91c028 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/3b10ecb0e6064d42a2521c4c4e91c028
I20260925 13:56:13.921419 20068 raft_consensus.cc:493] T 00b500182bb04a72b80cfa9103e85fa9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:13.921826 20068 raft_consensus.cc:515] T 00b500182bb04a72b80cfa9103e85fa9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:13.922225 20011 ts_tablet_manager.cc:1929] T d9455b98734147fb9e2d47c69513424a P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:13.922987 20011 log.cc:1199] T d9455b98734147fb9e2d47c69513424a P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/d9455b98734147fb9e2d47c69513424a
I20260925 13:56:13.924952 19938 tablet_bootstrap.cc:492] T b36aebfe984f454ea910e84645c4ac7d P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
W20260925 13:56:13.925148 19490 leader_election.cc:343] T 00b500182bb04a72b80cfa9103e85fa9 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 00b500182bb04a72b80cfa9103e85fa9
I20260925 13:56:13.925407 19938 ts_tablet_manager.cc:1403] T b36aebfe984f454ea910e84645c4ac7d P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.145s user 0.006s sys 0.003s
I20260925 13:56:13.927806 19955 ts_tablet_manager.cc:1950] T 3b10ecb0e6064d42a2521c4c4e91c028 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:13.929594 20068 leader_election.cc:290] T 00b500182bb04a72b80cfa9103e85fa9 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:13.930202 19939 consensus_queue.cc:260] T 47521fb74f9c40ee999a7f704aa551c9 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:13.930378 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet 3b10ecb0e6064d42a2521c4c4e91c028 (table default.Table_f9ae699facd148ca8bae38d481d136df [id=7a20968885384e23a7a65f63337a67d3]) successfully deleted
I20260925 13:56:13.935854 19277 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_9896a33d94e44d76ada6ec8d00c87ad7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:13.937647 19277 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9896a33d94e44d76ada6ec8d00c87ad7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:13.939138 19939 ts_tablet_manager.cc:1434] T 47521fb74f9c40ee999a7f704aa551c9 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.045s user 0.004s sys 0.002s
I20260925 13:56:13.940877 19939 tablet_bootstrap.cc:492] T 7292d04b411c415394e9d0df85aaf579 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:13.950556 19939 tablet_bootstrap.cc:654] T 7292d04b411c415394e9d0df85aaf579 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:13.951817 19938 raft_consensus.cc:359] T b36aebfe984f454ea910e84645c4ac7d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:13.952754 19938 raft_consensus.cc:385] T b36aebfe984f454ea910e84645c4ac7d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:13.953125 19938 raft_consensus.cc:740] T b36aebfe984f454ea910e84645c4ac7d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:13.953979 19938 consensus_queue.cc:260] T b36aebfe984f454ea910e84645c4ac7d P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:13.956068 19938 ts_tablet_manager.cc:1434] T b36aebfe984f454ea910e84645c4ac7d P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.030s user 0.006s sys 0.000s
I20260925 13:56:13.927098 19696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b500182bb04a72b80cfa9103e85fa9" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:13.958002 20011 ts_tablet_manager.cc:1950] T d9455b98734147fb9e2d47c69513424a P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:13.960005 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 33) took 1594 ms. Trace:
I20260925 13:56:13.943876 19942 tablet_bootstrap.cc:654] T b0a293e0094a4d9e937af6454e1ea16c P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
W20260925 13:56:13.966013 19488 leader_election.cc:343] T 00b500182bb04a72b80cfa9103e85fa9 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:13.966202 19280 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_826761206dd0483f8703cfaf2154e80a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:13.966590 19488 leader_election.cc:304] T 00b500182bb04a72b80cfa9103e85fa9 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3; no voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:13.967519 20117 raft_consensus.cc:2749] T 00b500182bb04a72b80cfa9103e85fa9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:56:13.968010 19280 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_826761206dd0483f8703cfaf2154e80a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:13.970306 19939 tablet_bootstrap.cc:492] T 7292d04b411c415394e9d0df85aaf579 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:13.970691 19939 ts_tablet_manager.cc:1403] T 7292d04b411c415394e9d0df85aaf579 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.031s user 0.007s sys 0.007s
I20260925 13:56:13.973043 19939 raft_consensus.cc:359] T 7292d04b411c415394e9d0df85aaf579 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:13.973659 19939 raft_consensus.cc:385] T 7292d04b411c415394e9d0df85aaf579 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:13.973915 19939 raft_consensus.cc:740] T 7292d04b411c415394e9d0df85aaf579 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:13.974494 19939 consensus_queue.cc:260] T 7292d04b411c415394e9d0df85aaf579 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:13.975920 19264 catalog_manager.cc:5030] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): delete failed for tablet ff3d09046b854fdaab2f95feff589ac1 because the tablet was not found. No further retry: Not found: Tablet not found: ff3d09046b854fdaab2f95feff589ac1
W20260925 13:56:13.977361 19264 catalog_manager.cc:5030] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): delete failed for tablet d9455b98734147fb9e2d47c69513424a because the tablet was not found. No further retry: Not found: Tablet not found: d9455b98734147fb9e2d47c69513424a
I20260925 13:56:13.978132 19955 tablet_replica.cc:333] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:13.978365 20011 rpcz_store.cc:276] 0925 13:56:12.365509 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:12.397406 (+ 31897us) service_pool.cc:224] Handling call
0925 13:56:13.646093 (+1248687us) ts_tablet_manager.cc:1151] Deleting tablet d9455b98734147fb9e2d47c69513424a
0925 13:56:13.646145 (+ 52us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:13.730422 (+ 84277us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:13.922065 (+191643us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:13.959946 (+ 37881us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1183424}
I20260925 13:56:13.979919 19939 ts_tablet_manager.cc:1434] T 7292d04b411c415394e9d0df85aaf579 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.009s user 0.003s sys 0.004s
I20260925 13:56:13.983780 19939 tablet_bootstrap.cc:492] T 980d1264c0ea45f3b2b762509e0b2c77 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:13.992527 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet d9455b98734147fb9e2d47c69513424a (table default.Table_c1c831f6c04f4135801dcafbaf7a0843 [id=018679db079c42d2b70631ecb8f241a5]) successfully deleted
I20260925 13:56:13.980005 19955 raft_consensus.cc:2243] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:13.993314 19939 tablet_bootstrap.cc:654] T 980d1264c0ea45f3b2b762509e0b2c77 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:13.993841 19955 raft_consensus.cc:2272] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:13.994445 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 38) took 1067 ms. Trace:
I20260925 13:56:13.971096 19959 ts_tablet_manager.cc:1929] T f0cdc99cde2f4dabb819e3e171c5d8d9 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:13.996346 20011 rpcz_store.cc:276] 0925 13:56:12.926573 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:12.926702 (+ 129us) service_pool.cc:224] Handling call
0925 13:56:13.993717 (+1067015us) ts_tablet_manager.cc:1151] Deleting tablet 00b500182bb04a72b80cfa9103e85fa9
0925 13:56:13.994106 (+ 389us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:13.994377 (+ 271us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1066399}
W20260925 13:56:13.997570 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 00b500182bb04a72b80cfa9103e85fa9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 00b500182bb04a72b80cfa9103e85fa9 already in progress: creating tablet
I20260925 13:56:13.998293 19955 ts_tablet_manager.cc:1916] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:13.999115 19959 log.cc:1199] T f0cdc99cde2f4dabb819e3e171c5d8d9 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/f0cdc99cde2f4dabb819e3e171c5d8d9
I20260925 13:56:14.000133 19959 ts_tablet_manager.cc:1950] T f0cdc99cde2f4dabb819e3e171c5d8d9 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:14.000690 19942 tablet_bootstrap.cc:492] T b0a293e0094a4d9e937af6454e1ea16c P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:14.001077 19942 ts_tablet_manager.cc:1403] T b0a293e0094a4d9e937af6454e1ea16c P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.127s user 0.005s sys 0.003s
I20260925 13:56:14.003582 19959 tablet_replica.cc:333] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:14.003917 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet f0cdc99cde2f4dabb819e3e171c5d8d9 (table default.Table_5d6ee0b4200c486ca1fdec8349603746 [id=7e7ee975d4bb457eb3cc4632e12f3cd5]) successfully deleted
I20260925 13:56:14.003268 19942 raft_consensus.cc:359] T b0a293e0094a4d9e937af6454e1ea16c P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:14.006534 19959 raft_consensus.cc:2243] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P f790493ba7ad403993b678f2df2b55be [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:14.006572 19942 raft_consensus.cc:385] T b0a293e0094a4d9e937af6454e1ea16c P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:14.006878 19942 raft_consensus.cc:740] T b0a293e0094a4d9e937af6454e1ea16c P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:14.007421 19959 pending_rounds.cc:70] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P f790493ba7ad403993b678f2df2b55be: Trying to abort 1 pending ops.
I20260925 13:56:14.007640 19959 pending_rounds.cc:77] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P f790493ba7ad403993b678f2df2b55be: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251371419066368 op_type: NO_OP noop_request { }
I20260925 13:56:14.008065 19959 raft_consensus.cc:2889] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P f790493ba7ad403993b678f2df2b55be [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:56:13.996037 19938 tablet_bootstrap.cc:492] T bae8c5b6e1014731901bc3f336104784 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:14.008430 19959 raft_consensus.cc:2272] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:14.017390 19959 ts_tablet_manager.cc:1916] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:14.019678 19942 consensus_queue.cc:260] T b0a293e0094a4d9e937af6454e1ea16c P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:14.022861 19938 tablet_bootstrap.cc:654] T bae8c5b6e1014731901bc3f336104784 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:14.005791 20011 tablet_replica.cc:333] T 3b10ecb0e6064d42a2521c4c4e91c028 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:14.042176 20032 consensus_queue.cc:1048] T f4429db544534f758883f2fae184ea6d P f790493ba7ad403993b678f2df2b55be [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:14.052039 19275 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_374792fc278744f5ad6b9c1818b7a370" } modify_external_catalogs: true
I20260925 13:56:14.052613 19275 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_374792fc278744f5ad6b9c1818b7a370" } modify_external_catalogs: true
I20260925 13:56:14.059891 19280 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_38f7207982be43189299b739031aa942" } new_table_name: "default.Table_99593e98f3394811b4c91e234185cbef" modify_external_catalogs: true
I20260925 13:56:14.075657 19273 catalog_manager.cc:5838] T b0a293e0094a4d9e937af6454e1ea16c P 8f578869bc084a5d8a9a69f143954eb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "8f578869bc084a5d8a9a69f143954eb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:14.077741 19273 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet b0a293e0094a4d9e937af6454e1ea16c (table default.Table_256401fd998c430db7f494657ef8e75d [id=b673ac6542c5403fbf0fad16c4917dfd]). Expected version 1 got 0
I20260925 13:56:14.083693 19534 tablet_service.cc:1511] Processing CreateTablet for tablet 4d3dbd9796564472b44a31ccaf56d898 (DEFAULT_TABLE table=default.Table_1aa8069da6504812a85380a42ca07583 [id=a6d1215ba3be4fca903af9b17566b4a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:14.085043 19534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d3dbd9796564472b44a31ccaf56d898. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:14.085132 19673 tablet_service.cc:1511] Processing CreateTablet for tablet 4d3dbd9796564472b44a31ccaf56d898 (DEFAULT_TABLE table=default.Table_1aa8069da6504812a85380a42ca07583 [id=a6d1215ba3be4fca903af9b17566b4a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:14.086374 19673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d3dbd9796564472b44a31ccaf56d898. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:14.098315 19535 tablet_service.cc:1511] Processing CreateTablet for tablet f1c6e2137700448a8ebf859a9ca9085d (DEFAULT_TABLE table=default.Table_4559d216bb6041bdba82905a01b2412a [id=5bf634ef72d243b1bfcafc4efe14f28a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:14.099268 19535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1c6e2137700448a8ebf859a9ca9085d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:14.103413 19820 tablet_service.cc:1511] Processing CreateTablet for tablet 4d3dbd9796564472b44a31ccaf56d898 (DEFAULT_TABLE table=default.Table_1aa8069da6504812a85380a42ca07583 [id=a6d1215ba3be4fca903af9b17566b4a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:14.104511 19820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d3dbd9796564472b44a31ccaf56d898. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:14.105443 19938 tablet_bootstrap.cc:492] T bae8c5b6e1014731901bc3f336104784 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:14.105932 19938 ts_tablet_manager.cc:1403] T bae8c5b6e1014731901bc3f336104784 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.110s user 0.011s sys 0.001s
I20260925 13:56:14.108107 19938 raft_consensus.cc:359] T bae8c5b6e1014731901bc3f336104784 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:14.108698 19938 raft_consensus.cc:385] T bae8c5b6e1014731901bc3f336104784 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:14.109133 19277 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_9b29863ad2ca4d698aebe4b655b19e50"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:14.110513 19277 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9b29863ad2ca4d698aebe4b655b19e50 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:14.113300 20100 raft_consensus.cc:493] T f4429db544534f758883f2fae184ea6d P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:14.113723 20100 raft_consensus.cc:515] T f4429db544534f758883f2fae184ea6d P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:14.108947 19938 raft_consensus.cc:740] T bae8c5b6e1014731901bc3f336104784 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:14.115700 20100 leader_election.cc:290] T f4429db544534f758883f2fae184ea6d P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:14.117105 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4429db544534f758883f2fae184ea6d" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:14.118286 19840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4429db544534f758883f2fae184ea6d" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:14.120165 19631 leader_election.cc:304] T f4429db544534f758883f2fae184ea6d P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:14.121784 19959 ts_tablet_manager.cc:1929] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:14.122116 19959 log.cc:1199] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/51a4c21e06ca4b5b971cf8f04ffa52a9
I20260925 13:56:14.125048 20100 raft_consensus.cc:2749] T f4429db544534f758883f2fae184ea6d P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:14.125523 19676 tablet_service.cc:1511] Processing CreateTablet for tablet f1c6e2137700448a8ebf859a9ca9085d (DEFAULT_TABLE table=default.Table_4559d216bb6041bdba82905a01b2412a [id=5bf634ef72d243b1bfcafc4efe14f28a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:14.116274 19938 consensus_queue.cc:260] T bae8c5b6e1014731901bc3f336104784 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:14.126430 19676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1c6e2137700448a8ebf859a9ca9085d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:14.127419 19817 tablet_service.cc:1511] Processing CreateTablet for tablet f1c6e2137700448a8ebf859a9ca9085d (DEFAULT_TABLE table=default.Table_4559d216bb6041bdba82905a01b2412a [id=5bf634ef72d243b1bfcafc4efe14f28a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:14.128423 19817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1c6e2137700448a8ebf859a9ca9085d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:14.131181 20011 raft_consensus.cc:2243] T 3b10ecb0e6064d42a2521c4c4e91c028 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:14.131671 20011 raft_consensus.cc:2272] T 3b10ecb0e6064d42a2521c4c4e91c028 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:14.133282 20011 ts_tablet_manager.cc:1916] T 3b10ecb0e6064d42a2521c4c4e91c028 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:14.135447 20117 raft_consensus.cc:493] T 7292d04b411c415394e9d0df85aaf579 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:14.135881 20117 raft_consensus.cc:515] T 7292d04b411c415394e9d0df85aaf579 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:14.137815 20117 leader_election.cc:290] T 7292d04b411c415394e9d0df85aaf579 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:14.138301 19942 ts_tablet_manager.cc:1434] T b0a293e0094a4d9e937af6454e1ea16c P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.137s user 0.008s sys 0.000s
I20260925 13:56:14.140641 19840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7292d04b411c415394e9d0df85aaf579" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:14.141194 19840 raft_consensus.cc:2468] T 7292d04b411c415394e9d0df85aaf579 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:14.142549 19696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7292d04b411c415394e9d0df85aaf579" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:14.143661 19490 leader_election.cc:304] T 7292d04b411c415394e9d0df85aaf579 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters:
W20260925 13:56:14.145138 19488 leader_election.cc:343] T 7292d04b411c415394e9d0df85aaf579 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:14.146332 20117 raft_consensus.cc:2804] T 7292d04b411c415394e9d0df85aaf579 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:14.146668 20117 raft_consensus.cc:493] T 7292d04b411c415394e9d0df85aaf579 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:14.146960 20117 raft_consensus.cc:3060] T 7292d04b411c415394e9d0df85aaf579 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:14.155695 20117 raft_consensus.cc:515] T 7292d04b411c415394e9d0df85aaf579 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:14.158197 19696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7292d04b411c415394e9d0df85aaf579" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
W20260925 13:56:14.159631 19488 leader_election.cc:343] T 7292d04b411c415394e9d0df85aaf579 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:14.161741 19938 ts_tablet_manager.cc:1434] T bae8c5b6e1014731901bc3f336104784 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.056s user 0.005s sys 0.000s
W20260925 13:56:14.163401 19280 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bae8c5b6e1014731901bc3f336104784 consensus state has no leader
I20260925 13:56:14.164358 19280 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_d37fb0770cc84c09ab355bca02212825"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:14.165764 19280 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d37fb0770cc84c09ab355bca02212825 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:14.175107 19959 ts_tablet_manager.cc:1950] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:14.176656 19278 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet f51c5ad86ce64891a6393c1dc8d414e5
W20260925 13:56:14.179020 19274 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2a1f34dba13143b097a38bab3782c404 has no consensus state
I20260925 13:56:14.181557 19676 tablet_service.cc:1558] Processing DeleteTablet for tablet f51c5ad86ce64891a6393c1dc8d414e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:13 UTC) from {username='slave'} at 127.0.0.1:45948
W20260925 13:56:14.181730 19276 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.194:34125 (request call id 19) took 1238 ms (client timeout 1000 ms). Trace:
I20260925 13:56:14.183390 19533 tablet_service.cc:1558] Processing DeleteTablet for tablet f51c5ad86ce64891a6393c1dc8d414e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:13 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:14.185034 19676 tablet_service.cc:1558] Processing DeleteTablet for tablet ff3d09046b854fdaab2f95feff589ac1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:11 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:14.186410 19533 tablet_service.cc:1511] Processing CreateTablet for tablet 8239d0cd3b0d40bdb721e807c0f403d3 (DEFAULT_TABLE table=default.Table_374792fc278744f5ad6b9c1818b7a370 [id=7bbf083be05c424fadc6fc2a1df86758]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:14.187458 19533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8239d0cd3b0d40bdb721e807c0f403d3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:14.182044 19276 rpcz_store.cc:269] 0925 13:56:12.942831 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:12.942996 (+ 165us) service_pool.cc:224] Handling call
0925 13:56:14.181662 (+1238666us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:13.480472 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:13.480876 (+ 404us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:13.480901 (+ 25us) write_op.cc:435] Acquired schema lock
0925 13:56:13.480914 (+ 13us) tablet.cc:662] Decoding operations
0925 13:56:13.481154 (+ 240us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:13.481189 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:13.481210 (+ 21us) tablet.cc:685] Acquiring locks for 1 operations
0925 13:56:13.481348 (+ 138us) tablet.cc:701] Row locks acquired
0925 13:56:13.481360 (+ 12us) write_op.cc:260] PREPARE: finished
0925 13:56:13.481463 (+ 103us) write_op.cc:270] Start()
0925 13:56:13.481541 (+ 78us) write_op.cc:276] Timestamp: P: 1790344573481448 usec, L: 0
0925 13:56:13.481556 (+ 15us) op_driver.cc:348] REPLICATION: starting
0925 13:56:13.488311 (+ 6755us) log.cc:844] Serialized 535 byte log entry
0925 13:56:13.490341 (+ 2030us) spinlock_profiling.cc:243] Waited 6.33 ms on lock 0x7b5400040454. stack: 00007fdf0a30b323 00007fdf0a30b220 00007fdf09eaf574 0000000000516964 00000000005168f9 00007fdf11408aa1 00007fdf1149b5e8 00007fdf1192b741 00007fdf1192af18 00007fdf1192fbb1 00007fdf1192fb69 00007fdf1192faf9 00007fdf1192fac1 00007fdf1192eced 00000000005bd4e4 00000000005bd319
0925 13:56:13.497741 (+ 7400us) op_driver.cc:464] REPLICATION: finished
0925 13:56:13.513819 (+ 16078us) write_op.cc:301] APPLY: starting
0925 13:56:13.513882 (+ 63us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:13.513970 (+ 88us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:13.513992 (+ 22us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:13.514121 (+ 129us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:13.514234 (+ 113us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 13:56:13.514274 (+ 40us) write_op.cc:312] APPLY: finished
0925 13:56:13.514428 (+ 154us) log.cc:844] Serialized 22 byte log entry
0925 13:56:13.515011 (+ 583us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:13.515071 (+ 60us) write_op.cc:454] Released schema lock
0925 13:56:13.515101 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0925 13:56:13.524259 (+ 9158us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:13.524283 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":15910,"apply.run_cpu_time_us":1034,"apply.run_wall_time_us":6151,"num_ops":1,"prepare.queue_time_us":184406,"prepare.run_cpu_time_us":3079,"prepare.run_wall_time_us":43779,"raft.queue_time_us":127,"raft.run_cpu_time_us":663,"raft.run_wall_time_us":17066,"replication_time_us":16101,"spinlock_wait_cycles":14565760}]]}
W20260925 13:56:14.192212 19271 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.195:57799 (request call id 13) took 907 ms (client timeout 1000 ms). Trace:
W20260925 13:56:14.194965 19264 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet bae8c5b6e1014731901bc3f336104784 (table default.Table_99593e98f3394811b4c91e234185cbef [id=0736e722eb8c43e9b6a151a7c2880fe3], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet bae8c5b6e1014731901bc3f336104784 consensus state has no leader
I20260925 13:56:14.198107 19676 tablet_service.cc:1511] Processing CreateTablet for tablet 8239d0cd3b0d40bdb721e807c0f403d3 (DEFAULT_TABLE table=default.Table_374792fc278744f5ad6b9c1818b7a370 [id=7bbf083be05c424fadc6fc2a1df86758]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:14.199100 19676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8239d0cd3b0d40bdb721e807c0f403d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:14.201125 19675 tablet_service.cc:1558] Processing DeleteTablet for tablet d9455b98734147fb9e2d47c69513424a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:11 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:14.203490 19818 tablet_service.cc:1558] Processing DeleteTablet for tablet f51c5ad86ce64891a6393c1dc8d414e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:13 UTC) from {username='slave'} at 127.0.0.1:51442
W20260925 13:56:14.203747 19266 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.182.252:36181: connect: Connection refused (error 111) [suppressed 8 similar messages]
I20260925 13:56:14.204869 19815 tablet_service.cc:1558] Processing DeleteTablet for tablet f0cdc99cde2f4dabb819e3e171c5d8d9 with delete_type TABLET_DATA_DELETED (replaced by tablet f514fa52e7524cd3ba07f3f65de532d4) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:14.205579 19818 tablet_service.cc:1558] Processing DeleteTablet for tablet 28ac2912b94a4928967efb7a731c42b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:12 UTC) from {username='slave'} at 127.0.0.1:51442
W20260925 13:56:14.195448 19271 rpcz_store.cc:269] 0925 13:56:13.285018 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:13.285208 (+ 190us) service_pool.cc:224] Handling call
0925 13:56:13.348288 (+ 63080us) spinlock_profiling.cc:243] Waited 60.5 ms on lock 0x7b5400040454. stack: 00007fdf0a30b323 00007fdf0a30b220 00007fdf09eaf574 0000000000516964 00000000005168f9 00007fdf114095c0 00007fdf114a86fa 00007fdf13219d8c 00007fdf1331801e 00007fdf10ebcfb4 00007fdf10ebcf42 00007fdf10ebce71 00007fdf10ebcdec 00007fdf10ebc0a2 00007fdf0f16e09c 00007fdf0f16d4d6
0925 13:56:14.192154 (+843866us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:13.760303 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:13.760713 (+ 410us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:13.760735 (+ 22us) write_op.cc:435] Acquired schema lock
0925 13:56:13.760748 (+ 13us) tablet.cc:662] Decoding operations
0925 13:56:13.760987 (+ 239us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:13.761025 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:13.761050 (+ 25us) tablet.cc:685] Acquiring locks for 1 operations
0925 13:56:13.761196 (+ 146us) tablet.cc:701] Row locks acquired
0925 13:56:13.761209 (+ 13us) write_op.cc:260] PREPARE: finished
0925 13:56:13.761306 (+ 97us) write_op.cc:270] Start()
0925 13:56:13.761382 (+ 76us) write_op.cc:276] Timestamp: P: 1790344573761385 usec, L: 0
0925 13:56:13.761398 (+ 16us) op_driver.cc:348] REPLICATION: starting
0925 13:56:13.762032 (+ 634us) log.cc:844] Serialized 535 byte log entry
0925 13:56:13.770868 (+ 8836us) op_driver.cc:464] REPLICATION: finished
0925 13:56:13.774890 (+ 4022us) write_op.cc:301] APPLY: starting
0925 13:56:13.774943 (+ 53us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:13.775025 (+ 82us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:13.775127 (+ 102us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:13.775257 (+ 130us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:13.775911 (+ 654us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 13:56:13.775958 (+ 47us) write_op.cc:312] APPLY: finished
0925 13:56:13.776108 (+ 150us) log.cc:844] Serialized 22 byte log entry
0925 13:56:13.776228 (+ 120us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:13.776284 (+ 56us) write_op.cc:454] Released schema lock
0925 13:56:13.776312 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0925 13:56:13.779919 (+ 3607us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:13.779945 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":10,"spinlock_wait_cycles":139129728,"child_traces":[["op",{"apply.queue_time_us":3863,"apply.run_cpu_time_us":1097,"apply.run_wall_time_us":4988,"num_ops":1,"prepare.queue_time_us":97674,"prepare.run_cpu_time_us":1995,"prepare.run_wall_time_us":3063,"raft.queue_time_us":135,"raft.run_cpu_time_us":762,"raft.run_wall_time_us":762,"replication_time_us":9391,"spinlock_wait_cycles":345472}]]}
I20260925 13:56:14.207687 19840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7292d04b411c415394e9d0df85aaf579" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:14.208228 19840 raft_consensus.cc:3060] T 7292d04b411c415394e9d0df85aaf579 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:14.210875 19532 tablet_service.cc:1558] Processing DeleteTablet for tablet 00b500182bb04a72b80cfa9103e85fa9 with delete_type TABLET_DATA_DELETED (replaced by tablet f5a0bd0e205f4889be93739dfbd2a816) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:14.217222 20032 raft_consensus.cc:493] T b36aebfe984f454ea910e84645c4ac7d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:14.217602 20032 raft_consensus.cc:515] T b36aebfe984f454ea910e84645c4ac7d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:14.219225 20032 leader_election.cc:290] T b36aebfe984f454ea910e84645c4ac7d P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:14.221246 19696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b36aebfe984f454ea910e84645c4ac7d" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:14.221799 19942 tablet_bootstrap.cc:492] T 1513c0048bd244028d3dfa8d118bc14e P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:14.221840 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b36aebfe984f454ea910e84645c4ac7d" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
W20260925 13:56:14.223147 19773 leader_election.cc:343] T b36aebfe984f454ea910e84645c4ac7d P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:14.226037 19532 tablet_service.cc:1558] Processing DeleteTablet for tablet 3b10ecb0e6064d42a2521c4c4e91c028 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:11 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:14.227056 19942 tablet_bootstrap.cc:654] T 1513c0048bd244028d3dfa8d118bc14e P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:14.227747 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 51a4c21e06ca4b5b971cf8f04ffa52a9 (table default.Table_3cf69a468c014c279a928211b6ff89a2 [id=fe0f78d1c6e44f589aede852f77f88fc]) successfully deleted
I20260925 13:56:14.228237 19532 tablet_service.cc:1558] Processing DeleteTablet for tablet f0cdc99cde2f4dabb819e3e171c5d8d9 with delete_type TABLET_DATA_DELETED (replaced by tablet f514fa52e7524cd3ba07f3f65de532d4) from {username='slave'} at 127.0.0.1:40018
W20260925 13:56:14.228497 19773 leader_election.cc:343] T b36aebfe984f454ea910e84645c4ac7d P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:14.228925 19773 leader_election.cc:304] T b36aebfe984f454ea910e84645c4ac7d P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:14.230062 19275 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 8239d0cd3b0d40bdb721e807c0f403d3
I20260925 13:56:14.230119 20032 raft_consensus.cc:2749] T b36aebfe984f454ea910e84645c4ac7d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:14.231664 19532 tablet_service.cc:1558] Processing DeleteTablet for tablet 8239d0cd3b0d40bdb721e807c0f403d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:14 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:14.235836 19532 tablet_service.cc:1511] Processing CreateTablet for tablet 26644768f516412f8c95d68a76bb99c5 (DEFAULT_TABLE table=default.Table_bd52a86b8ff14bdbb06fcf88a60b1e82 [id=203024d67e444f839391c8670d7c7d3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:14.237118 19532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26644768f516412f8c95d68a76bb99c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:14.240653 19675 tablet_service.cc:1511] Processing CreateTablet for tablet f5a0bd0e205f4889be93739dfbd2a816 (DEFAULT_TABLE table=default.Table_04d053f331f243ce8e98beb7c00d4159 [id=745c9e0d4f2c4ce58c3a201c34f1f1e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:14.241691 19675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5a0bd0e205f4889be93739dfbd2a816. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:14.243942 19534 tablet_service.cc:1511] Processing CreateTablet for tablet f5a0bd0e205f4889be93739dfbd2a816 (DEFAULT_TABLE table=default.Table_04d053f331f243ce8e98beb7c00d4159 [id=745c9e0d4f2c4ce58c3a201c34f1f1e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:14.244874 19534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5a0bd0e205f4889be93739dfbd2a816. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:14.245229 19840 raft_consensus.cc:2468] T 7292d04b411c415394e9d0df85aaf579 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:14.246552 19490 leader_election.cc:304] T 7292d04b411c415394e9d0df85aaf579 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:14.247383 20068 raft_consensus.cc:2804] T 7292d04b411c415394e9d0df85aaf579 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:14.247498 19535 tablet_service.cc:1511] Processing CreateTablet for tablet a88bbe1ebee7468d8f49dfd25637fe83 (DEFAULT_TABLE table=default.Table_65608937f2cd4986b2588a4831750601 [id=7079cc0c725b4578a1b11c7dd3ef9342]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:14.248469 19535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a88bbe1ebee7468d8f49dfd25637fe83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:14.202699 19938 tablet_bootstrap.cc:492] T 608bed5b04394bfca1776b5d6c502df1 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:14.255484 20117 leader_election.cc:290] T 7292d04b411c415394e9d0df85aaf579 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:14.259688 19275 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 47521fb74f9c40ee999a7f704aa551c9 (table default.Table_3a3e48a0641a43a1a783c6ba6374a708 [id=e64fe44a237f48859a288a87a565ee5e]). Expected version 1 got 0
I20260925 13:56:14.259542 19273 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_5d692242213f4df6863089b93b4e9636"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:14.260430 19275 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet b0a293e0094a4d9e937af6454e1ea16c (table default.Table_256401fd998c430db7f494657ef8e75d [id=b673ac6542c5403fbf0fad16c4917dfd]). Expected version 1 got 0
W20260925 13:56:14.261183 19273 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5d692242213f4df6863089b93b4e9636 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:14.261363 19275 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet bae8c5b6e1014731901bc3f336104784 (table default.Table_99593e98f3394811b4c91e234185cbef [id=0736e722eb8c43e9b6a151a7c2880fe3]). Expected version 1 got 0
I20260925 13:56:14.262619 19676 tablet_service.cc:1558] Processing DeleteTablet for tablet 8239d0cd3b0d40bdb721e807c0f403d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:14 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:14.267550 19959 tablet_replica.cc:333] T 980d1264c0ea45f3b2b762509e0b2c77 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:14.268262 19959 raft_consensus.cc:2243] T 980d1264c0ea45f3b2b762509e0b2c77 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:14.268368 19817 tablet_service.cc:1511] Processing CreateTablet for tablet 8239d0cd3b0d40bdb721e807c0f403d3 (DEFAULT_TABLE table=default.Table_374792fc278744f5ad6b9c1818b7a370 [id=7bbf083be05c424fadc6fc2a1df86758]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:14.268708 19959 raft_consensus.cc:2272] T 980d1264c0ea45f3b2b762509e0b2c77 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:14.269383 19817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8239d0cd3b0d40bdb721e807c0f403d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:14.269754 19820 tablet_service.cc:1558] Processing DeleteTablet for tablet 8239d0cd3b0d40bdb721e807c0f403d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:14 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:14.280821 19676 tablet_service.cc:1511] Processing CreateTablet for tablet a88bbe1ebee7468d8f49dfd25637fe83 (DEFAULT_TABLE table=default.Table_65608937f2cd4986b2588a4831750601 [id=7079cc0c725b4578a1b11c7dd3ef9342]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:14.281910 19531 tablet_service.cc:1511] Processing CreateTablet for tablet c65a8cd3060c405c871270543b4623f7 (DEFAULT_TABLE table=default.Table_430d31bd6eb64dfea4d41806e919d5bf [id=28a472597bfc45619e1da98d6504841a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:14.282689 19939 tablet_bootstrap.cc:492] T 980d1264c0ea45f3b2b762509e0b2c77 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:14.282961 19531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c65a8cd3060c405c871270543b4623f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:14.283094 19939 ts_tablet_manager.cc:1403] T 980d1264c0ea45f3b2b762509e0b2c77 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.299s user 0.032s sys 0.014s
I20260925 13:56:14.287256 19939 raft_consensus.cc:359] T 980d1264c0ea45f3b2b762509e0b2c77 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:14.287865 19939 raft_consensus.cc:385] T 980d1264c0ea45f3b2b762509e0b2c77 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:14.288102 19939 raft_consensus.cc:740] T 980d1264c0ea45f3b2b762509e0b2c77 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:14.288664 19939 consensus_queue.cc:260] T 980d1264c0ea45f3b2b762509e0b2c77 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:14.290417 19939 ts_tablet_manager.cc:1434] T 980d1264c0ea45f3b2b762509e0b2c77 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20260925 13:56:14.292094 19959 ts_tablet_manager.cc:1916] T 980d1264c0ea45f3b2b762509e0b2c77 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:14.295339 20135 tablet.cc:1751] T b0a293e0094a4d9e937af6454e1ea16c P f790493ba7ad403993b678f2df2b55be: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:14.296625 19938 tablet_bootstrap.cc:654] T 608bed5b04394bfca1776b5d6c502df1 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:14.297653 20134 tablet.cc:1751] T b0a293e0094a4d9e937af6454e1ea16c P 8f578869bc084a5d8a9a69f143954eb3: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:14.299439 20068 raft_consensus.cc:697] T 7292d04b411c415394e9d0df85aaf579 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Running, Role: LEADER
I20260925 13:56:14.300252 20068 consensus_queue.cc:237] T 7292d04b411c415394e9d0df85aaf579 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:14.311208 19273 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_ab121c553eee48ccb03ed0c1e875a43a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:14.312879 19273 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ab121c553eee48ccb03ed0c1e875a43a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:14.313007 19820 tablet_service.cc:1511] Processing CreateTablet for tablet 26644768f516412f8c95d68a76bb99c5 (DEFAULT_TABLE table=default.Table_bd52a86b8ff14bdbb06fcf88a60b1e82 [id=203024d67e444f839391c8670d7c7d3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:14.313993 19820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26644768f516412f8c95d68a76bb99c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:14.314352 19675 tablet_service.cc:1511] Processing CreateTablet for tablet c65a8cd3060c405c871270543b4623f7 (DEFAULT_TABLE table=default.Table_430d31bd6eb64dfea4d41806e919d5bf [id=28a472597bfc45619e1da98d6504841a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:14.315299 19271 master_service.cc:960] ReplaceTablet: received request to replace tablet 980d1264c0ea45f3b2b762509e0b2c77 from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:14.329054 19818 tablet_service.cc:1511] Processing CreateTablet for tablet f5a0bd0e205f4889be93739dfbd2a816 (DEFAULT_TABLE table=default.Table_04d053f331f243ce8e98beb7c00d4159 [id=745c9e0d4f2c4ce58c3a201c34f1f1e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:14.330245 19818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5a0bd0e205f4889be93739dfbd2a816. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:14.341032 19815 tablet_service.cc:1511] Processing CreateTablet for tablet a88bbe1ebee7468d8f49dfd25637fe83 (DEFAULT_TABLE table=default.Table_65608937f2cd4986b2588a4831750601 [id=7079cc0c725b4578a1b11c7dd3ef9342]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:14.341555 19819 tablet_service.cc:1511] Processing CreateTablet for tablet c65a8cd3060c405c871270543b4623f7 (DEFAULT_TABLE table=default.Table_430d31bd6eb64dfea4d41806e919d5bf [id=28a472597bfc45619e1da98d6504841a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:14.342044 19815 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a88bbe1ebee7468d8f49dfd25637fe83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:14.342530 19819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c65a8cd3060c405c871270543b4623f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:14.344079 19272 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet b0a293e0094a4d9e937af6454e1ea16c (table default.Table_256401fd998c430db7f494657ef8e75d [id=b673ac6542c5403fbf0fad16c4917dfd]). Expected version 1 got 0
I20260925 13:56:14.344298 19673 tablet_service.cc:1511] Processing CreateTablet for tablet 26644768f516412f8c95d68a76bb99c5 (DEFAULT_TABLE table=default.Table_bd52a86b8ff14bdbb06fcf88a60b1e82 [id=203024d67e444f839391c8670d7c7d3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:14.355652 19676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a88bbe1ebee7468d8f49dfd25637fe83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:14.356727 19673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26644768f516412f8c95d68a76bb99c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:14.357326 19675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c65a8cd3060c405c871270543b4623f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:14.366034 19942 tablet_bootstrap.cc:492] T 1513c0048bd244028d3dfa8d118bc14e P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:14.366428 19942 ts_tablet_manager.cc:1403] T 1513c0048bd244028d3dfa8d118bc14e P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.145s user 0.008s sys 0.000s
I20260925 13:56:14.367384 19271 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_826761206dd0483f8703cfaf2154e80a" } new_table_name: "default.Table_d98372484b48457abb338cba82de44b7" modify_external_catalogs: true
W20260925 13:56:14.376960 19631 consensus_peers.cc:597] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f -> Peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Couldn't send request to peer f790493ba7ad403993b678f2df2b55be. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f0cdc99cde2f4dabb819e3e171c5d8d9. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:14.378655 19942 raft_consensus.cc:359] T 1513c0048bd244028d3dfa8d118bc14e P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:14.379266 19942 raft_consensus.cc:385] T 1513c0048bd244028d3dfa8d118bc14e P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:14.379547 19942 raft_consensus.cc:740] T 1513c0048bd244028d3dfa8d118bc14e P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:14.384873 19272 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_3cf69a468c014c279a928211b6ff89a2" } modify_external_catalogs: true
I20260925 13:56:14.386041 19277 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_b194d76d005f44b7a93ec10df4e3c59c" } new_table_name: "default.Table_546097946d054781b0acae5186c5d43d" modify_external_catalogs: true
I20260925 13:56:14.395309 19959 ts_tablet_manager.cc:1929] T 980d1264c0ea45f3b2b762509e0b2c77 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:14.396621 19959 log.cc:1199] T 980d1264c0ea45f3b2b762509e0b2c77 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/980d1264c0ea45f3b2b762509e0b2c77
I20260925 13:56:14.398252 19280 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_9d043ec520e1422995a293b09996a713"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:14.398739 19959 ts_tablet_manager.cc:1950] T 980d1264c0ea45f3b2b762509e0b2c77 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
W20260925 13:56:14.399612 19280 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9d043ec520e1422995a293b09996a713 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:14.400275 19942 consensus_queue.cc:260] T 1513c0048bd244028d3dfa8d118bc14e P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:14.402216 19942 ts_tablet_manager.cc:1434] T 1513c0048bd244028d3dfa8d118bc14e P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.036s user 0.006s sys 0.000s
I20260925 13:56:14.406706 19959 tablet_replica.cc:333] T f51c5ad86ce64891a6393c1dc8d414e5 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:14.395640 19272 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_3cf69a468c014c279a928211b6ff89a2" } modify_external_catalogs: true
I20260925 13:56:14.407567 19942 tablet_bootstrap.cc:492] T 00b500182bb04a72b80cfa9103e85fa9 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:14.412746 19942 tablet_bootstrap.cc:654] T 00b500182bb04a72b80cfa9103e85fa9 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:14.424204 19273 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_86e7099f24054da589b5a6f7de9ab131"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:14.426126 19273 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_86e7099f24054da589b5a6f7de9ab131 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:14.430753 19276 master_service.cc:960] ReplaceTablet: received request to replace tablet c0f05269ea6c43a98d2137eae7a9a401 from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:14.434907 19274 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_03fdc4ff0d534d8c9db1aba08970a310" } modify_external_catalogs: true
I20260925 13:56:14.435480 19274 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_03fdc4ff0d534d8c9db1aba08970a310" } modify_external_catalogs: true
W20260925 13:56:14.437817 19629 consensus_peers.cc:597] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f -> Peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Couldn't send request to peer 8f578869bc084a5d8a9a69f143954eb3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f0cdc99cde2f4dabb819e3e171c5d8d9. This is attempt 1: this message will repeat every 5th retry.
W20260925 13:56:14.447839 19271 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 137c43fb74ec4cc48d30ed3edac81d98 has no consensus state
W20260925 13:56:14.458151 19277 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e6e59d6f4f6e4de1be1cf450620cfa59 has no consensus state
I20260925 13:56:14.461740 19942 tablet_bootstrap.cc:492] T 00b500182bb04a72b80cfa9103e85fa9 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:14.462157 19942 ts_tablet_manager.cc:1403] T 00b500182bb04a72b80cfa9103e85fa9 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.055s user 0.008s sys 0.000s
I20260925 13:56:14.463984 19276 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet c0f05269ea6c43a98d2137eae7a9a401
I20260925 13:56:14.464991 19942 raft_consensus.cc:359] T 00b500182bb04a72b80cfa9103e85fa9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:14.465430 19276 catalog_manager.cc:6934] ReplaceTablet: tablet c0f05269ea6c43a98d2137eae7a9a401 deleted and replaced by tablet 371b5b10fc4948f49c515de7b9049220
I20260925 13:56:14.465610 19942 raft_consensus.cc:385] T 00b500182bb04a72b80cfa9103e85fa9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:14.465843 19942 raft_consensus.cc:740] T 00b500182bb04a72b80cfa9103e85fa9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:14.466180 19673 tablet_service.cc:1558] Processing DeleteTablet for tablet c0f05269ea6c43a98d2137eae7a9a401 with delete_type TABLET_DATA_DELETED (replaced by tablet 371b5b10fc4948f49c515de7b9049220) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:14.468321 19531 tablet_service.cc:1558] Processing DeleteTablet for tablet c0f05269ea6c43a98d2137eae7a9a401 with delete_type TABLET_DATA_DELETED (replaced by tablet 371b5b10fc4948f49c515de7b9049220) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:14.467630 19942 consensus_queue.cc:260] T 00b500182bb04a72b80cfa9103e85fa9 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:14.484133 19816 tablet_service.cc:1558] Processing DeleteTablet for tablet c0f05269ea6c43a98d2137eae7a9a401 with delete_type TABLET_DATA_DELETED (replaced by tablet 371b5b10fc4948f49c515de7b9049220) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:14.484341 19942 ts_tablet_manager.cc:1434] T 00b500182bb04a72b80cfa9103e85fa9 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.022s user 0.004s sys 0.003s
I20260925 13:56:14.485493 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 980d1264c0ea45f3b2b762509e0b2c77 (table default.Table_8290acbcb9d84a48aad169c54251bc19 [id=90312b1b0c254ff9ad5693f561114a7f]) successfully deleted
I20260925 13:56:14.505471 19942 tablet_bootstrap.cc:492] T c1f15b9de572479aae6c2e644da3392a P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:14.517503 19942 tablet_bootstrap.cc:654] T c1f15b9de572479aae6c2e644da3392a P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:14.517926 19273 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_16c91b8d55be449899b5e3a1884d81b4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:14.521410 20068 raft_consensus.cc:493] T c1f15b9de572479aae6c2e644da3392a P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260925 13:56:14.521746 19273 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_16c91b8d55be449899b5e3a1884d81b4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:14.521850 20068 raft_consensus.cc:515] T c1f15b9de572479aae6c2e644da3392a P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:14.523710 20068 leader_election.cc:290] T c1f15b9de572479aae6c2e644da3392a P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:14.525549 19696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f15b9de572479aae6c2e644da3392a" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:14.526100 19938 tablet_bootstrap.cc:492] T 608bed5b04394bfca1776b5d6c502df1 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:14.526496 19938 ts_tablet_manager.cc:1403] T 608bed5b04394bfca1776b5d6c502df1 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.324s user 0.015s sys 0.026s
W20260925 13:56:14.527211 19488 leader_election.cc:343] T c1f15b9de572479aae6c2e644da3392a P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:14.528067 19959 raft_consensus.cc:2243] T f51c5ad86ce64891a6393c1dc8d414e5 P f790493ba7ad403993b678f2df2b55be [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:14.530187 19839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f15b9de572479aae6c2e644da3392a" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:14.530643 20068 raft_consensus.cc:493] T 980d1264c0ea45f3b2b762509e0b2c77 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:14.530864 19839 raft_consensus.cc:2468] T c1f15b9de572479aae6c2e644da3392a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:14.531052 20068 raft_consensus.cc:515] T 980d1264c0ea45f3b2b762509e0b2c77 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:14.532102 19490 leader_election.cc:304] T c1f15b9de572479aae6c2e644da3392a P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:14.533170 20136 raft_consensus.cc:2804] T c1f15b9de572479aae6c2e644da3392a P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:14.533567 20136 raft_consensus.cc:493] T c1f15b9de572479aae6c2e644da3392a P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:14.533906 20136 raft_consensus.cc:3060] T c1f15b9de572479aae6c2e644da3392a P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:14.534595 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "980d1264c0ea45f3b2b762509e0b2c77" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:14.537537 19274 catalog_manager.cc:6128] Not sending DeleteTablet requests; no consensus state for tablet 9512bc7fe1374bf482986f7d844c6727
I20260925 13:56:14.540226 20068 leader_election.cc:290] T 980d1264c0ea45f3b2b762509e0b2c77 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:14.540326 19959 pending_rounds.cc:70] T f51c5ad86ce64891a6393c1dc8d414e5 P f790493ba7ad403993b678f2df2b55be: Trying to abort 1 pending ops.
I20260925 13:56:14.540555 19959 pending_rounds.cc:77] T f51c5ad86ce64891a6393c1dc8d414e5 P f790493ba7ad403993b678f2df2b55be: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7333251376538038272 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "f51c5ad86ce64891a6393c1dc8d414e5" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 1 new_table_name: "default.Table_824449a97a864e3dbbf06e2ad678edc5" dest_uuid: "f790493ba7ad403993b678f2df2b55be" new_extra_config { } }
W20260925 13:56:14.541608 19959 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:51442: Aborted: Op aborted
W20260925 13:56:14.542897 19490 leader_election.cc:343] T 980d1264c0ea45f3b2b762509e0b2c77 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 980d1264c0ea45f3b2b762509e0b2c77
I20260925 13:56:14.542959 19959 raft_consensus.cc:2272] T f51c5ad86ce64891a6393c1dc8d414e5 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:14.545054 19959 ts_tablet_manager.cc:1916] T f51c5ad86ce64891a6393c1dc8d414e5 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:14.547490 19955 ts_tablet_manager.cc:1929] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:14.547900 19955 log.cc:1199] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/51a4c21e06ca4b5b971cf8f04ffa52a9
W20260925 13:56:14.548586 19488 leader_election.cc:343] T 980d1264c0ea45f3b2b762509e0b2c77 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:14.549008 19488 leader_election.cc:304] T 980d1264c0ea45f3b2b762509e0b2c77 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3; no voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:14.550203 20068 raft_consensus.cc:2749] T 980d1264c0ea45f3b2b762509e0b2c77 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:14.291218 19939 tablet_bootstrap.cc:492] T dc14c37aa393407a87af0d0eb8b88d82 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:14.552024 20068 raft_consensus.cc:493] T 18a6d1c4eb7948149e709a380da50d93 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:14.552493 20068 raft_consensus.cc:515] T 18a6d1c4eb7948149e709a380da50d93 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:14.555269 19839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a6d1c4eb7948149e709a380da50d93" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:14.555928 19839 raft_consensus.cc:2468] T 18a6d1c4eb7948149e709a380da50d93 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:14.556493 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a6d1c4eb7948149e709a380da50d93" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:14.558542 19490 leader_election.cc:304] T 18a6d1c4eb7948149e709a380da50d93 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:14.558825 20136 raft_consensus.cc:515] T c1f15b9de572479aae6c2e644da3392a P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:14.565965 19696 raft_consensus.cc:3060] T f4429db544534f758883f2fae184ea6d P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:14.568209 20032 raft_consensus.cc:493] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:14.568614 20032 raft_consensus.cc:515] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:14.559453 20123 raft_consensus.cc:2804] T 18a6d1c4eb7948149e709a380da50d93 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:14.569607 20123 raft_consensus.cc:493] T 18a6d1c4eb7948149e709a380da50d93 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:14.569952 20123 raft_consensus.cc:3060] T 18a6d1c4eb7948149e709a380da50d93 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
W20260925 13:56:14.560055 19488 leader_election.cc:343] T 18a6d1c4eb7948149e709a380da50d93 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:14.570492 20032 leader_election.cc:290] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:14.572613 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47521fb74f9c40ee999a7f704aa551c9" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:14.573915 19773 leader_election.cc:343] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:14.577425 19938 raft_consensus.cc:359] T 608bed5b04394bfca1776b5d6c502df1 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:14.578047 19938 raft_consensus.cc:385] T 608bed5b04394bfca1776b5d6c502df1 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:14.578991 19938 raft_consensus.cc:740] T 608bed5b04394bfca1776b5d6c502df1 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:14.580767 20011 ts_tablet_manager.cc:1929] T 3b10ecb0e6064d42a2521c4c4e91c028 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:14.581147 20011 log.cc:1199] T 3b10ecb0e6064d42a2521c4c4e91c028 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/3b10ecb0e6064d42a2521c4c4e91c028
I20260925 13:56:14.581823 20011 ts_tablet_manager.cc:1950] T 3b10ecb0e6064d42a2521c4c4e91c028 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:14.582973 19938 consensus_queue.cc:260] T 608bed5b04394bfca1776b5d6c502df1 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:14.560521 20068 leader_election.cc:290] T 18a6d1c4eb7948149e709a380da50d93 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:14.548563 19955 ts_tablet_manager.cc:1950] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:14.599478 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47521fb74f9c40ee999a7f704aa551c9" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:14.599975 19938 ts_tablet_manager.cc:1434] T 608bed5b04394bfca1776b5d6c502df1 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.073s user 0.007s sys 0.000s
I20260925 13:56:14.591085 20117 raft_consensus.cc:493] T 47521fb74f9c40ee999a7f704aa551c9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:14.600359 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 53) took 1017 ms. Trace:
I20260925 13:56:14.600312 20117 raft_consensus.cc:515] T 47521fb74f9c40ee999a7f704aa551c9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:14.600754 19955 rpcz_store.cc:276] 0925 13:56:13.582531 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:13.583073 (+ 542us) service_pool.cc:224] Handling call
0925 13:56:13.978007 (+394934us) ts_tablet_manager.cc:1151] Deleting tablet 51a4c21e06ca4b5b971cf8f04ffa52a9
0925 13:56:13.978040 (+ 33us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:14.306168 (+328128us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:14.547384 (+241216us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:14.600308 (+ 52924us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75,"tablet-delete.queue_time_us":357052}
I20260925 13:56:14.602149 20117 leader_election.cc:290] T 47521fb74f9c40ee999a7f704aa551c9 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:14.602679 19555 raft_consensus.cc:2468] T 47521fb74f9c40ee999a7f704aa551c9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 0.
I20260925 13:56:14.605947 19938 tablet_bootstrap.cc:492] T 4d3dbd9796564472b44a31ccaf56d898 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:14.606108 19839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47521fb74f9c40ee999a7f704aa551c9" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:14.606650 19839 raft_consensus.cc:2468] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:14.609858 20123 raft_consensus.cc:515] T 18a6d1c4eb7948149e709a380da50d93 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:14.611387 19773 leader_election.cc:304] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:14.612386 20032 raft_consensus.cc:2804] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:14.612707 20032 raft_consensus.cc:493] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:14.613008 20032 raft_consensus.cc:3060] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:14.616395 19490 leader_election.cc:304] T 47521fb74f9c40ee999a7f704aa551c9 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:14.618369 20068 raft_consensus.cc:2804] T 47521fb74f9c40ee999a7f704aa551c9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:14.618726 20068 raft_consensus.cc:493] T 47521fb74f9c40ee999a7f704aa551c9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:14.619040 20068 raft_consensus.cc:3060] T 47521fb74f9c40ee999a7f704aa551c9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:14.621924 20032 raft_consensus.cc:515] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:14.625461 19959 ts_tablet_manager.cc:1929] T f51c5ad86ce64891a6393c1dc8d414e5 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:56:14.625839 19959 log.cc:1199] T f51c5ad86ce64891a6393c1dc8d414e5 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/f51c5ad86ce64891a6393c1dc8d414e5
I20260925 13:56:14.626296 19839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a6d1c4eb7948149e709a380da50d93" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:14.626551 19959 ts_tablet_manager.cc:1950] T f51c5ad86ce64891a6393c1dc8d414e5 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:14.626798 19839 raft_consensus.cc:3060] T 18a6d1c4eb7948149e709a380da50d93 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:14.627041 19942 tablet_bootstrap.cc:492] T c1f15b9de572479aae6c2e644da3392a P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:14.627566 19942 ts_tablet_manager.cc:1403] T c1f15b9de572479aae6c2e644da3392a P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.122s user 0.010s sys 0.000s
I20260925 13:56:14.629951 19942 raft_consensus.cc:359] T c1f15b9de572479aae6c2e644da3392a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:14.629009 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet f51c5ad86ce64891a6393c1dc8d414e5 (table default.Table_824449a97a864e3dbbf06e2ad678edc5 [id=af708d3ba4c54ba19c24eca862bbddc1]) successfully deleted
I20260925 13:56:14.624949 20123 leader_election.cc:290] T 18a6d1c4eb7948149e709a380da50d93 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:14.634133 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet 51a4c21e06ca4b5b971cf8f04ffa52a9 (table default.Table_3cf69a468c014c279a928211b6ff89a2 [id=fe0f78d1c6e44f589aede852f77f88fc]) successfully deleted
I20260925 13:56:14.635996 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47521fb74f9c40ee999a7f704aa551c9" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:14.636933 19955 tablet_replica.cc:333] T 980d1264c0ea45f3b2b762509e0b2c77 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:14.639062 19840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f15b9de572479aae6c2e644da3392a" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:14.639628 19840 raft_consensus.cc:3060] T c1f15b9de572479aae6c2e644da3392a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:14.611233 19938 tablet_bootstrap.cc:654] T 4d3dbd9796564472b44a31ccaf56d898 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:14.646411 19840 raft_consensus.cc:2468] T c1f15b9de572479aae6c2e644da3392a P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
W20260925 13:56:14.646643 19266 catalog_manager.cc:5030] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): delete failed for tablet f0cdc99cde2f4dabb819e3e171c5d8d9 because the tablet was not found. No further retry: Not found: Tablet not found: f0cdc99cde2f4dabb819e3e171c5d8d9
W20260925 13:56:14.648068 19266 catalog_manager.cc:5030] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): delete failed for tablet 28ac2912b94a4928967efb7a731c42b9 because the tablet was not found. No further retry: Not found: Tablet not found: 28ac2912b94a4928967efb7a731c42b9
I20260925 13:56:14.649082 19955 raft_consensus.cc:2243] T 980d1264c0ea45f3b2b762509e0b2c77 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260925 13:56:14.649452 19266 catalog_manager.cc:5043] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): delete failed for tablet 8239d0cd3b0d40bdb721e807c0f403d3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8239d0cd3b0d40bdb721e807c0f403d3 already in progress: creating tablet
I20260925 13:56:14.649240 20068 raft_consensus.cc:515] T 47521fb74f9c40ee999a7f704aa551c9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:14.649554 19955 raft_consensus.cc:2272] T 980d1264c0ea45f3b2b762509e0b2c77 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:14.650271 20032 leader_election.cc:290] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:14.650817 19555 raft_consensus.cc:2393] T 47521fb74f9c40ee999a7f704aa551c9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f790493ba7ad403993b678f2df2b55be in current term 1: Already voted for candidate 8f578869bc084a5d8a9a69f143954eb3 in this term.
I20260925 13:56:14.652160 19959 tablet_replica.cc:333] T c0f05269ea6c43a98d2137eae7a9a401 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:14.652890 19959 raft_consensus.cc:2243] T c0f05269ea6c43a98d2137eae7a9a401 P f790493ba7ad403993b678f2df2b55be [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:14.653575 19955 ts_tablet_manager.cc:1916] T 980d1264c0ea45f3b2b762509e0b2c77 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:14.653648 19959 raft_consensus.cc:2272] T c0f05269ea6c43a98d2137eae7a9a401 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:14.655488 19942 raft_consensus.cc:385] T c1f15b9de572479aae6c2e644da3392a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:14.655789 19942 raft_consensus.cc:740] T c1f15b9de572479aae6c2e644da3392a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:14.656383 19942 consensus_queue.cc:260] T c1f15b9de572479aae6c2e644da3392a P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:14.650907 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 39) took 1720 ms. Trace:
I20260925 13:56:14.657227 20011 rpcz_store.cc:276] 0925 13:56:12.930852 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:12.935579 (+ 4727us) service_pool.cc:224] Handling call
0925 13:56:14.005589 (+1070010us) ts_tablet_manager.cc:1151] Deleting tablet 3b10ecb0e6064d42a2521c4c4e91c028
0925 13:56:14.005647 (+ 58us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:14.383644 (+377997us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:14.580650 (+197006us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:14.650841 (+ 70191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11215,"tablet-delete.queue_time_us":1069448}
I20260925 13:56:14.658355 20011 tablet_replica.cc:333] T 28ac2912b94a4928967efb7a731c42b9 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:14.659078 20011 raft_consensus.cc:2243] T 28ac2912b94a4928967efb7a731c42b9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:14.647781 19490 leader_election.cc:304] T c1f15b9de572479aae6c2e644da3392a P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:14.661891 20123 raft_consensus.cc:2804] T c1f15b9de572479aae6c2e644da3392a P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:14.661767 19273 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_93893aaa73a645b0939b9fd65f21d37a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:14.662307 20123 raft_consensus.cc:697] T c1f15b9de572479aae6c2e644da3392a P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Running, Role: LEADER
I20260925 13:56:14.663985 19277 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_86a5b8762a65441aacd09147e7fb78fd" } new_table_name: "default.Table_36b4dd2097aa4e2dbd58d98277e90281" modify_external_catalogs: true
I20260925 13:56:14.664857 20123 consensus_queue.cc:237] T c1f15b9de572479aae6c2e644da3392a P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:14.639240 20136 leader_election.cc:290] T c1f15b9de572479aae6c2e644da3392a P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:14.666177 19938 tablet_bootstrap.cc:492] T 4d3dbd9796564472b44a31ccaf56d898 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:14.666533 19938 ts_tablet_manager.cc:1403] T 4d3dbd9796564472b44a31ccaf56d898 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.063s user 0.008s sys 0.000s
I20260925 13:56:14.668149 19274 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_dfe9d9863d364dfc8df4c405e87dd635"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:14.670897 19959 ts_tablet_manager.cc:1916] T c0f05269ea6c43a98d2137eae7a9a401 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:14.669709 19938 raft_consensus.cc:359] T 4d3dbd9796564472b44a31ccaf56d898 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:14.672816 19942 ts_tablet_manager.cc:1434] T c1f15b9de572479aae6c2e644da3392a P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.045s user 0.004s sys 0.000s
I20260925 13:56:14.673686 19942 tablet_bootstrap.cc:492] T 18a6d1c4eb7948149e709a380da50d93 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:14.674333 19271 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_d98372484b48457abb338cba82de44b7" } new_table_name: "default.Table_5321051e9f6e4cd28b9b77619c908dc9" modify_external_catalogs: true
I20260925 13:56:14.672538 19938 raft_consensus.cc:385] T 4d3dbd9796564472b44a31ccaf56d898 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:14.674919 19938 raft_consensus.cc:740] T 4d3dbd9796564472b44a31ccaf56d898 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:14.676968 20011 raft_consensus.cc:2272] T 28ac2912b94a4928967efb7a731c42b9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:14.678258 19275 master_service.cc:960] ReplaceTablet: received request to replace tablet 7292d04b411c415394e9d0df85aaf579 from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:14.678527 20011 ts_tablet_manager.cc:1916] T 28ac2912b94a4928967efb7a731c42b9 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:14.646636 19839 raft_consensus.cc:2468] T 18a6d1c4eb7948149e709a380da50d93 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:14.684333 19490 leader_election.cc:304] T 18a6d1c4eb7948149e709a380da50d93 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:14.683573 19938 consensus_queue.cc:260] T 4d3dbd9796564472b44a31ccaf56d898 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:14.566267 19939 tablet_bootstrap.cc:654] T dc14c37aa393407a87af0d0eb8b88d82 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
W20260925 13:56:14.693451 19274 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dfe9d9863d364dfc8df4c405e87dd635 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:14.693617 19938 ts_tablet_manager.cc:1434] T 4d3dbd9796564472b44a31ccaf56d898 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.027s user 0.007s sys 0.000s
I20260925 13:56:14.685186 20136 raft_consensus.cc:2804] T 18a6d1c4eb7948149e709a380da50d93 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:14.697974 20136 raft_consensus.cc:697] T 18a6d1c4eb7948149e709a380da50d93 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Running, Role: LEADER
I20260925 13:56:14.698735 20136 consensus_queue.cc:237] T 18a6d1c4eb7948149e709a380da50d93 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
W20260925 13:56:14.700273 19273 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_93893aaa73a645b0939b9fd65f21d37a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:14.706369 19938 tablet_bootstrap.cc:492] T f1c6e2137700448a8ebf859a9ca9085d P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:14.722801 20068 leader_election.cc:290] T 47521fb74f9c40ee999a7f704aa551c9 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:14.724133 19839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47521fb74f9c40ee999a7f704aa551c9" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:14.725353 19839 raft_consensus.cc:2393] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f578869bc084a5d8a9a69f143954eb3 in current term 1: Already voted for candidate f790493ba7ad403993b678f2df2b55be in this term.
I20260925 13:56:14.725962 19938 tablet_bootstrap.cc:654] T f1c6e2137700448a8ebf859a9ca9085d P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
W20260925 13:56:14.735458 19277 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5943795bc8b24bb48b80d6d06d17750b has no consensus state
W20260925 13:56:14.736610 19271 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 137c43fb74ec4cc48d30ed3edac81d98 has no consensus state
I20260925 13:56:14.740094 19280 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_899d461298254c7aa8b6c4abbb123d6f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:14.741506 19280 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_899d461298254c7aa8b6c4abbb123d6f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:14.743443 19959 ts_tablet_manager.cc:1929] T c0f05269ea6c43a98d2137eae7a9a401 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:56:14.743815 19959 log.cc:1199] T c0f05269ea6c43a98d2137eae7a9a401 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/c0f05269ea6c43a98d2137eae7a9a401
I20260925 13:56:14.745841 19271 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_3b49fb893abb48ac8646283360e8e681" } modify_external_catalogs: true
I20260925 13:56:14.746345 19271 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_3b49fb893abb48ac8646283360e8e681" } modify_external_catalogs: true
I20260925 13:56:14.746577 19959 ts_tablet_manager.cc:1950] T c0f05269ea6c43a98d2137eae7a9a401 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:14.749032 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet c0f05269ea6c43a98d2137eae7a9a401 (table default.Table_72b4fdbfbca745a286dca8c191dc2286 [id=41786d7cdac8440ca86768cac5b28f49]) successfully deleted
I20260925 13:56:14.754807 19939 tablet_bootstrap.cc:492] T dc14c37aa393407a87af0d0eb8b88d82 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:14.759582 19938 tablet_bootstrap.cc:492] T f1c6e2137700448a8ebf859a9ca9085d P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:14.760098 19938 ts_tablet_manager.cc:1403] T f1c6e2137700448a8ebf859a9ca9085d P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.054s user 0.008s sys 0.001s
I20260925 13:56:14.763042 19938 raft_consensus.cc:359] T f1c6e2137700448a8ebf859a9ca9085d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:14.768131 19938 raft_consensus.cc:385] T f1c6e2137700448a8ebf859a9ca9085d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:14.768406 19938 raft_consensus.cc:740] T f1c6e2137700448a8ebf859a9ca9085d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:14.755273 19939 ts_tablet_manager.cc:1403] T dc14c37aa393407a87af0d0eb8b88d82 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.464s user 0.007s sys 0.003s
I20260925 13:56:14.769959 19277 master_service.cc:960] ReplaceTablet: received request to replace tablet a88bbe1ebee7468d8f49dfd25637fe83 from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:14.772521 19938 consensus_queue.cc:260] T f1c6e2137700448a8ebf859a9ca9085d P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:14.792778 20100 raft_consensus.cc:493] T b0a293e0094a4d9e937af6454e1ea16c P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:14.793758 20100 raft_consensus.cc:515] T b0a293e0094a4d9e937af6454e1ea16c P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:14.796569 19942 tablet_bootstrap.cc:654] T 18a6d1c4eb7948149e709a380da50d93 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:14.798760 19938 ts_tablet_manager.cc:1434] T f1c6e2137700448a8ebf859a9ca9085d P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.038s user 0.007s sys 0.000s
I20260925 13:56:14.804246 20100 leader_election.cc:290] T b0a293e0094a4d9e937af6454e1ea16c P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:14.804792 20100 raft_consensus.cc:493] T 1513c0048bd244028d3dfa8d118bc14e P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:14.805197 20100 raft_consensus.cc:515] T 1513c0048bd244028d3dfa8d118bc14e P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:14.806979 20100 leader_election.cc:290] T 1513c0048bd244028d3dfa8d118bc14e P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:14.809576 19938 tablet_bootstrap.cc:492] T c65a8cd3060c405c871270543b4623f7 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:14.816744 19938 tablet_bootstrap.cc:654] T c65a8cd3060c405c871270543b4623f7 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:14.819032 19274 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_193437ac4a794e79af9a3edb1c29a54d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:14.821206 19274 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_193437ac4a794e79af9a3edb1c29a54d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:14.822773 19939 raft_consensus.cc:359] T dc14c37aa393407a87af0d0eb8b88d82 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:14.833601 19939 raft_consensus.cc:385] T dc14c37aa393407a87af0d0eb8b88d82 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:14.833917 19939 raft_consensus.cc:740] T dc14c37aa393407a87af0d0eb8b88d82 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:14.836090 20011 ts_tablet_manager.cc:1929] T 28ac2912b94a4928967efb7a731c42b9 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:14.836617 20011 log.cc:1199] T 28ac2912b94a4928967efb7a731c42b9 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/28ac2912b94a4928967efb7a731c42b9
I20260925 13:56:14.837366 19942 tablet_bootstrap.cc:492] T 18a6d1c4eb7948149e709a380da50d93 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:14.840886 19942 ts_tablet_manager.cc:1403] T 18a6d1c4eb7948149e709a380da50d93 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.167s user 0.009s sys 0.004s
I20260925 13:56:14.837435 20011 ts_tablet_manager.cc:1950] T 28ac2912b94a4928967efb7a731c42b9 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:14.846344 19939 consensus_queue.cc:260] T dc14c37aa393407a87af0d0eb8b88d82 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
W20260925 13:56:14.850431 19629 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.193:38269, user_credentials={real_user=slave}} blocked reactor thread for 49579.8us
I20260925 13:56:14.853878 19942 raft_consensus.cc:359] T 18a6d1c4eb7948149e709a380da50d93 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:14.854825 19942 raft_consensus.cc:385] T 18a6d1c4eb7948149e709a380da50d93 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:14.855203 19942 raft_consensus.cc:740] T 18a6d1c4eb7948149e709a380da50d93 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
W20260925 13:56:14.855337 19266 consensus_peers.cc:597] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 -> Peer fc3953a2ba984a658630c423ec9b9728 (127.18.182.252:36181): Couldn't send request to peer fc3953a2ba984a658630c423ec9b9728. Status: Network error: Client connection negotiation failed: client connection to 127.18.182.252:36181: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260925 13:56:14.852415 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0a293e0094a4d9e937af6454e1ea16c" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:14.853201 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1513c0048bd244028d3dfa8d118bc14e" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:14.856066 19942 consensus_queue.cc:260] T 18a6d1c4eb7948149e709a380da50d93 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:14.859227 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 40) took 1870 ms. Trace:
I20260925 13:56:14.860430 20011 rpcz_store.cc:276] 0925 13:56:12.988880 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:12.989044 (+ 164us) service_pool.cc:224] Handling call
0925 13:56:14.658211 (+1669167us) ts_tablet_manager.cc:1151] Deleting tablet 28ac2912b94a4928967efb7a731c42b9
0925 13:56:14.658253 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:14.800700 (+142447us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:14.835861 (+ 35161us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:14.859125 (+ 23264us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29620,"tablet-delete.queue_time_us":1668620}
I20260925 13:56:14.861980 20011 tablet_replica.cc:333] T 293b017a83cc464a9f023a0923310796 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:14.864163 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 3b10ecb0e6064d42a2521c4c4e91c028 (table default.Table_f9ae699facd148ca8bae38d481d136df [id=7a20968885384e23a7a65f63337a67d3]) successfully deleted
I20260925 13:56:14.866526 19939 ts_tablet_manager.cc:1434] T dc14c37aa393407a87af0d0eb8b88d82 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.097s user 0.004s sys 0.000s
I20260925 13:56:14.866716 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 28ac2912b94a4928967efb7a731c42b9 (table default.Table_52f57b4f58b44f28989ffd9a4afe2eac [id=56afa81b40064727a7facfe3ccb22571]) successfully deleted
I20260925 13:56:14.871387 19938 tablet_bootstrap.cc:492] T c65a8cd3060c405c871270543b4623f7 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:14.871768 19938 ts_tablet_manager.cc:1403] T c65a8cd3060c405c871270543b4623f7 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.062s user 0.029s sys 0.026s
I20260925 13:56:14.874181 19938 raft_consensus.cc:359] T c65a8cd3060c405c871270543b4623f7 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:14.874783 19938 raft_consensus.cc:385] T c65a8cd3060c405c871270543b4623f7 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:14.875023 19938 raft_consensus.cc:740] T c65a8cd3060c405c871270543b4623f7 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:14.876101 20011 raft_consensus.cc:2243] T 293b017a83cc464a9f023a0923310796 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:14.875782 19938 consensus_queue.cc:260] T c65a8cd3060c405c871270543b4623f7 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:14.885913 19938 ts_tablet_manager.cc:1434] T c65a8cd3060c405c871270543b4623f7 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.014s user 0.005s sys 0.000s
I20260925 13:56:14.889211 19938 tablet_bootstrap.cc:492] T a88bbe1ebee7468d8f49dfd25637fe83 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:14.896726 19938 tablet_bootstrap.cc:654] T a88bbe1ebee7468d8f49dfd25637fe83 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:14.884721 19939 tablet_bootstrap.cc:492] T b36aebfe984f454ea910e84645c4ac7d P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:14.902184 19939 tablet_bootstrap.cc:654] T b36aebfe984f454ea910e84645c4ac7d P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:14.903076 20011 raft_consensus.cc:2272] T 293b017a83cc464a9f023a0923310796 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:14.905308 20011 ts_tablet_manager.cc:1916] T 293b017a83cc464a9f023a0923310796 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:14.911551 19942 ts_tablet_manager.cc:1434] T 18a6d1c4eb7948149e709a380da50d93 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.070s user 0.008s sys 0.000s
I20260925 13:56:14.913069 19942 tablet_bootstrap.cc:492] T 47521fb74f9c40ee999a7f704aa551c9 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:14.927205 19840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0a293e0094a4d9e937af6454e1ea16c" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:14.928504 19838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1513c0048bd244028d3dfa8d118bc14e" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:14.929230 19838 raft_consensus.cc:2393] T 1513c0048bd244028d3dfa8d118bc14e P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7c1b52895a14482fb54a6758f5054f8f in current term 1: Already voted for candidate 8f578869bc084a5d8a9a69f143954eb3 in this term.
I20260925 13:56:14.933369 19696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47521fb74f9c40ee999a7f704aa551c9" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:14.934458 20032 raft_consensus.cc:493] T bae8c5b6e1014731901bc3f336104784 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:14.922873 19278 catalog_manager.cc:5838] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "8f578869bc084a5d8a9a69f143954eb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:14.934902 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47521fb74f9c40ee999a7f704aa551c9" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:14.934893 20032 raft_consensus.cc:515] T bae8c5b6e1014731901bc3f336104784 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:14.935721 19278 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet 1513c0048bd244028d3dfa8d118bc14e (table default.Table_74b6076b22674a31bc58fef71cc9bfb1 [id=d81cf07e5f9a49d4bbf5aceda3da5a25]). Expected version 1 got 0
I20260925 13:56:14.936057 19693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a6d1c4eb7948149e709a380da50d93" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:14.936543 19693 raft_consensus.cc:3060] T 18a6d1c4eb7948149e709a380da50d93 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:14.936823 20032 leader_election.cc:290] T bae8c5b6e1014731901bc3f336104784 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:14.936553 19278 catalog_manager.cc:5838] T 7292d04b411c415394e9d0df85aaf579 P 8f578869bc084a5d8a9a69f143954eb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "8f578869bc084a5d8a9a69f143954eb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:14.939841 19694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1f15b9de572479aae6c2e644da3392a" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:14.940330 19694 raft_consensus.cc:3060] T c1f15b9de572479aae6c2e644da3392a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:14.943275 19938 tablet_bootstrap.cc:492] T a88bbe1ebee7468d8f49dfd25637fe83 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:14.943688 19938 ts_tablet_manager.cc:1403] T a88bbe1ebee7468d8f49dfd25637fe83 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.055s user 0.009s sys 0.020s
I20260925 13:56:14.943797 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47521fb74f9c40ee999a7f704aa551c9" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:14.945418 19691 raft_consensus.cc:3060] T b0a293e0094a4d9e937af6454e1ea16c P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:14.946086 19817 tablet_service.cc:1511] Processing CreateTablet for tablet bbfc4f99a5964e2ba6146e390e8dba44 (DEFAULT_TABLE table=default.Table_5d692242213f4df6863089b93b4e9636 [id=6d79107dc2d941058c5ecbe9a4410843]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:14.946062 19938 raft_consensus.cc:359] T a88bbe1ebee7468d8f49dfd25637fe83 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:14.946789 19938 raft_consensus.cc:385] T a88bbe1ebee7468d8f49dfd25637fe83 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:14.947039 19938 raft_consensus.cc:740] T a88bbe1ebee7468d8f49dfd25637fe83 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:14.947211 19817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbfc4f99a5964e2ba6146e390e8dba44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:14.947983 19690 raft_consensus.cc:3060] T 1513c0048bd244028d3dfa8d118bc14e P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:14.947676 19938 consensus_queue.cc:260] T a88bbe1ebee7468d8f49dfd25637fe83 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:14.953554 19818 tablet_service.cc:1511] Processing CreateTablet for tablet c2d333f892f84a258f8a148a90ce3b06 (DEFAULT_TABLE table=default.Table_d37fb0770cc84c09ab355bca02212825 [id=d06e00bf26bd4c47a9e27f7eb35835df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:14.954574 19818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2d333f892f84a258f8a148a90ce3b06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:14.957756 19631 leader_election.cc:304] T b0a293e0094a4d9e937af6454e1ea16c P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:14.959740 19631 leader_election.cc:304] T 1513c0048bd244028d3dfa8d118bc14e P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:14.937976 19278 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet 18a6d1c4eb7948149e709a380da50d93 (table default.Table_abc00b1f4c4c455e8086a46c395f2656 [id=6c0ec9465714408da93ff6e47827eb48]). Expected version 1 got 0
I20260925 13:56:14.962893 19278 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet b0a293e0094a4d9e937af6454e1ea16c (table default.Table_256401fd998c430db7f494657ef8e75d [id=b673ac6542c5403fbf0fad16c4917dfd]). Expected version 1 got 0
I20260925 13:56:14.963788 19278 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet 47521fb74f9c40ee999a7f704aa551c9 (table default.Table_3a3e48a0641a43a1a783c6ba6374a708 [id=e64fe44a237f48859a288a87a565ee5e]). Expected version 1 got 0
I20260925 13:56:14.964363 19276 catalog_manager.cc:6391] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: default.Table_256401fd998c430db7f494657ef8e75d [id=b673ac6542c5403fbf0fad16c4917dfd] alter complete (version 1)
I20260925 13:56:14.965420 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae8c5b6e1014731901bc3f336104784" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:14.968291 19675 tablet_service.cc:1511] Processing CreateTablet for tablet c2d333f892f84a258f8a148a90ce3b06 (DEFAULT_TABLE table=default.Table_d37fb0770cc84c09ab355bca02212825 [id=d06e00bf26bd4c47a9e27f7eb35835df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:14.969136 20011 ts_tablet_manager.cc:1929] T 293b017a83cc464a9f023a0923310796 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:14.969412 19675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2d333f892f84a258f8a148a90ce3b06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:14.972090 19676 tablet_service.cc:1511] Processing CreateTablet for tablet bbfc4f99a5964e2ba6146e390e8dba44 (DEFAULT_TABLE table=default.Table_5d692242213f4df6863089b93b4e9636 [id=6d79107dc2d941058c5ecbe9a4410843]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:14.973093 19676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbfc4f99a5964e2ba6146e390e8dba44. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:14.975214 19773 leader_election.cc:343] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:14.975627 19773 leader_election.cc:304] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:14.976226 20032 raft_consensus.cc:2749] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260925 13:56:14.980494 19938 ts_tablet_manager.cc:1434] T a88bbe1ebee7468d8f49dfd25637fe83 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.037s user 0.005s sys 0.000s
I20260925 13:56:14.981642 19693 raft_consensus.cc:2468] T 18a6d1c4eb7948149e709a380da50d93 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:14.984822 19938 tablet_bootstrap.cc:492] T 26644768f516412f8c95d68a76bb99c5 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:14.987793 19819 tablet_service.cc:1511] Processing CreateTablet for tablet 5943795bc8b24bb48b80d6d06d17750b (DEFAULT_TABLE table=default.Table_36b4dd2097aa4e2dbd58d98277e90281 [id=90bd44e8847549f2964a32f926ef1d99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:14.988811 19819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5943795bc8b24bb48b80d6d06d17750b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:14.989990 19276 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_93893aaa73a645b0939b9fd65f21d37a" } new_table_name: "default.Table_4d6d747ea4034b3ab48710e10ff4544f" modify_external_catalogs: true
I20260925 13:56:14.991161 19938 tablet_bootstrap.cc:654] T 26644768f516412f8c95d68a76bb99c5 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:14.991740 20011 log.cc:1199] T 293b017a83cc464a9f023a0923310796 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/293b017a83cc464a9f023a0923310796
I20260925 13:56:14.992089 19273 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_5d6ee0b4200c486ca1fdec8349603746" } new_table_name: "default.Table_ba97a54a19744b058cffd5638ffb97db" modify_external_catalogs: true
I20260925 13:56:14.992432 20011 ts_tablet_manager.cc:1950] T 293b017a83cc464a9f023a0923310796 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
W20260925 13:56:15.007905 19631 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 48272.6us
W20260925 13:56:15.011588 19773 leader_election.cc:343] T bae8c5b6e1014731901bc3f336104784 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:15.018731 19938 tablet_bootstrap.cc:492] T 26644768f516412f8c95d68a76bb99c5 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:15.019122 19938 ts_tablet_manager.cc:1403] T 26644768f516412f8c95d68a76bb99c5 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.034s user 0.007s sys 0.004s
I20260925 13:56:15.009243 19673 tablet_service.cc:1511] Processing CreateTablet for tablet 1832cff5eb3e4da4b9766ae1591df81a (DEFAULT_TABLE table=default.Table_9896a33d94e44d76ada6ec8d00c87ad7 [id=a5549f9e40c644afa1fafcc16d649f68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:15.012460 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 41) took 2004 ms. Trace:
I20260925 13:56:15.020905 19673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1832cff5eb3e4da4b9766ae1591df81a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:15.020407 20011 rpcz_store.cc:276] 0925 13:56:13.008055 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:13.008217 (+ 162us) service_pool.cc:224] Handling call
0925 13:56:14.861776 (+1853559us) ts_tablet_manager.cc:1151] Deleting tablet 293b017a83cc464a9f023a0923310796
0925 13:56:14.861858 (+ 82us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:14.951552 (+ 89694us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:14.969022 (+ 17470us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:15.012403 (+ 43381us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17789,"tablet-delete.queue_time_us":1852946}
I20260925 13:56:15.021586 20011 tablet_replica.cc:333] T 491b8b5c0f51408b882adee10ce80728 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:15.022207 20011 raft_consensus.cc:2243] T 491b8b5c0f51408b882adee10ce80728 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:15.022701 20011 raft_consensus.cc:2272] T 491b8b5c0f51408b882adee10ce80728 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:15.024626 20011 ts_tablet_manager.cc:1916] T 491b8b5c0f51408b882adee10ce80728 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:15.014483 19694 raft_consensus.cc:2468] T c1f15b9de572479aae6c2e644da3392a P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:15.033315 19942 tablet_bootstrap.cc:654] T 47521fb74f9c40ee999a7f704aa551c9 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:15.036949 19938 raft_consensus.cc:359] T 26644768f516412f8c95d68a76bb99c5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:15.037575 19938 raft_consensus.cc:385] T 26644768f516412f8c95d68a76bb99c5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:15.037863 19938 raft_consensus.cc:740] T 26644768f516412f8c95d68a76bb99c5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:15.038534 19938 consensus_queue.cc:260] T 26644768f516412f8c95d68a76bb99c5 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:15.054615 20032 raft_consensus.cc:493] T f1c6e2137700448a8ebf859a9ca9085d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:15.054921 19818 tablet_service.cc:1511] Processing CreateTablet for tablet 1832cff5eb3e4da4b9766ae1591df81a (DEFAULT_TABLE table=default.Table_9896a33d94e44d76ada6ec8d00c87ad7 [id=a5549f9e40c644afa1fafcc16d649f68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:15.055028 20032 raft_consensus.cc:515] T f1c6e2137700448a8ebf859a9ca9085d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:15.055958 19818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1832cff5eb3e4da4b9766ae1591df81a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:15.057070 20032 leader_election.cc:290] T f1c6e2137700448a8ebf859a9ca9085d P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:15.059219 19938 ts_tablet_manager.cc:1434] T 26644768f516412f8c95d68a76bb99c5 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.040s user 0.006s sys 0.001s
I20260925 13:56:15.063180 19938 tablet_bootstrap.cc:492] T f5a0bd0e205f4889be93739dfbd2a816 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:15.069885 19938 tablet_bootstrap.cc:654] T f5a0bd0e205f4889be93739dfbd2a816 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:15.034694 19672 tablet_service.cc:1511] Processing CreateTablet for tablet 5943795bc8b24bb48b80d6d06d17750b (DEFAULT_TABLE table=default.Table_36b4dd2097aa4e2dbd58d98277e90281 [id=90bd44e8847549f2964a32f926ef1d99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:15.076086 19672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5943795bc8b24bb48b80d6d06d17750b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:15.078706 19820 tablet_service.cc:1511] Processing CreateTablet for tablet b12a6c6cff024d70b53cc2c9169dd48b (DEFAULT_TABLE table=default.Table_02ff478a0f98431cacbe0cb5cfba6e8c [id=54685374ee92481fbbf6543c920b782d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:15.040194 19674 tablet_service.cc:1511] Processing CreateTablet for tablet b12a6c6cff024d70b53cc2c9169dd48b (DEFAULT_TABLE table=default.Table_02ff478a0f98431cacbe0cb5cfba6e8c [id=54685374ee92481fbbf6543c920b782d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:15.051649 20011 ts_tablet_manager.cc:1929] T 491b8b5c0f51408b882adee10ce80728 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:56:15.079555 20011 log.cc:1199] T 491b8b5c0f51408b882adee10ce80728 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/491b8b5c0f51408b882adee10ce80728
I20260925 13:56:15.079617 19820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b12a6c6cff024d70b53cc2c9169dd48b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:15.080119 19674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b12a6c6cff024d70b53cc2c9169dd48b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:15.080284 20011 ts_tablet_manager.cc:1950] T 491b8b5c0f51408b882adee10ce80728 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:15.082368 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 44) took 1812 ms. Trace:
I20260925 13:56:15.082671 20011 rpcz_store.cc:276] 0925 13:56:13.270004 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:13.272451 (+ 2447us) service_pool.cc:224] Handling call
0925 13:56:15.021457 (+1749006us) ts_tablet_manager.cc:1151] Deleting tablet 491b8b5c0f51408b882adee10ce80728
0925 13:56:15.021504 (+ 47us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:15.045089 (+ 23585us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:15.051526 (+ 6437us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:15.082311 (+ 30785us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1746340}
I20260925 13:56:15.083806 20011 tablet_replica.cc:333] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:15.052724 20100 raft_consensus.cc:2749] T 1513c0048bd244028d3dfa8d118bc14e P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:15.061408 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1c6e2137700448a8ebf859a9ca9085d" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:15.088557 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 491b8b5c0f51408b882adee10ce80728 (table default.Table_81fefa73a0904acb9b300de03fb48254 [id=dadfbd02a0a640949b1dc46c01db7bc8]) successfully deleted
I20260925 13:56:15.089162 19938 tablet_bootstrap.cc:492] T f5a0bd0e205f4889be93739dfbd2a816 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:15.089087 19671 tablet_service.cc:1511] Processing CreateTablet for tablet 1da6eadb1648479889027fa911f4676b (DEFAULT_TABLE table=default.Table_d3801e388b7b4b4d95188dc60386cce9 [id=479b481f813f4a60b2700734d4f51da3]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 13:56:15.089232 19773 leader_election.cc:343] T f1c6e2137700448a8ebf859a9ca9085d P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:15.089567 19938 ts_tablet_manager.cc:1403] T f5a0bd0e205f4889be93739dfbd2a816 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.027s user 0.012s sys 0.003s
I20260925 13:56:15.090102 19671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1da6eadb1648479889027fa911f4676b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:15.092118 19274 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_34fbf7824415492a8d02f09e2eb91bc6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:15.092263 19938 raft_consensus.cc:359] T f5a0bd0e205f4889be93739dfbd2a816 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:15.092828 19938 raft_consensus.cc:385] T f5a0bd0e205f4889be93739dfbd2a816 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:15.093345 19938 raft_consensus.cc:740] T f5a0bd0e205f4889be93739dfbd2a816 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
W20260925 13:56:15.093892 19274 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_34fbf7824415492a8d02f09e2eb91bc6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:15.093953 19938 consensus_queue.cc:260] T f5a0bd0e205f4889be93739dfbd2a816 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:15.101774 19272 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 2a1f34dba13143b097a38bab3782c404
I20260925 13:56:15.101969 19938 ts_tablet_manager.cc:1434] T f5a0bd0e205f4889be93739dfbd2a816 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.012s user 0.005s sys 0.000s
I20260925 13:56:15.107465 19938 tablet_bootstrap.cc:492] T 8239d0cd3b0d40bdb721e807c0f403d3 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:15.109673 19815 tablet_service.cc:1558] Processing DeleteTablet for tablet 2a1f34dba13143b097a38bab3782c404 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:14 UTC) from {username='slave'} at 127.0.0.1:51442
W20260925 13:56:15.111085 19266 catalog_manager.cc:5030] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): delete failed for tablet 2a1f34dba13143b097a38bab3782c404 because the tablet was not found. No further retry: Not found: Tablet not found: 2a1f34dba13143b097a38bab3782c404
I20260925 13:56:15.114912 19938 tablet_bootstrap.cc:654] T 8239d0cd3b0d40bdb721e807c0f403d3 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:15.126277 19670 tablet_service.cc:1558] Processing DeleteTablet for tablet 2a1f34dba13143b097a38bab3782c404 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:14 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:14.938556 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae8c5b6e1014731901bc3f336104784" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:14.941963 19531 tablet_service.cc:1511] Processing CreateTablet for tablet c2d333f892f84a258f8a148a90ce3b06 (DEFAULT_TABLE table=default.Table_d37fb0770cc84c09ab355bca02212825 [id=d06e00bf26bd4c47a9e27f7eb35835df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:15.140161 19531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2d333f892f84a258f8a148a90ce3b06. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:15.140632 19773 leader_election.cc:343] T bae8c5b6e1014731901bc3f336104784 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:15.141125 19773 leader_election.cc:304] T bae8c5b6e1014731901bc3f336104784 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:15.142282 20032 raft_consensus.cc:2749] T bae8c5b6e1014731901bc3f336104784 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:14.951989 19534 tablet_service.cc:1511] Processing CreateTablet for tablet bbfc4f99a5964e2ba6146e390e8dba44 (DEFAULT_TABLE table=default.Table_5d692242213f4df6863089b93b4e9636 [id=6d79107dc2d941058c5ecbe9a4410843]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:15.144390 19534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbfc4f99a5964e2ba6146e390e8dba44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:15.151999 20011 raft_consensus.cc:2243] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:15.152449 19530 tablet_service.cc:1511] Processing CreateTablet for tablet 1da6eadb1648479889027fa911f4676b (DEFAULT_TABLE table=default.Table_d3801e388b7b4b4d95188dc60386cce9 [id=479b481f813f4a60b2700734d4f51da3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:15.153443 19530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1da6eadb1648479889027fa911f4676b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:15.153604 19817 tablet_service.cc:1511] Processing CreateTablet for tablet 1da6eadb1648479889027fa911f4676b (DEFAULT_TABLE table=default.Table_d3801e388b7b4b4d95188dc60386cce9 [id=479b481f813f4a60b2700734d4f51da3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:15.154515 19817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1da6eadb1648479889027fa911f4676b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:15.154670 20011 pending_rounds.cc:70] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f: Trying to abort 1 pending ops.
I20260925 13:56:15.154925 20011 pending_rounds.cc:77] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7333251379008999424 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "f0cdc99cde2f4dabb819e3e171c5d8d9" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 1 new_table_name: "default.Table_5d6ee0b4200c486ca1fdec8349603746" dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" new_extra_config { } }
W20260925 13:56:15.156032 20011 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45948: Aborted: Op aborted
I20260925 13:56:15.156944 20011 raft_consensus.cc:2272] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 13:56:14.964154 19488 leader_election.cc:343] T 47521fb74f9c40ee999a7f704aa551c9 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:15.152016 19670 tablet_service.cc:1511] Processing CreateTablet for tablet e6e59d6f4f6e4de1be1cf450620cfa59 (DEFAULT_TABLE table=default.Table_546097946d054781b0acae5186c5d43d [id=912de4c02216461f8a69b880207d8652]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 13:56:15.157608 19488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.194:38641, user_credentials={real_user=slave}} blocked reactor thread for 193532us
I20260925 13:56:15.158565 19670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6e59d6f4f6e4de1be1cf450620cfa59. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:15.158540 19488 leader_election.cc:343] T 47521fb74f9c40ee999a7f704aa551c9 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:15.158891 19488 leader_election.cc:304] T 47521fb74f9c40ee999a7f704aa551c9 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3; no voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:14.981884 19535 tablet_service.cc:1511] Processing CreateTablet for tablet 1832cff5eb3e4da4b9766ae1591df81a (DEFAULT_TABLE table=default.Table_9896a33d94e44d76ada6ec8d00c87ad7 [id=a5549f9e40c644afa1fafcc16d649f68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:15.160915 19272 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_052e4e4ee53e4e408b18cd0ce8063d31"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:15.161355 19535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1832cff5eb3e4da4b9766ae1591df81a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:15.162364 19272 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_052e4e4ee53e4e408b18cd0ce8063d31 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:14.986420 19533 tablet_service.cc:1511] Processing CreateTablet for tablet 5943795bc8b24bb48b80d6d06d17750b (DEFAULT_TABLE table=default.Table_36b4dd2097aa4e2dbd58d98277e90281 [id=90bd44e8847549f2964a32f926ef1d99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:15.163981 19533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5943795bc8b24bb48b80d6d06d17750b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:15.165186 19942 tablet_bootstrap.cc:492] T 47521fb74f9c40ee999a7f704aa551c9 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:15.014837 19955 ts_tablet_manager.cc:1929] T 980d1264c0ea45f3b2b762509e0b2c77 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:15.059700 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1c6e2137700448a8ebf859a9ca9085d" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:15.165611 19942 ts_tablet_manager.cc:1403] T 47521fb74f9c40ee999a7f704aa551c9 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.253s user 0.010s sys 0.001s
I20260925 13:56:15.165711 20068 raft_consensus.cc:2749] T 47521fb74f9c40ee999a7f704aa551c9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260925 13:56:15.061220 19532 tablet_service.cc:1511] Processing CreateTablet for tablet b12a6c6cff024d70b53cc2c9169dd48b (DEFAULT_TABLE table=default.Table_02ff478a0f98431cacbe0cb5cfba6e8c [id=54685374ee92481fbbf6543c920b782d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:15.166680 19532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b12a6c6cff024d70b53cc2c9169dd48b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:15.107172 20136 consensus_queue.cc:1048] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:15.118832 19529 tablet_service.cc:1558] Processing DeleteTablet for tablet 2a1f34dba13143b097a38bab3782c404 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:14 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:15.169003 20068 consensus_queue.cc:1048] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:15.171646 19955 log.cc:1199] T 980d1264c0ea45f3b2b762509e0b2c77 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/980d1264c0ea45f3b2b762509e0b2c77
W20260925 13:56:15.173077 19773 leader_election.cc:343] T f1c6e2137700448a8ebf859a9ca9085d P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:15.173480 19773 leader_election.cc:304] T f1c6e2137700448a8ebf859a9ca9085d P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:15.175436 20032 raft_consensus.cc:2749] T f1c6e2137700448a8ebf859a9ca9085d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:15.176201 19955 ts_tablet_manager.cc:1950] T 980d1264c0ea45f3b2b762509e0b2c77 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:15.177917 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 54) took 1323 ms. Trace:
I20260925 13:56:15.178215 19955 rpcz_store.cc:276] 0925 13:56:13.854381 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:13.854531 (+ 150us) service_pool.cc:224] Handling call
0925 13:56:14.636786 (+782255us) ts_tablet_manager.cc:1151] Deleting tablet 980d1264c0ea45f3b2b762509e0b2c77
0925 13:56:14.636828 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:14.821476 (+184648us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:15.014723 (+193247us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:15.177865 (+163142us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":781720}
I20260925 13:56:15.180279 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet 980d1264c0ea45f3b2b762509e0b2c77 (table default.Table_8290acbcb9d84a48aad169c54251bc19 [id=90312b1b0c254ff9ad5693f561114a7f]) successfully deleted
W20260925 13:56:15.180437 20162 consensus_queue.cc:1175] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7c1b52895a14482fb54a6758f5054f8f" status { last_received { term: 1 index: 2 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 2 } }
W20260925 13:56:15.180943 19264 catalog_manager.cc:5142] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): alter failed for tablet f0cdc99cde2f4dabb819e3e171c5d8d9 (table default.Table_5d6ee0b4200c486ca1fdec8349603746 [id=7e7ee975d4bb457eb3cc4632e12f3cd5]): Aborted: Op aborted
I20260925 13:56:15.181895 19955 tablet_replica.cc:333] T f51c5ad86ce64891a6393c1dc8d414e5 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:15.182552 19955 raft_consensus.cc:2243] T f51c5ad86ce64891a6393c1dc8d414e5 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:15.182929 19955 pending_rounds.cc:70] T f51c5ad86ce64891a6393c1dc8d414e5 P 8f578869bc084a5d8a9a69f143954eb3: Trying to abort 1 pending ops.
I20260925 13:56:15.183123 19955 pending_rounds.cc:77] T f51c5ad86ce64891a6393c1dc8d414e5 P 8f578869bc084a5d8a9a69f143954eb3: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7333251376538038272 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "f51c5ad86ce64891a6393c1dc8d414e5" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 1 new_table_name: "default.Table_824449a97a864e3dbbf06e2ad678edc5" dest_uuid: "f790493ba7ad403993b678f2df2b55be" new_extra_config { } }
I20260925 13:56:15.184275 19955 raft_consensus.cc:2272] T f51c5ad86ce64891a6393c1dc8d414e5 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:15.191594 20011 ts_tablet_manager.cc:1916] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:15.197345 20153 raft_consensus.cc:2749] T b0a293e0094a4d9e937af6454e1ea16c P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:15.203554 19271 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 608bed5b04394bfca1776b5d6c502df1
I20260925 13:56:15.211527 19815 tablet_service.cc:1558] Processing DeleteTablet for tablet 608bed5b04394bfca1776b5d6c502df1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:14 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:15.212212 19959 tablet_replica.cc:333] T 608bed5b04394bfca1776b5d6c502df1 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:15.212901 19959 raft_consensus.cc:2243] T 608bed5b04394bfca1776b5d6c502df1 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:15.213289 19959 raft_consensus.cc:2272] T 608bed5b04394bfca1776b5d6c502df1 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:15.214891 19959 ts_tablet_manager.cc:1916] T 608bed5b04394bfca1776b5d6c502df1 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:15.227195 19942 raft_consensus.cc:359] T 47521fb74f9c40ee999a7f704aa551c9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:15.227833 19942 raft_consensus.cc:385] T 47521fb74f9c40ee999a7f704aa551c9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:15.228114 19942 raft_consensus.cc:740] T 47521fb74f9c40ee999a7f704aa551c9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:15.228811 19942 consensus_queue.cc:260] T 47521fb74f9c40ee999a7f704aa551c9 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:15.196703 20171 tablet.cc:1751] T b0a293e0094a4d9e937af6454e1ea16c P 7c1b52895a14482fb54a6758f5054f8f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:15.236277 19529 tablet_service.cc:1558] Processing DeleteTablet for tablet 608bed5b04394bfca1776b5d6c502df1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:14 UTC) from {username='slave'} at 127.0.0.1:40018
W20260925 13:56:15.255474 19264 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet bae8c5b6e1014731901bc3f336104784 (table default.Table_99593e98f3394811b4c91e234185cbef [id=0736e722eb8c43e9b6a151a7c2880fe3], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet bae8c5b6e1014731901bc3f336104784 consensus state has no leader
I20260925 13:56:15.258463 19672 tablet_service.cc:1558] Processing DeleteTablet for tablet 608bed5b04394bfca1776b5d6c502df1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:14 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:15.264454 19942 ts_tablet_manager.cc:1434] T 47521fb74f9c40ee999a7f704aa551c9 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.099s user 0.005s sys 0.000s
I20260925 13:56:15.268623 19942 tablet_bootstrap.cc:492] T dc14c37aa393407a87af0d0eb8b88d82 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:15.270215 19819 tablet_service.cc:1511] Processing CreateTablet for tablet e6e59d6f4f6e4de1be1cf450620cfa59 (DEFAULT_TABLE table=default.Table_546097946d054781b0acae5186c5d43d [id=912de4c02216461f8a69b880207d8652]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:15.271302 19819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6e59d6f4f6e4de1be1cf450620cfa59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:15.271333 19535 tablet_service.cc:1511] Processing CreateTablet for tablet e6e59d6f4f6e4de1be1cf450620cfa59 (DEFAULT_TABLE table=default.Table_546097946d054781b0acae5186c5d43d [id=912de4c02216461f8a69b880207d8652]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:15.272431 20032 raft_consensus.cc:493] T a88bbe1ebee7468d8f49dfd25637fe83 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:15.272925 20032 raft_consensus.cc:515] T a88bbe1ebee7468d8f49dfd25637fe83 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:15.283267 19271 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_1930b257c6824b38ab3c8cba088f9c42"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:15.285198 19271 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1930b257c6824b38ab3c8cba088f9c42 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:15.271852 19939 tablet_bootstrap.cc:492] T b36aebfe984f454ea910e84645c4ac7d P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:15.272154 19535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6e59d6f4f6e4de1be1cf450620cfa59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:15.296125 19939 ts_tablet_manager.cc:1403] T b36aebfe984f454ea910e84645c4ac7d P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.412s user 0.004s sys 0.005s
I20260925 13:56:15.297909 19942 tablet_bootstrap.cc:654] T dc14c37aa393407a87af0d0eb8b88d82 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:15.298512 19939 raft_consensus.cc:359] T b36aebfe984f454ea910e84645c4ac7d P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:15.299891 19955 ts_tablet_manager.cc:1916] T f51c5ad86ce64891a6393c1dc8d414e5 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:15.308660 19820 tablet_service.cc:1511] Processing CreateTablet for tablet 2a1f34dba13143b097a38bab3782c404 (DEFAULT_TABLE table=default.Table_3cf69a468c014c279a928211b6ff89a2 [id=fe0f78d1c6e44f589aede852f77f88fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:15.308719 19676 tablet_service.cc:1511] Processing CreateTablet for tablet 2a1f34dba13143b097a38bab3782c404 (DEFAULT_TABLE table=default.Table_3cf69a468c014c279a928211b6ff89a2 [id=fe0f78d1c6e44f589aede852f77f88fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:15.310032 19676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a1f34dba13143b097a38bab3782c404. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:15.307828 19534 tablet_service.cc:1511] Processing CreateTablet for tablet 2a1f34dba13143b097a38bab3782c404 (DEFAULT_TABLE table=default.Table_3cf69a468c014c279a928211b6ff89a2 [id=fe0f78d1c6e44f589aede852f77f88fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:15.311486 19534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a1f34dba13143b097a38bab3782c404. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:15.299172 19939 raft_consensus.cc:385] T b36aebfe984f454ea910e84645c4ac7d P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:15.317554 19939 raft_consensus.cc:740] T b36aebfe984f454ea910e84645c4ac7d P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:15.321213 19939 consensus_queue.cc:260] T b36aebfe984f454ea910e84645c4ac7d P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:15.329545 20011 ts_tablet_manager.cc:1929] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:56:15.329869 20011 log.cc:1199] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/f0cdc99cde2f4dabb819e3e171c5d8d9
I20260925 13:56:15.330507 20011 ts_tablet_manager.cc:1950] T f0cdc99cde2f4dabb819e3e171c5d8d9 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:15.336485 20032 leader_election.cc:290] T a88bbe1ebee7468d8f49dfd25637fe83 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:15.337056 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a88bbe1ebee7468d8f49dfd25637fe83" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
W20260925 13:56:15.341393 19773 leader_election.cc:343] T a88bbe1ebee7468d8f49dfd25637fe83 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:15.342321 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a88bbe1ebee7468d8f49dfd25637fe83" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:15.344108 19773 leader_election.cc:343] T a88bbe1ebee7468d8f49dfd25637fe83 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:15.344259 19939 ts_tablet_manager.cc:1434] T b36aebfe984f454ea910e84645c4ac7d P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.048s user 0.006s sys 0.000s
I20260925 13:56:15.344504 19773 leader_election.cc:304] T a88bbe1ebee7468d8f49dfd25637fe83 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:15.345083 19939 tablet_bootstrap.cc:492] T bae8c5b6e1014731901bc3f336104784 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:15.347908 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 46) took 1916 ms. Trace:
I20260925 13:56:15.348218 20011 rpcz_store.cc:276] 0925 13:56:13.431120 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:13.433384 (+ 2264us) service_pool.cc:224] Handling call
0925 13:56:15.083658 (+1650274us) ts_tablet_manager.cc:1151] Deleting tablet f0cdc99cde2f4dabb819e3e171c5d8d9
0925 13:56:15.083710 (+ 52us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:15.154096 (+ 70386us) spinlock_profiling.cc:243] Waited 67.3 ms on lock 0x7b54000b0254. stack: 00007f1ff9889323 00007f1ff9889220 00007f1ff942d574 0000000000516964 00000000005168f9 00007f2000986aa1 00007f2000a25b22 00007f2000e6650c 00007f2001acfc3a 00007f2001b00960 00007f2001ae6e6d 00007f2001ae6e19 00007f2001ae6da9 00007f2001ae6d71 00007f2001ae5f9d 00000000005bd4e4
0925 13:56:15.248488 (+ 94392us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:15.329441 (+ 80953us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:15.347851 (+ 18410us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":154807168,"tablet-delete.queue_time_us":1649759}
I20260925 13:56:15.349334 20032 raft_consensus.cc:493] T c65a8cd3060c405c871270543b4623f7 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:15.351637 19820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a1f34dba13143b097a38bab3782c404. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:15.356084 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet f0cdc99cde2f4dabb819e3e171c5d8d9 (table default.Table_5d6ee0b4200c486ca1fdec8349603746 [id=7e7ee975d4bb457eb3cc4632e12f3cd5]) successfully deleted
I20260925 13:56:15.349740 20032 raft_consensus.cc:515] T c65a8cd3060c405c871270543b4623f7 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:15.358734 20032 leader_election.cc:290] T c65a8cd3060c405c871270543b4623f7 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:15.359150 20032 raft_consensus.cc:2749] T a88bbe1ebee7468d8f49dfd25637fe83 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:15.361114 19938 tablet_bootstrap.cc:492] T 8239d0cd3b0d40bdb721e807c0f403d3 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:15.361479 19938 ts_tablet_manager.cc:1403] T 8239d0cd3b0d40bdb721e807c0f403d3 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.254s user 0.009s sys 0.043s
I20260925 13:56:15.363554 20011 tablet_replica.cc:333] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:15.364200 20011 raft_consensus.cc:2243] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:15.365075 20011 raft_consensus.cc:2272] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:15.366117 19277 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet a88bbe1ebee7468d8f49dfd25637fe83
I20260925 13:56:15.366873 20011 ts_tablet_manager.cc:1916] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:15.366595 19938 raft_consensus.cc:359] T 8239d0cd3b0d40bdb721e807c0f403d3 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:15.367166 19938 raft_consensus.cc:385] T 8239d0cd3b0d40bdb721e807c0f403d3 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:15.367420 19938 raft_consensus.cc:740] T 8239d0cd3b0d40bdb721e807c0f403d3 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:15.368233 19277 catalog_manager.cc:6934] ReplaceTablet: tablet a88bbe1ebee7468d8f49dfd25637fe83 deleted and replaced by tablet a26f5a02311c435a8c2434fd030c80a6
I20260925 13:56:15.370589 19938 consensus_queue.cc:260] T 8239d0cd3b0d40bdb721e807c0f403d3 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:15.373639 19942 tablet_bootstrap.cc:492] T dc14c37aa393407a87af0d0eb8b88d82 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:15.374027 19942 ts_tablet_manager.cc:1403] T dc14c37aa393407a87af0d0eb8b88d82 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.106s user 0.005s sys 0.004s
I20260925 13:56:15.374843 19534 tablet_service.cc:1558] Processing DeleteTablet for tablet a88bbe1ebee7468d8f49dfd25637fe83 with delete_type TABLET_DATA_DELETED (replaced by tablet a26f5a02311c435a8c2434fd030c80a6) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:15.375619 19672 tablet_service.cc:1558] Processing DeleteTablet for tablet a88bbe1ebee7468d8f49dfd25637fe83 with delete_type TABLET_DATA_DELETED (replaced by tablet a26f5a02311c435a8c2434fd030c80a6) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:15.377112 19942 raft_consensus.cc:359] T dc14c37aa393407a87af0d0eb8b88d82 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:15.377744 19942 raft_consensus.cc:385] T dc14c37aa393407a87af0d0eb8b88d82 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:15.378050 19942 raft_consensus.cc:740] T dc14c37aa393407a87af0d0eb8b88d82 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:15.378290 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65a8cd3060c405c871270543b4623f7" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:15.379336 19942 consensus_queue.cc:260] T dc14c37aa393407a87af0d0eb8b88d82 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:15.380592 19815 tablet_service.cc:1558] Processing DeleteTablet for tablet a88bbe1ebee7468d8f49dfd25637fe83 with delete_type TABLET_DATA_DELETED (replaced by tablet a26f5a02311c435a8c2434fd030c80a6) from {username='slave'} at 127.0.0.1:51442
W20260925 13:56:15.381278 19773 leader_election.cc:343] T c65a8cd3060c405c871270543b4623f7 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:15.382032 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65a8cd3060c405c871270543b4623f7" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:15.382369 19939 tablet_bootstrap.cc:654] T bae8c5b6e1014731901bc3f336104784 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
W20260925 13:56:15.383302 19773 leader_election.cc:343] T c65a8cd3060c405c871270543b4623f7 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:15.383693 19773 leader_election.cc:304] T c65a8cd3060c405c871270543b4623f7 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:15.384306 20032 raft_consensus.cc:2749] T c65a8cd3060c405c871270543b4623f7 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:15.385986 19277 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_07d4243785b14434b731eb80cf4f242c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:15.386687 19938 ts_tablet_manager.cc:1434] T 8239d0cd3b0d40bdb721e807c0f403d3 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.025s user 0.007s sys 0.000s
W20260925 13:56:15.387454 19277 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_07d4243785b14434b731eb80cf4f242c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:15.394414 19938 tablet_bootstrap.cc:492] T c2d333f892f84a258f8a148a90ce3b06 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:15.402709 19942 ts_tablet_manager.cc:1434] T dc14c37aa393407a87af0d0eb8b88d82 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.028s user 0.007s sys 0.000s
I20260925 13:56:15.405766 20153 raft_consensus.cc:493] T 00b500182bb04a72b80cfa9103e85fa9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:15.406214 20153 raft_consensus.cc:515] T 00b500182bb04a72b80cfa9103e85fa9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:15.408588 19938 tablet_bootstrap.cc:654] T c2d333f892f84a258f8a148a90ce3b06 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:15.408474 20153 leader_election.cc:290] T 00b500182bb04a72b80cfa9103e85fa9 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:15.409927 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b500182bb04a72b80cfa9103e85fa9" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:15.410506 19554 raft_consensus.cc:2468] T 00b500182bb04a72b80cfa9103e85fa9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
W20260925 13:56:15.411154 19631 leader_election.cc:343] T 00b500182bb04a72b80cfa9103e85fa9 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 00b500182bb04a72b80cfa9103e85fa9
I20260925 13:56:15.413370 19629 leader_election.cc:304] T 00b500182bb04a72b80cfa9103e85fa9 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3; no voters: f790493ba7ad403993b678f2df2b55be
I20260925 13:56:15.421221 20153 raft_consensus.cc:2804] T 00b500182bb04a72b80cfa9103e85fa9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:15.421639 20153 raft_consensus.cc:493] T 00b500182bb04a72b80cfa9103e85fa9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:15.422065 20153 raft_consensus.cc:3060] T 00b500182bb04a72b80cfa9103e85fa9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
W20260925 13:56:15.425479 19488 consensus_peers.cc:597] T 7292d04b411c415394e9d0df85aaf579 P 8f578869bc084a5d8a9a69f143954eb3 -> Peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Couldn't send request to peer 7c1b52895a14482fb54a6758f5054f8f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260925 13:56:15.436271 19276 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e3cf9676160c4af08f1f62ae952b4c57 has no consensus state
W20260925 13:56:15.443958 19273 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f514fa52e7524cd3ba07f3f65de532d4 consensus state has no leader
I20260925 13:56:15.445089 19273 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_731e8683c6714e22a725eae1b86ff4ec" } modify_external_catalogs: true
I20260925 13:56:15.445631 19273 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_731e8683c6714e22a725eae1b86ff4ec" } modify_external_catalogs: true
I20260925 13:56:15.450233 19942 tablet_bootstrap.cc:492] T 980d1264c0ea45f3b2b762509e0b2c77 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:15.451077 19676 tablet_service.cc:1511] Processing CreateTablet for tablet 8dc5ea164d6749b0926c63d6bda9dcc5 (DEFAULT_TABLE table=default.Table_9b29863ad2ca4d698aebe4b655b19e50 [id=e49e015b173c48c1b6ef8112f60b33e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:15.452313 19676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dc5ea164d6749b0926c63d6bda9dcc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:15.456707 19534 tablet_service.cc:1511] Processing CreateTablet for tablet 8dc5ea164d6749b0926c63d6bda9dcc5 (DEFAULT_TABLE table=default.Table_9b29863ad2ca4d698aebe4b655b19e50 [id=e49e015b173c48c1b6ef8112f60b33e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:15.457674 19534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dc5ea164d6749b0926c63d6bda9dcc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:15.461707 20153 raft_consensus.cc:515] T 00b500182bb04a72b80cfa9103e85fa9 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:15.464179 19819 tablet_service.cc:1511] Processing CreateTablet for tablet 8dc5ea164d6749b0926c63d6bda9dcc5 (DEFAULT_TABLE table=default.Table_9b29863ad2ca4d698aebe4b655b19e50 [id=e49e015b173c48c1b6ef8112f60b33e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:15.465322 19819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dc5ea164d6749b0926c63d6bda9dcc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:15.478439 20032 raft_consensus.cc:493] T f5a0bd0e205f4889be93739dfbd2a816 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:15.478873 20032 raft_consensus.cc:515] T f5a0bd0e205f4889be93739dfbd2a816 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:15.480659 20032 leader_election.cc:290] T f5a0bd0e205f4889be93739dfbd2a816 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:15.480944 19939 tablet_bootstrap.cc:492] T bae8c5b6e1014731901bc3f336104784 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:15.481485 20153 leader_election.cc:290] T 00b500182bb04a72b80cfa9103e85fa9 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:15.481339 19939 ts_tablet_manager.cc:1403] T bae8c5b6e1014731901bc3f336104784 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.136s user 0.002s sys 0.030s
I20260925 13:56:15.481657 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00b500182bb04a72b80cfa9103e85fa9" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:15.482167 19554 raft_consensus.cc:3060] T 00b500182bb04a72b80cfa9103e85fa9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:15.482764 19942 tablet_bootstrap.cc:654] T 980d1264c0ea45f3b2b762509e0b2c77 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:15.484089 19939 raft_consensus.cc:359] T bae8c5b6e1014731901bc3f336104784 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:15.484660 19939 raft_consensus.cc:385] T bae8c5b6e1014731901bc3f336104784 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:15.484884 19939 raft_consensus.cc:740] T bae8c5b6e1014731901bc3f336104784 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:15.485389 19939 consensus_queue.cc:260] T bae8c5b6e1014731901bc3f336104784 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:15.485668 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a0bd0e205f4889be93739dfbd2a816" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
W20260925 13:56:15.487064 19773 leader_election.cc:343] T f5a0bd0e205f4889be93739dfbd2a816 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:15.487741 19631 leader_election.cc:343] T 00b500182bb04a72b80cfa9103e85fa9 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 00b500182bb04a72b80cfa9103e85fa9
I20260925 13:56:15.487948 20011 ts_tablet_manager.cc:1929] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:15.488300 20011 log.cc:1199] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/51a4c21e06ca4b5b971cf8f04ffa52a9
I20260925 13:56:15.488866 20011 ts_tablet_manager.cc:1950] T 51a4c21e06ca4b5b971cf8f04ffa52a9 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:15.490626 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 47) took 1906 ms. Trace:
I20260925 13:56:15.492151 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a0bd0e205f4889be93739dfbd2a816" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:15.493463 19773 leader_election.cc:343] T f5a0bd0e205f4889be93739dfbd2a816 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:15.493692 19939 ts_tablet_manager.cc:1434] T bae8c5b6e1014731901bc3f336104784 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.012s user 0.004s sys 0.000s
I20260925 13:56:15.493799 19773 leader_election.cc:304] T f5a0bd0e205f4889be93739dfbd2a816 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:15.494751 20032 raft_consensus.cc:2749] T f5a0bd0e205f4889be93739dfbd2a816 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:15.496824 19938 tablet_bootstrap.cc:492] T c2d333f892f84a258f8a148a90ce3b06 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:15.497210 19938 ts_tablet_manager.cc:1403] T c2d333f892f84a258f8a148a90ce3b06 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.103s user 0.014s sys 0.015s
I20260925 13:56:15.490934 20011 rpcz_store.cc:276] 0925 13:56:13.584412 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:13.584529 (+ 117us) service_pool.cc:224] Handling call
0925 13:56:15.363388 (+1778859us) ts_tablet_manager.cc:1151] Deleting tablet 51a4c21e06ca4b5b971cf8f04ffa52a9
0925 13:56:15.363453 (+ 65us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:15.390435 (+ 26982us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:15.487846 (+ 97411us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:15.490575 (+ 2729us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1771586}
I20260925 13:56:15.498628 19554 raft_consensus.cc:2468] T 00b500182bb04a72b80cfa9103e85fa9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
I20260925 13:56:15.508697 19939 tablet_bootstrap.cc:492] T 608bed5b04394bfca1776b5d6c502df1 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:15.509971 19629 leader_election.cc:304] T 00b500182bb04a72b80cfa9103e85fa9 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3; no voters: f790493ba7ad403993b678f2df2b55be
I20260925 13:56:15.511471 20153 raft_consensus.cc:2804] T 00b500182bb04a72b80cfa9103e85fa9 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:15.511938 20153 raft_consensus.cc:697] T 00b500182bb04a72b80cfa9103e85fa9 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Becoming Leader. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Running, Role: LEADER
I20260925 13:56:15.513062 19942 tablet_bootstrap.cc:492] T 980d1264c0ea45f3b2b762509e0b2c77 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:15.513398 19942 ts_tablet_manager.cc:1403] T 980d1264c0ea45f3b2b762509e0b2c77 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.063s user 0.009s sys 0.000s
I20260925 13:56:15.512840 20153 consensus_queue.cc:237] T 00b500182bb04a72b80cfa9103e85fa9 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:15.513576 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 48) took 1660 ms. Trace:
I20260925 13:56:15.513881 20011 rpcz_store.cc:276] 0925 13:56:13.847149 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:13.847313 (+ 164us) service_pool.cc:224] Handling call
0925 13:56:15.506928 (+1659615us) ts_tablet_manager.cc:1151] Deleting tablet 980d1264c0ea45f3b2b762509e0b2c77
0925 13:56:15.506989 (+ 61us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:15.507295 (+ 306us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1659004}
W20260925 13:56:15.514992 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 980d1264c0ea45f3b2b762509e0b2c77 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 980d1264c0ea45f3b2b762509e0b2c77 already in progress: creating tablet
I20260925 13:56:15.516237 19938 raft_consensus.cc:359] T c2d333f892f84a258f8a148a90ce3b06 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:15.516819 19938 raft_consensus.cc:385] T c2d333f892f84a258f8a148a90ce3b06 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:15.517073 19938 raft_consensus.cc:740] T c2d333f892f84a258f8a148a90ce3b06 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:15.521991 19939 tablet_bootstrap.cc:654] T 608bed5b04394bfca1776b5d6c502df1 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:15.523789 20011 tablet_replica.cc:333] T f51c5ad86ce64891a6393c1dc8d414e5 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:15.524571 20011 raft_consensus.cc:2243] T f51c5ad86ce64891a6393c1dc8d414e5 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:15.524616 19672 tablet_service.cc:1511] Processing CreateTablet for tablet 3e6c8b9bbf4549b3a8685f886973ed9c (DEFAULT_TABLE table=default.Table_d0dadd9b31b143fd8dd5adcb7f752c5f [id=beee0ac537e44859b0f7c76d6808b0e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:15.525105 20011 pending_rounds.cc:70] T f51c5ad86ce64891a6393c1dc8d414e5 P 7c1b52895a14482fb54a6758f5054f8f: Trying to abort 1 pending ops.
I20260925 13:56:15.525651 19672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e6c8b9bbf4549b3a8685f886973ed9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:15.525341 20011 pending_rounds.cc:77] T f51c5ad86ce64891a6393c1dc8d414e5 P 7c1b52895a14482fb54a6758f5054f8f: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7333251376538038272 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "f51c5ad86ce64891a6393c1dc8d414e5" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 1 new_table_name: "default.Table_824449a97a864e3dbbf06e2ad678edc5" dest_uuid: "f790493ba7ad403993b678f2df2b55be" new_extra_config { } }
I20260925 13:56:15.526527 20011 raft_consensus.cc:2272] T f51c5ad86ce64891a6393c1dc8d414e5 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:15.516023 19942 raft_consensus.cc:359] T 980d1264c0ea45f3b2b762509e0b2c77 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:15.531850 19942 raft_consensus.cc:385] T 980d1264c0ea45f3b2b762509e0b2c77 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:15.532104 19942 raft_consensus.cc:740] T 980d1264c0ea45f3b2b762509e0b2c77 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:15.537796 20011 ts_tablet_manager.cc:1916] T f51c5ad86ce64891a6393c1dc8d414e5 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:15.538882 19942 consensus_queue.cc:260] T 980d1264c0ea45f3b2b762509e0b2c77 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:15.542025 19942 ts_tablet_manager.cc:1434] T 980d1264c0ea45f3b2b762509e0b2c77 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.028s user 0.006s sys 0.000s
I20260925 13:56:15.542770 19942 tablet_bootstrap.cc:492] T 7292d04b411c415394e9d0df85aaf579 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:15.555430 19279 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 1513c0048bd244028d3dfa8d118bc14e (table default.Table_74b6076b22674a31bc58fef71cc9bfb1 [id=d81cf07e5f9a49d4bbf5aceda3da5a25]). Expected version 1 got 0
I20260925 13:56:15.517674 19938 consensus_queue.cc:260] T c2d333f892f84a258f8a148a90ce3b06 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:15.567476 19942 tablet_bootstrap.cc:654] T 7292d04b411c415394e9d0df85aaf579 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:15.569852 20123 consensus_queue.cc:1048] T 7292d04b411c415394e9d0df85aaf579 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:15.570194 19938 ts_tablet_manager.cc:1434] T c2d333f892f84a258f8a148a90ce3b06 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.073s user 0.002s sys 0.005s
I20260925 13:56:15.563164 20011 ts_tablet_manager.cc:1929] T f51c5ad86ce64891a6393c1dc8d414e5 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260925 13:56:15.572157 19279 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.194:34125 (request call id 22) took 1127 ms (client timeout 1000 ms). Trace:
I20260925 13:56:15.572553 20011 log.cc:1199] T f51c5ad86ce64891a6393c1dc8d414e5 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/f51c5ad86ce64891a6393c1dc8d414e5
W20260925 13:56:15.572580 19279 rpcz_store.cc:269] 0925 13:56:14.444704 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:14.444808 (+ 104us) service_pool.cc:224] Handling call
0925 13:56:15.572074 (+1127266us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1}
I20260925 13:56:15.573271 20011 ts_tablet_manager.cc:1950] T f51c5ad86ce64891a6393c1dc8d414e5 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:15.575281 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 51) took 1394 ms. Trace:
I20260925 13:56:15.575676 20011 rpcz_store.cc:276] 0925 13:56:14.181262 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:14.181414 (+ 152us) service_pool.cc:224] Handling call
0925 13:56:15.523594 (+1342180us) ts_tablet_manager.cc:1151] Deleting tablet f51c5ad86ce64891a6393c1dc8d414e5
0925 13:56:15.523678 (+ 84us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:15.550640 (+ 26962us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:15.563059 (+ 12419us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:15.575217 (+ 12158us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1341656}
I20260925 13:56:15.577179 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 52) took 1392 ms. Trace:
I20260925 13:56:15.577188 19280 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 1513c0048bd244028d3dfa8d118bc14e (table default.Table_74b6076b22674a31bc58fef71cc9bfb1 [id=d81cf07e5f9a49d4bbf5aceda3da5a25]). Expected version 1 got 0
I20260925 13:56:15.577791 19942 tablet_bootstrap.cc:492] T 7292d04b411c415394e9d0df85aaf579 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:15.577790 19280 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 26644768f516412f8c95d68a76bb99c5 (table default.Table_bd52a86b8ff14bdbb06fcf88a60b1e82 [id=203024d67e444f839391c8670d7c7d3b]). Expected version 2 got 0
I20260925 13:56:15.578151 19942 ts_tablet_manager.cc:1403] T 7292d04b411c415394e9d0df85aaf579 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.036s user 0.005s sys 0.004s
I20260925 13:56:15.584323 19938 tablet_bootstrap.cc:492] T bbfc4f99a5964e2ba6146e390e8dba44 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:15.584441 19942 raft_consensus.cc:359] T 7292d04b411c415394e9d0df85aaf579 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:15.585035 19942 raft_consensus.cc:385] T 7292d04b411c415394e9d0df85aaf579 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:15.585273 19942 raft_consensus.cc:740] T 7292d04b411c415394e9d0df85aaf579 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:15.585842 19942 consensus_queue.cc:260] T 7292d04b411c415394e9d0df85aaf579 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:15.589324 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet f51c5ad86ce64891a6393c1dc8d414e5 (table default.Table_824449a97a864e3dbbf06e2ad678edc5 [id=af708d3ba4c54ba19c24eca862bbddc1]) successfully deleted
I20260925 13:56:15.593375 20154 raft_consensus.cc:493] T 47521fb74f9c40ee999a7f704aa551c9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:15.593780 20154 raft_consensus.cc:515] T 47521fb74f9c40ee999a7f704aa551c9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:15.595729 19942 ts_tablet_manager.cc:1434] T 7292d04b411c415394e9d0df85aaf579 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.017s user 0.005s sys 0.000s
I20260925 13:56:15.595772 20154 leader_election.cc:290] T 47521fb74f9c40ee999a7f704aa551c9 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:15.596060 19938 tablet_bootstrap.cc:654] T bbfc4f99a5964e2ba6146e390e8dba44 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:15.596545 19942 tablet_bootstrap.cc:492] T b36aebfe984f454ea910e84645c4ac7d P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:15.598899 19838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47521fb74f9c40ee999a7f704aa551c9" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:15.599659 19838 raft_consensus.cc:2393] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7c1b52895a14482fb54a6758f5054f8f in current term 1: Already voted for candidate f790493ba7ad403993b678f2df2b55be in this term.
I20260925 13:56:15.577489 20011 rpcz_store.cc:276] 0925 13:56:14.184785 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:14.184914 (+ 129us) service_pool.cc:224] Handling call
0925 13:56:15.576874 (+1391960us) ts_tablet_manager.cc:1151] Deleting tablet ff3d09046b854fdaab2f95feff589ac1
0925 13:56:15.576933 (+ 59us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:15.577129 (+ 196us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1391479}
I20260925 13:56:15.616546 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 54) took 1415 ms. Trace:
I20260925 13:56:15.616815 20011 rpcz_store.cc:276] 0925 13:56:14.200837 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:14.200963 (+ 126us) service_pool.cc:224] Handling call
0925 13:56:15.616280 (+1415317us) ts_tablet_manager.cc:1151] Deleting tablet d9455b98734147fb9e2d47c69513424a
0925 13:56:15.616336 (+ 56us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:15.616499 (+ 163us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1414729}
I20260925 13:56:15.617835 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 55) took 1386 ms. Trace:
I20260925 13:56:15.618090 20011 rpcz_store.cc:276] 0925 13:56:14.231923 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:14.262478 (+ 30555us) service_pool.cc:224] Handling call
0925 13:56:15.617523 (+1355045us) ts_tablet_manager.cc:1151] Deleting tablet 8239d0cd3b0d40bdb721e807c0f403d3
0925 13:56:15.617565 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:15.617791 (+ 226us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1354522}
I20260925 13:56:15.618983 20011 tablet_replica.cc:333] T c0f05269ea6c43a98d2137eae7a9a401 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
W20260925 13:56:15.623950 19264 catalog_manager.cc:5030] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet ff3d09046b854fdaab2f95feff589ac1 because the tablet was not found. No further retry: Not found: Tablet not found: ff3d09046b854fdaab2f95feff589ac1
W20260925 13:56:15.626123 19264 catalog_manager.cc:5030] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet d9455b98734147fb9e2d47c69513424a because the tablet was not found. No further retry: Not found: Tablet not found: d9455b98734147fb9e2d47c69513424a
W20260925 13:56:15.626780 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 8239d0cd3b0d40bdb721e807c0f403d3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8239d0cd3b0d40bdb721e807c0f403d3 already in progress: creating tablet
I20260925 13:56:15.626878 19942 tablet_bootstrap.cc:654] T b36aebfe984f454ea910e84645c4ac7d P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:15.629050 20011 raft_consensus.cc:2243] T c0f05269ea6c43a98d2137eae7a9a401 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:15.629562 20011 raft_consensus.cc:2272] T c0f05269ea6c43a98d2137eae7a9a401 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:15.642967 20011 ts_tablet_manager.cc:1916] T c0f05269ea6c43a98d2137eae7a9a401 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:15.642736 20136 consensus_queue.cc:1048] T c1f15b9de572479aae6c2e644da3392a P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:15.647327 19942 tablet_bootstrap.cc:492] T b36aebfe984f454ea910e84645c4ac7d P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:15.652149 19942 ts_tablet_manager.cc:1403] T b36aebfe984f454ea910e84645c4ac7d P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.056s user 0.005s sys 0.005s
I20260925 13:56:15.653437 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47521fb74f9c40ee999a7f704aa551c9" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:15.654242 19554 raft_consensus.cc:2393] T 47521fb74f9c40ee999a7f704aa551c9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7c1b52895a14482fb54a6758f5054f8f in current term 1: Already voted for candidate 8f578869bc084a5d8a9a69f143954eb3 in this term.
I20260925 13:56:15.655390 19629 leader_election.cc:304] T 47521fb74f9c40ee999a7f704aa551c9 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:15.657120 20157 raft_consensus.cc:3060] T 47521fb74f9c40ee999a7f704aa551c9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:15.657339 19942 raft_consensus.cc:359] T b36aebfe984f454ea910e84645c4ac7d P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:15.657863 19942 raft_consensus.cc:385] T b36aebfe984f454ea910e84645c4ac7d P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:15.658075 19942 raft_consensus.cc:740] T b36aebfe984f454ea910e84645c4ac7d P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:15.658605 19942 consensus_queue.cc:260] T b36aebfe984f454ea910e84645c4ac7d P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:15.660403 19942 ts_tablet_manager.cc:1434] T b36aebfe984f454ea910e84645c4ac7d P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.008s user 0.000s sys 0.004s
I20260925 13:56:15.667034 20157 raft_consensus.cc:2749] T 47521fb74f9c40ee999a7f704aa551c9 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:15.668370 19938 tablet_bootstrap.cc:492] T bbfc4f99a5964e2ba6146e390e8dba44 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:15.668773 19938 ts_tablet_manager.cc:1403] T bbfc4f99a5964e2ba6146e390e8dba44 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.085s user 0.002s sys 0.011s
I20260925 13:56:15.671615 19938 raft_consensus.cc:359] T bbfc4f99a5964e2ba6146e390e8dba44 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:15.672323 19938 raft_consensus.cc:385] T bbfc4f99a5964e2ba6146e390e8dba44 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:15.672765 19938 raft_consensus.cc:740] T bbfc4f99a5964e2ba6146e390e8dba44 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:15.673480 19938 consensus_queue.cc:260] T bbfc4f99a5964e2ba6146e390e8dba44 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:15.675729 20032 raft_consensus.cc:493] T 4d3dbd9796564472b44a31ccaf56d898 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:15.676172 20032 raft_consensus.cc:515] T 4d3dbd9796564472b44a31ccaf56d898 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:15.677975 20032 leader_election.cc:290] T 4d3dbd9796564472b44a31ccaf56d898 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:15.679939 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d3dbd9796564472b44a31ccaf56d898" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:15.681366 19773 leader_election.cc:343] T 4d3dbd9796564472b44a31ccaf56d898 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:15.682588 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d3dbd9796564472b44a31ccaf56d898" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:15.683320 19938 ts_tablet_manager.cc:1434] T bbfc4f99a5964e2ba6146e390e8dba44 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.014s user 0.006s sys 0.001s
W20260925 13:56:15.684015 19773 leader_election.cc:343] T 4d3dbd9796564472b44a31ccaf56d898 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:15.684414 19773 leader_election.cc:304] T 4d3dbd9796564472b44a31ccaf56d898 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:15.685334 20032 raft_consensus.cc:2749] T 4d3dbd9796564472b44a31ccaf56d898 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:15.686517 19938 tablet_bootstrap.cc:492] T 5943795bc8b24bb48b80d6d06d17750b P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:15.692337 20136 raft_consensus.cc:493] T dc14c37aa393407a87af0d0eb8b88d82 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:15.693096 20011 ts_tablet_manager.cc:1929] T c0f05269ea6c43a98d2137eae7a9a401 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:56:15.693481 20011 log.cc:1199] T c0f05269ea6c43a98d2137eae7a9a401 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/c0f05269ea6c43a98d2137eae7a9a401
I20260925 13:56:15.694290 20011 ts_tablet_manager.cc:1950] T c0f05269ea6c43a98d2137eae7a9a401 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:15.694689 19942 tablet_bootstrap.cc:492] T bae8c5b6e1014731901bc3f336104784 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:15.704085 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 60) took 1238 ms. Trace:
I20260925 13:56:15.704464 20011 rpcz_store.cc:276] 0925 13:56:14.465840 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:14.466014 (+ 174us) service_pool.cc:224] Handling call
0925 13:56:15.618851 (+1152837us) ts_tablet_manager.cc:1151] Deleting tablet c0f05269ea6c43a98d2137eae7a9a401
0925 13:56:15.618894 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:15.664974 (+ 46080us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:15.692974 (+ 28000us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:15.704022 (+ 11048us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1152158}
I20260925 13:56:15.706000 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet c0f05269ea6c43a98d2137eae7a9a401 (table default.Table_72b4fdbfbca745a286dca8c191dc2286 [id=41786d7cdac8440ca86768cac5b28f49]) successfully deleted
I20260925 13:56:15.707050 19938 tablet_bootstrap.cc:654] T 5943795bc8b24bb48b80d6d06d17750b P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:15.405920 19959 ts_tablet_manager.cc:1929] T 608bed5b04394bfca1776b5d6c502df1 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:15.712191 19959 log.cc:1199] T 608bed5b04394bfca1776b5d6c502df1 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/608bed5b04394bfca1776b5d6c502df1
I20260925 13:56:15.713122 19959 ts_tablet_manager.cc:1950] T 608bed5b04394bfca1776b5d6c502df1 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:15.715806 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 608bed5b04394bfca1776b5d6c502df1 (table default.Table_3b49fb893abb48ac8646283360e8e681 [id=9621507077d04649a58715b250fcef05]) successfully deleted
W20260925 13:56:15.717466 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 2a1f34dba13143b097a38bab3782c404 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2a1f34dba13143b097a38bab3782c404 already in progress: creating tablet
I20260925 13:56:15.717834 19942 tablet_bootstrap.cc:654] T bae8c5b6e1014731901bc3f336104784 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:15.767879 20068 raft_consensus.cc:493] T bae8c5b6e1014731901bc3f336104784 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:15.768362 20068 raft_consensus.cc:515] T bae8c5b6e1014731901bc3f336104784 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:15.797389 20177 raft_consensus.cc:493] T 8239d0cd3b0d40bdb721e807c0f403d3 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:15.797801 20177 raft_consensus.cc:515] T 8239d0cd3b0d40bdb721e807c0f403d3 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:15.799880 20177 leader_election.cc:290] T 8239d0cd3b0d40bdb721e807c0f403d3 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:15.800877 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8239d0cd3b0d40bdb721e807c0f403d3" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:15.802183 19773 leader_election.cc:343] T 8239d0cd3b0d40bdb721e807c0f403d3 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:15.803320 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8239d0cd3b0d40bdb721e807c0f403d3" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
W20260925 13:56:15.804729 19773 leader_election.cc:343] T 8239d0cd3b0d40bdb721e807c0f403d3 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:15.805115 19773 leader_election.cc:304] T 8239d0cd3b0d40bdb721e807c0f403d3 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:15.833014 19959 tablet_replica.cc:333] T a88bbe1ebee7468d8f49dfd25637fe83 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:15.833945 19959 raft_consensus.cc:2243] T a88bbe1ebee7468d8f49dfd25637fe83 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:15.834470 19959 raft_consensus.cc:2272] T a88bbe1ebee7468d8f49dfd25637fe83 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:15.835841 20190 raft_consensus.cc:2749] T 8239d0cd3b0d40bdb721e807c0f403d3 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:15.836517 20190 raft_consensus.cc:493] T 26644768f516412f8c95d68a76bb99c5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:15.836941 20190 raft_consensus.cc:515] T 26644768f516412f8c95d68a76bb99c5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:15.840596 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26644768f516412f8c95d68a76bb99c5" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:15.842432 19773 leader_election.cc:343] T 26644768f516412f8c95d68a76bb99c5 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:15.849910 19955 ts_tablet_manager.cc:1929] T f51c5ad86ce64891a6393c1dc8d414e5 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:56:15.855583 19840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae8c5b6e1014731901bc3f336104784" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:15.856119 19840 raft_consensus.cc:2468] T bae8c5b6e1014731901bc3f336104784 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
W20260925 13:56:15.873332 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 608bed5b04394bfca1776b5d6c502df1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 608bed5b04394bfca1776b5d6c502df1 already in progress: creating tablet
W20260925 13:56:15.874950 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet a88bbe1ebee7468d8f49dfd25637fe83 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a88bbe1ebee7468d8f49dfd25637fe83 already in progress: creating tablet
I20260925 13:56:15.880599 19818 tablet_service.cc:1511] Processing CreateTablet for tablet 3e6c8b9bbf4549b3a8685f886973ed9c (DEFAULT_TABLE table=default.Table_d0dadd9b31b143fd8dd5adcb7f752c5f [id=beee0ac537e44859b0f7c76d6808b0e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:15.881631 19818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e6c8b9bbf4549b3a8685f886973ed9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:15.886924 19674 tablet_service.cc:1511] Processing CreateTablet for tablet 137c43fb74ec4cc48d30ed3edac81d98 (DEFAULT_TABLE table=default.Table_5321051e9f6e4cd28b9b77619c908dc9 [id=9dac142084c948f9a50cfc9acf24d461]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:15.888033 19674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 137c43fb74ec4cc48d30ed3edac81d98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:15.892289 19942 tablet_bootstrap.cc:492] T bae8c5b6e1014731901bc3f336104784 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:15.894769 19942 ts_tablet_manager.cc:1403] T bae8c5b6e1014731901bc3f336104784 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.200s user 0.004s sys 0.022s
I20260925 13:56:15.897132 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae8c5b6e1014731901bc3f336104784" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:15.903118 19942 raft_consensus.cc:359] T bae8c5b6e1014731901bc3f336104784 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:15.903786 19942 raft_consensus.cc:385] T bae8c5b6e1014731901bc3f336104784 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:15.904090 19942 raft_consensus.cc:740] T bae8c5b6e1014731901bc3f336104784 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:15.904671 19938 tablet_bootstrap.cc:492] T 5943795bc8b24bb48b80d6d06d17750b P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:15.904755 19942 consensus_queue.cc:260] T bae8c5b6e1014731901bc3f336104784 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:15.909410 19942 ts_tablet_manager.cc:1434] T bae8c5b6e1014731901bc3f336104784 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.014s user 0.000s sys 0.006s
I20260925 13:56:15.910261 19942 tablet_bootstrap.cc:492] T 608bed5b04394bfca1776b5d6c502df1 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:15.915863 19938 ts_tablet_manager.cc:1403] T 5943795bc8b24bb48b80d6d06d17750b P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.230s user 0.015s sys 0.013s
I20260925 13:56:15.924865 19938 raft_consensus.cc:359] T 5943795bc8b24bb48b80d6d06d17750b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:15.925642 19938 raft_consensus.cc:385] T 5943795bc8b24bb48b80d6d06d17750b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:15.925959 19938 raft_consensus.cc:740] T 5943795bc8b24bb48b80d6d06d17750b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:15.931367 19938 consensus_queue.cc:260] T 5943795bc8b24bb48b80d6d06d17750b P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:15.937124 19938 ts_tablet_manager.cc:1434] T 5943795bc8b24bb48b80d6d06d17750b P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.015s user 0.006s sys 0.000s
I20260925 13:56:15.937116 20190 leader_election.cc:290] T 26644768f516412f8c95d68a76bb99c5 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:15.938380 19959 ts_tablet_manager.cc:1916] T a88bbe1ebee7468d8f49dfd25637fe83 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:15.946740 19938 tablet_bootstrap.cc:492] T b12a6c6cff024d70b53cc2c9169dd48b P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:15.952160 19938 tablet_bootstrap.cc:654] T b12a6c6cff024d70b53cc2c9169dd48b P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:15.963552 19274 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_36b4dd2097aa4e2dbd58d98277e90281" } modify_external_catalogs: true
I20260925 13:56:15.964251 19274 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_36b4dd2097aa4e2dbd58d98277e90281" } modify_external_catalogs: true
I20260925 13:56:15.975685 19272 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_d37fb0770cc84c09ab355bca02212825" } new_table_name: "default.Table_c90b2c8024944acb9ffe76b506fff5a2" modify_external_catalogs: true
I20260925 13:56:15.988902 19271 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_193437ac4a794e79af9a3edb1c29a54d" } new_table_name: "default.Table_079723ff92144ba293733e54ab25d782" modify_external_catalogs: true
I20260925 13:56:15.692718 20136 raft_consensus.cc:515] T dc14c37aa393407a87af0d0eb8b88d82 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:16.010540 19955 log.cc:1199] T f51c5ad86ce64891a6393c1dc8d414e5 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/f51c5ad86ce64891a6393c1dc8d414e5
I20260925 13:56:16.011277 19955 ts_tablet_manager.cc:1950] T f51c5ad86ce64891a6393c1dc8d414e5 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:16.011910 20136 leader_election.cc:290] T dc14c37aa393407a87af0d0eb8b88d82 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:15.851027 20068 leader_election.cc:290] T bae8c5b6e1014731901bc3f336104784 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:15.865643 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26644768f516412f8c95d68a76bb99c5" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:15.869339 19490 leader_election.cc:304] T bae8c5b6e1014731901bc3f336104784 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters:
W20260925 13:56:16.015898 19488 leader_election.cc:343] T bae8c5b6e1014731901bc3f336104784 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:16.016000 19490 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 146788us
I20260925 13:56:15.871634 20198 consensus_queue.cc:1048] T c1f15b9de572479aae6c2e644da3392a P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 13:56:15.884709 19939 tablet_bootstrap.cc:492] T 608bed5b04394bfca1776b5d6c502df1 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:15.891974 19534 tablet_service.cc:1511] Processing CreateTablet for tablet 3e6c8b9bbf4549b3a8685f886973ed9c (DEFAULT_TABLE table=default.Table_d0dadd9b31b143fd8dd5adcb7f752c5f [id=beee0ac537e44859b0f7c76d6808b0e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:16.017973 20198 raft_consensus.cc:2804] T bae8c5b6e1014731901bc3f336104784 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:16.018267 19939 ts_tablet_manager.cc:1403] T 608bed5b04394bfca1776b5d6c502df1 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.510s user 0.030s sys 0.064s
I20260925 13:56:16.018316 20198 raft_consensus.cc:493] T bae8c5b6e1014731901bc3f336104784 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:16.018688 19534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e6c8b9bbf4549b3a8685f886973ed9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:16.018663 20198 raft_consensus.cc:3060] T bae8c5b6e1014731901bc3f336104784 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:16.019299 20153 raft_consensus.cc:493] T bae8c5b6e1014731901bc3f336104784 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:16.019729 20153 raft_consensus.cc:515] T bae8c5b6e1014731901bc3f336104784 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:16.020622 19939 raft_consensus.cc:359] T 608bed5b04394bfca1776b5d6c502df1 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:16.021270 19939 raft_consensus.cc:385] T 608bed5b04394bfca1776b5d6c502df1 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:16.021505 19939 raft_consensus.cc:740] T 608bed5b04394bfca1776b5d6c502df1 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:16.021598 20153 leader_election.cc:290] T bae8c5b6e1014731901bc3f336104784 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:16.022054 19939 consensus_queue.cc:260] T 608bed5b04394bfca1776b5d6c502df1 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:16.023442 19840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae8c5b6e1014731901bc3f336104784" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:16.024000 19840 raft_consensus.cc:2468] T bae8c5b6e1014731901bc3f336104784 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:15.897267 20202 consensus_queue.cc:1048] T 18a6d1c4eb7948149e709a380da50d93 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 13:56:15.955950 19535 tablet_service.cc:1511] Processing CreateTablet for tablet 137c43fb74ec4cc48d30ed3edac81d98 (DEFAULT_TABLE table=default.Table_5321051e9f6e4cd28b9b77619c908dc9 [id=9dac142084c948f9a50cfc9acf24d461]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:16.025803 19535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 137c43fb74ec4cc48d30ed3edac81d98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:16.027463 19939 ts_tablet_manager.cc:1434] T 608bed5b04394bfca1776b5d6c502df1 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260925 13:56:16.027547 19631 leader_election.cc:304] T bae8c5b6e1014731901bc3f336104784 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:16.002326 19533 tablet_service.cc:1511] Processing CreateTablet for tablet 7ea4fd6817654bff9c0fe8c6e42ed6b9 (DEFAULT_TABLE table=default.Table_d647554488064f3ea60ad9b397cabe30 [id=b889cefa13cb486f9fb18991d8b6a9ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:16.028285 19939 tablet_bootstrap.cc:492] T 4d3dbd9796564472b44a31ccaf56d898 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:16.028942 19533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ea4fd6817654bff9c0fe8c6e42ed6b9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:16.017995 19488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.194:38641, user_credentials={real_user=slave}} blocked reactor thread for 110351us
I20260925 13:56:16.035209 19840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc14c37aa393407a87af0d0eb8b88d82" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:16.035852 19840 raft_consensus.cc:2468] T dc14c37aa393407a87af0d0eb8b88d82 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:16.035986 20157 raft_consensus.cc:2804] T bae8c5b6e1014731901bc3f336104784 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:16.036252 19939 tablet_bootstrap.cc:654] T 4d3dbd9796564472b44a31ccaf56d898 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:16.036298 20157 raft_consensus.cc:493] T bae8c5b6e1014731901bc3f336104784 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:16.036468 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 57) took 1854 ms. Trace:
I20260925 13:56:16.036597 20157 raft_consensus.cc:3060] T bae8c5b6e1014731901bc3f336104784 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:16.036760 19955 rpcz_store.cc:276] 0925 13:56:14.182326 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:14.183217 (+ 891us) service_pool.cc:224] Handling call
0925 13:56:15.181770 (+998553us) ts_tablet_manager.cc:1151] Deleting tablet f51c5ad86ce64891a6393c1dc8d414e5
0925 13:56:15.181804 (+ 34us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:15.520595 (+338791us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:15.849802 (+329207us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:16.036405 (+186603us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":997974}
I20260925 13:56:16.037926 19955 tablet_replica.cc:333] T 00b500182bb04a72b80cfa9103e85fa9 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:16.038585 19955 raft_consensus.cc:2243] T 00b500182bb04a72b80cfa9103e85fa9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:16.038985 19955 raft_consensus.cc:2272] T 00b500182bb04a72b80cfa9103e85fa9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:16.041275 19490 leader_election.cc:304] T dc14c37aa393407a87af0d0eb8b88d82 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:16.046051 20157 raft_consensus.cc:515] T bae8c5b6e1014731901bc3f336104784 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
W20260925 13:56:16.047000 19773 leader_election.cc:343] T 26644768f516412f8c95d68a76bb99c5 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:16.047405 19773 leader_election.cc:304] T 26644768f516412f8c95d68a76bb99c5 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:16.049220 20177 raft_consensus.cc:2749] T 26644768f516412f8c95d68a76bb99c5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:16.052023 19939 tablet_bootstrap.cc:492] T 4d3dbd9796564472b44a31ccaf56d898 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:16.052448 19939 ts_tablet_manager.cc:1403] T 4d3dbd9796564472b44a31ccaf56d898 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.024s user 0.009s sys 0.003s
I20260925 13:56:16.052377 20177 raft_consensus.cc:493] T 5943795bc8b24bb48b80d6d06d17750b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:16.052774 20177 raft_consensus.cc:515] T 5943795bc8b24bb48b80d6d06d17750b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:16.054865 20177 leader_election.cc:290] T 5943795bc8b24bb48b80d6d06d17750b P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:16.057299 20202 raft_consensus.cc:2804] T dc14c37aa393407a87af0d0eb8b88d82 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:16.057629 20202 raft_consensus.cc:493] T dc14c37aa393407a87af0d0eb8b88d82 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:16.057931 20202 raft_consensus.cc:3060] T dc14c37aa393407a87af0d0eb8b88d82 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:16.058089 19939 raft_consensus.cc:359] T 4d3dbd9796564472b44a31ccaf56d898 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:16.058640 19275 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 7292d04b411c415394e9d0df85aaf579
I20260925 13:56:16.058734 19939 raft_consensus.cc:385] T 4d3dbd9796564472b44a31ccaf56d898 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:16.058985 19939 raft_consensus.cc:740] T 4d3dbd9796564472b44a31ccaf56d898 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:16.060199 19275 catalog_manager.cc:6934] ReplaceTablet: tablet 7292d04b411c415394e9d0df85aaf579 deleted and replaced by tablet eb794cef052b43da831531455e2b5f15
I20260925 13:56:16.060652 19275 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:39288 (request call id 121) took 1382 ms. Trace:
I20260925 13:56:16.062216 19939 consensus_queue.cc:260] T 4d3dbd9796564472b44a31ccaf56d898 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:16.061060 19275 rpcz_store.cc:276] 0925 13:56:14.678044 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:14.678160 (+ 116us) service_pool.cc:224] Handling call
0925 13:56:16.060551 (+1382391us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:15.990380 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:15.990758 (+ 378us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:15.990795 (+ 37us) write_op.cc:435] Acquired schema lock
0925 13:56:15.990808 (+ 13us) tablet.cc:662] Decoding operations
0925 13:56:15.991102 (+ 294us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:15.991132 (+ 30us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:16.033237 (+ 42105us) tablet.cc:685] Acquiring locks for 2 operations
0925 13:56:16.033477 (+ 240us) tablet.cc:701] Row locks acquired
0925 13:56:16.033494 (+ 17us) write_op.cc:260] PREPARE: finished
0925 13:56:16.033611 (+ 117us) write_op.cc:270] Start()
0925 13:56:16.033701 (+ 90us) write_op.cc:276] Timestamp: P: 1790344576033747 usec, L: 0
0925 13:56:16.033716 (+ 15us) op_driver.cc:348] REPLICATION: starting
0925 13:56:16.034218 (+ 502us) log.cc:844] Serialized 665 byte log entry
0925 13:56:16.049450 (+ 15232us) op_driver.cc:464] REPLICATION: finished
0925 13:56:16.054268 (+ 4818us) write_op.cc:301] APPLY: starting
0925 13:56:16.054328 (+ 60us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:16.054450 (+ 122us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:16.054560 (+ 110us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:16.054890 (+ 330us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:16.055082 (+ 192us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 13:56:16.055139 (+ 57us) write_op.cc:312] APPLY: finished
0925 13:56:16.055321 (+ 182us) log.cc:844] Serialized 29 byte log entry
0925 13:56:16.055943 (+ 622us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:16.056045 (+ 102us) write_op.cc:454] Released schema lock
0925 13:56:16.056080 (+ 35us) write_op.cc:341] FINISH: Updating metrics
0925 13:56:16.057343 (+ 1263us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:16.057366 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":1418624,"child_traces":[["op",{"apply.queue_time_us":4653,"apply.run_cpu_time_us":1609,"apply.run_wall_time_us":2037,"num_ops":2,"prepare.queue_time_us":437601,"prepare.run_cpu_time_us":2122,"prepare.run_wall_time_us":66952,"raft.queue_time_us":81,"raft.run_cpu_time_us":636,"raft.run_wall_time_us":635,"replication_time_us":15645}]]}
I20260925 13:56:16.066087 20068 consensus_queue.cc:1048] T 18a6d1c4eb7948149e709a380da50d93 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:16.068256 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae8c5b6e1014731901bc3f336104784" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:16.069787 20202 raft_consensus.cc:515] T dc14c37aa393407a87af0d0eb8b88d82 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:16.072712 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5943795bc8b24bb48b80d6d06d17750b" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:16.073874 19955 ts_tablet_manager.cc:1916] T 00b500182bb04a72b80cfa9103e85fa9 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:16.075142 19939 ts_tablet_manager.cc:1434] T 4d3dbd9796564472b44a31ccaf56d898 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.021s user 0.006s sys 0.000s
W20260925 13:56:16.077589 19773 leader_election.cc:343] T 5943795bc8b24bb48b80d6d06d17750b P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:16.079404 19939 tablet_bootstrap.cc:492] T f1c6e2137700448a8ebf859a9ca9085d P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:16.083001 20198 raft_consensus.cc:515] T bae8c5b6e1014731901bc3f336104784 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:16.084172 19554 raft_consensus.cc:2393] T bae8c5b6e1014731901bc3f336104784 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7c1b52895a14482fb54a6758f5054f8f in current term 1: Already voted for candidate 8f578869bc084a5d8a9a69f143954eb3 in this term.
I20260925 13:56:16.091122 20068 raft_consensus.cc:493] T b36aebfe984f454ea910e84645c4ac7d P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:16.091580 20068 raft_consensus.cc:515] T b36aebfe984f454ea910e84645c4ac7d P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:16.093467 20068 leader_election.cc:290] T b36aebfe984f454ea910e84645c4ac7d P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:16.096856 19939 tablet_bootstrap.cc:654] T f1c6e2137700448a8ebf859a9ca9085d P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:16.105226 19534 tablet_service.cc:1558] Processing DeleteTablet for tablet 7292d04b411c415394e9d0df85aaf579 with delete_type TABLET_DATA_DELETED (replaced by tablet eb794cef052b43da831531455e2b5f15) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:16.106096 19817 tablet_service.cc:1558] Processing DeleteTablet for tablet 7292d04b411c415394e9d0df85aaf579 with delete_type TABLET_DATA_DELETED (replaced by tablet eb794cef052b43da831531455e2b5f15) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:16.106268 20198 leader_election.cc:290] T bae8c5b6e1014731901bc3f336104784 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:16.111477 20202 leader_election.cc:290] T dc14c37aa393407a87af0d0eb8b88d82 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:16.118309 19838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b36aebfe984f454ea910e84645c4ac7d" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:16.118930 19838 raft_consensus.cc:2468] T b36aebfe984f454ea910e84645c4ac7d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:16.120235 19838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae8c5b6e1014731901bc3f336104784" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:16.120729 19838 raft_consensus.cc:3060] T bae8c5b6e1014731901bc3f336104784 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:16.123536 20190 raft_consensus.cc:493] T c2d333f892f84a258f8a148a90ce3b06 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:16.124104 20190 raft_consensus.cc:515] T c2d333f892f84a258f8a148a90ce3b06 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:16.126055 20190 leader_election.cc:290] T c2d333f892f84a258f8a148a90ce3b06 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:16.127763 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d333f892f84a258f8a148a90ce3b06" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
W20260925 13:56:16.129259 19773 leader_election.cc:343] T c2d333f892f84a258f8a148a90ce3b06 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:16.136256 19839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc14c37aa393407a87af0d0eb8b88d82" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:16.136801 19839 raft_consensus.cc:3060] T dc14c37aa393407a87af0d0eb8b88d82 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:16.137724 19939 tablet_bootstrap.cc:492] T f1c6e2137700448a8ebf859a9ca9085d P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:16.138204 19939 ts_tablet_manager.cc:1403] T f1c6e2137700448a8ebf859a9ca9085d P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.062s user 0.017s sys 0.003s
I20260925 13:56:16.139915 19490 leader_election.cc:304] T b36aebfe984f454ea910e84645c4ac7d P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:16.140645 20202 raft_consensus.cc:2804] T b36aebfe984f454ea910e84645c4ac7d P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:16.141253 20202 raft_consensus.cc:493] T b36aebfe984f454ea910e84645c4ac7d P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:16.141665 20202 raft_consensus.cc:3060] T b36aebfe984f454ea910e84645c4ac7d P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:16.145162 19839 raft_consensus.cc:2468] T dc14c37aa393407a87af0d0eb8b88d82 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:16.088680 19276 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 1513c0048bd244028d3dfa8d118bc14e (table default.Table_74b6076b22674a31bc58fef71cc9bfb1 [id=d81cf07e5f9a49d4bbf5aceda3da5a25]). Expected version 1 got 0
I20260925 13:56:16.104423 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet f51c5ad86ce64891a6393c1dc8d414e5 (table default.Table_824449a97a864e3dbbf06e2ad678edc5 [id=af708d3ba4c54ba19c24eca862bbddc1]) successfully deleted
I20260925 13:56:16.123047 19274 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 5943795bc8b24bb48b80d6d06d17750b
I20260925 13:56:16.147737 19276 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 18a6d1c4eb7948149e709a380da50d93 (table default.Table_abc00b1f4c4c455e8086a46c395f2656 [id=6c0ec9465714408da93ff6e47827eb48]). Expected version 1 got 0
W20260925 13:56:16.148859 19264 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.18.182.193:38269, user_credentials={real_user=slave}} blocked reactor thread for 44484.1us
I20260925 13:56:16.148957 19276 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 47521fb74f9c40ee999a7f704aa551c9 (table default.Table_3a3e48a0641a43a1a783c6ba6374a708 [id=e64fe44a237f48859a288a87a565ee5e]). Expected version 1 got 0
I20260925 13:56:16.152097 20202 raft_consensus.cc:515] T b36aebfe984f454ea910e84645c4ac7d P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:16.153770 19274 master_service.cc:960] ReplaceTablet: received request to replace tablet 4d3dbd9796564472b44a31ccaf56d898 from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:16.157666 19490 leader_election.cc:304] T dc14c37aa393407a87af0d0eb8b88d82 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:16.158318 20198 raft_consensus.cc:2804] T dc14c37aa393407a87af0d0eb8b88d82 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:16.158735 20198 raft_consensus.cc:697] T dc14c37aa393407a87af0d0eb8b88d82 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Running, Role: LEADER
I20260925 13:56:16.159694 20198 consensus_queue.cc:237] T dc14c37aa393407a87af0d0eb8b88d82 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:16.161067 19280 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 1513c0048bd244028d3dfa8d118bc14e (table default.Table_74b6076b22674a31bc58fef71cc9bfb1 [id=d81cf07e5f9a49d4bbf5aceda3da5a25]). Expected version 1 got 0
I20260925 13:56:16.166034 19279 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet 1513c0048bd244028d3dfa8d118bc14e (table default.Table_74b6076b22674a31bc58fef71cc9bfb1 [id=d81cf07e5f9a49d4bbf5aceda3da5a25]). Expected version 1 got 0
I20260925 13:56:16.069238 19693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5943795bc8b24bb48b80d6d06d17750b" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:16.167726 19534 tablet_service.cc:1558] Processing DeleteTablet for tablet 5943795bc8b24bb48b80d6d06d17750b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:15 UTC) from {username='slave'} at 127.0.0.1:40018
W20260925 13:56:16.169831 19272 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c2d333f892f84a258f8a148a90ce3b06 consensus state has no leader
W20260925 13:56:16.170321 19773 leader_election.cc:343] T 5943795bc8b24bb48b80d6d06d17750b P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:16.170687 19773 leader_election.cc:304] T 5943795bc8b24bb48b80d6d06d17750b P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:16.166778 19279 catalog_manager.cc:5838] T c1f15b9de572479aae6c2e644da3392a P 8f578869bc084a5d8a9a69f143954eb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "8f578869bc084a5d8a9a69f143954eb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:16.085423 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc14c37aa393407a87af0d0eb8b88d82" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:16.095534 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b36aebfe984f454ea910e84645c4ac7d" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:16.171854 19279 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet 47521fb74f9c40ee999a7f704aa551c9 (table default.Table_3a3e48a0641a43a1a783c6ba6374a708 [id=e64fe44a237f48859a288a87a565ee5e]). Expected version 1 got 0
I20260925 13:56:16.098105 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae8c5b6e1014731901bc3f336104784" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:16.101964 19696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc14c37aa393407a87af0d0eb8b88d82" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:16.172561 19696 raft_consensus.cc:3060] T dc14c37aa393407a87af0d0eb8b88d82 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:16.174103 19939 raft_consensus.cc:359] T f1c6e2137700448a8ebf859a9ca9085d P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:16.174688 19939 raft_consensus.cc:385] T f1c6e2137700448a8ebf859a9ca9085d P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:16.174907 19939 raft_consensus.cc:740] T f1c6e2137700448a8ebf859a9ca9085d P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:16.105590 19674 tablet_service.cc:1558] Processing DeleteTablet for tablet 7292d04b411c415394e9d0df85aaf579 with delete_type TABLET_DATA_DELETED (replaced by tablet eb794cef052b43da831531455e2b5f15) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:16.127388 19689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d333f892f84a258f8a148a90ce3b06" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:16.177376 20011 tablet_replica.cc:333] T 7292d04b411c415394e9d0df85aaf579 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:16.153254 19672 tablet_service.cc:1558] Processing DeleteTablet for tablet 5943795bc8b24bb48b80d6d06d17750b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:15 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:16.154914 19688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b36aebfe984f454ea910e84645c4ac7d" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:16.178082 20011 raft_consensus.cc:2243] T 7292d04b411c415394e9d0df85aaf579 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:16.178495 20011 raft_consensus.cc:2272] T 7292d04b411c415394e9d0df85aaf579 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:16.176631 19279 catalog_manager.cc:5838] T 18a6d1c4eb7948149e709a380da50d93 P 8f578869bc084a5d8a9a69f143954eb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "8f578869bc084a5d8a9a69f143954eb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:16.179476 19279 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet 18a6d1c4eb7948149e709a380da50d93 (table default.Table_abc00b1f4c4c455e8086a46c395f2656 [id=6c0ec9465714408da93ff6e47827eb48]). Expected version 1 got 0
W20260925 13:56:16.179623 19773 leader_election.cc:343] T c2d333f892f84a258f8a148a90ce3b06 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:16.179972 19773 leader_election.cc:304] T c2d333f892f84a258f8a148a90ce3b06 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:16.180824 20177 raft_consensus.cc:2749] T 5943795bc8b24bb48b80d6d06d17750b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:16.181262 19273 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet f4429db544534f758883f2fae184ea6d
I20260925 13:56:16.181418 20177 raft_consensus.cc:2749] T c2d333f892f84a258f8a148a90ce3b06 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:16.183102 20202 leader_election.cc:290] T b36aebfe984f454ea910e84645c4ac7d P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:16.175535 19939 consensus_queue.cc:260] T f1c6e2137700448a8ebf859a9ca9085d P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:16.186594 19939 ts_tablet_manager.cc:1434] T f1c6e2137700448a8ebf859a9ca9085d P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.048s user 0.003s sys 0.004s
I20260925 13:56:16.187271 19272 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_475efa7896f84e658eb81238ceb58d8f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:16.188596 20083 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:56:16.188843 19278 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.193:37907 (request call id 24) took 1750 ms (client timeout 1000 ms). Trace:
W20260925 13:56:16.189545 20083 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
I20260925 13:56:16.190482 20011 ts_tablet_manager.cc:1916] T 7292d04b411c415394e9d0df85aaf579 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:16.192993 19939 tablet_bootstrap.cc:492] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:16.196345 19534 tablet_service.cc:1558] Processing DeleteTablet for tablet f4429db544534f758883f2fae184ea6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:15 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:16.198531 19672 tablet_service.cc:1558] Processing DeleteTablet for tablet f4429db544534f758883f2fae184ea6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:15 UTC) from {username='slave'} at 127.0.0.1:45948
W20260925 13:56:16.200067 20083 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260925 13:56:16.159694 20153 raft_consensus.cc:493] T b36aebfe984f454ea910e84645c4ac7d P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:16.201689 20153 raft_consensus.cc:515] T b36aebfe984f454ea910e84645c4ac7d P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:16.202564 19692 raft_consensus.cc:2468] T b36aebfe984f454ea910e84645c4ac7d P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:16.203255 19939 tablet_bootstrap.cc:654] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:16.210175 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b36aebfe984f454ea910e84645c4ac7d" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:16.211020 19554 raft_consensus.cc:2393] T b36aebfe984f454ea910e84645c4ac7d P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7c1b52895a14482fb54a6758f5054f8f in current term 1: Already voted for candidate 8f578869bc084a5d8a9a69f143954eb3 in this term.
I20260925 13:56:16.212857 19819 tablet_service.cc:1558] Processing DeleteTablet for tablet f4429db544534f758883f2fae184ea6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:15 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:16.219240 19837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b36aebfe984f454ea910e84645c4ac7d" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:16.219888 19837 raft_consensus.cc:2468] T b36aebfe984f454ea910e84645c4ac7d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:16.221166 19939 tablet_bootstrap.cc:492] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:16.221575 19939 ts_tablet_manager.cc:1403] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.029s user 0.004s sys 0.004s
I20260925 13:56:16.222464 19819 tablet_service.cc:1558] Processing DeleteTablet for tablet 8239d0cd3b0d40bdb721e807c0f403d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:14 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:16.223099 19631 leader_election.cc:304] T b36aebfe984f454ea910e84645c4ac7d P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters: 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:16.223742 20154 raft_consensus.cc:2804] T b36aebfe984f454ea910e84645c4ac7d P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:16.224037 20154 raft_consensus.cc:493] T b36aebfe984f454ea910e84645c4ac7d P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:16.224323 20154 raft_consensus.cc:3060] T b36aebfe984f454ea910e84645c4ac7d P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:16.223977 19939 raft_consensus.cc:359] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:16.224593 19939 raft_consensus.cc:385] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:16.224840 19939 raft_consensus.cc:740] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:16.225234 19672 tablet_service.cc:1558] Processing DeleteTablet for tablet 00b500182bb04a72b80cfa9103e85fa9 with delete_type TABLET_DATA_DELETED (replaced by tablet f5a0bd0e205f4889be93739dfbd2a816) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:16.231431 20153 leader_election.cc:290] T b36aebfe984f454ea910e84645c4ac7d P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:16.232178 19817 tablet_service.cc:1558] Processing DeleteTablet for tablet 5943795bc8b24bb48b80d6d06d17750b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:15 UTC) from {username='slave'} at 127.0.0.1:51442
W20260925 13:56:16.189142 19272 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_475efa7896f84e658eb81238ceb58d8f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:16.234925 19939 consensus_queue.cc:260] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:16.205291 19273 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_10868b99c6ec43b2a38a3a4a5bdcf960"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:16.236454 19696 raft_consensus.cc:2468] T dc14c37aa393407a87af0d0eb8b88d82 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
W20260925 13:56:16.236627 19273 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_10868b99c6ec43b2a38a3a4a5bdcf960 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:16.238919 19939 ts_tablet_manager.cc:1434] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.017s user 0.006s sys 0.000s
I20260925 13:56:16.239719 19939 tablet_bootstrap.cc:492] T 8239d0cd3b0d40bdb721e807c0f403d3 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:16.244956 19939 tablet_bootstrap.cc:654] T 8239d0cd3b0d40bdb721e807c0f403d3 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:16.230978 20154 raft_consensus.cc:515] T b36aebfe984f454ea910e84645c4ac7d P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:16.249464 19837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b36aebfe984f454ea910e84645c4ac7d" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:16.250129 19837 raft_consensus.cc:3060] T b36aebfe984f454ea910e84645c4ac7d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:16.250288 20154 leader_election.cc:290] T b36aebfe984f454ea910e84645c4ac7d P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:16.251441 19839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b36aebfe984f454ea910e84645c4ac7d" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:16.251654 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b36aebfe984f454ea910e84645c4ac7d" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:16.252471 19554 raft_consensus.cc:2393] T b36aebfe984f454ea910e84645c4ac7d P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7c1b52895a14482fb54a6758f5054f8f in current term 1: Already voted for candidate 8f578869bc084a5d8a9a69f143954eb3 in this term.
I20260925 13:56:16.253017 19695 raft_consensus.cc:2393] T bae8c5b6e1014731901bc3f336104784 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f578869bc084a5d8a9a69f143954eb3 in current term 1: Already voted for candidate 7c1b52895a14482fb54a6758f5054f8f in this term.
I20260925 13:56:16.252925 19490 leader_election.cc:304] T b36aebfe984f454ea910e84645c4ac7d P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3; no voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:16.253966 20202 raft_consensus.cc:2749] T b36aebfe984f454ea910e84645c4ac7d P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260925 13:56:16.256500 20157 leader_election.cc:290] T bae8c5b6e1014731901bc3f336104784 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:16.256945 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae8c5b6e1014731901bc3f336104784" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
W20260925 13:56:16.257368 19271 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 436b8d46e1d244deb8bb9dc128ee6279 has no consensus state
I20260925 13:56:16.257882 19554 raft_consensus.cc:2393] T bae8c5b6e1014731901bc3f336104784 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7c1b52895a14482fb54a6758f5054f8f in current term 1: Already voted for candidate 8f578869bc084a5d8a9a69f143954eb3 in this term.
W20260925 13:56:16.217650 19276 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.194:34125 (request call id 23) took 797 ms (client timeout 1000 ms). Trace:
W20260925 13:56:16.261101 19276 rpcz_store.cc:269] 0925 13:56:15.420115 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:15.437108 (+ 16993us) service_pool.cc:224] Handling call
0925 13:56:16.217578 (+780470us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":8}
I20260925 13:56:16.262368 20202 raft_consensus.cc:493] T f1c6e2137700448a8ebf859a9ca9085d P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:16.262769 20202 raft_consensus.cc:515] T f1c6e2137700448a8ebf859a9ca9085d P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:16.232707 19278 rpcz_store.cc:269] 0925 13:56:14.438645 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:14.438750 (+ 105us) service_pool.cc:224] Handling call
0925 13:56:16.188655 (+1749905us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:15.421305 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:15.535504 (+114199us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:15.535531 (+ 27us) write_op.cc:435] Acquired schema lock
0925 13:56:15.535545 (+ 14us) tablet.cc:662] Decoding operations
0925 13:56:15.535952 (+ 407us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:15.535993 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:15.536015 (+ 22us) tablet.cc:685] Acquiring locks for 2 operations
0925 13:56:15.536259 (+ 244us) tablet.cc:701] Row locks acquired
0925 13:56:15.536272 (+ 13us) write_op.cc:260] PREPARE: finished
0925 13:56:15.537045 (+ 773us) write_op.cc:270] Start()
0925 13:56:15.537128 (+ 83us) write_op.cc:276] Timestamp: P: 1790344575537031 usec, L: 0
0925 13:56:15.537143 (+ 15us) op_driver.cc:348] REPLICATION: starting
0925 13:56:15.537586 (+ 443us) log.cc:844] Serialized 919 byte log entry
0925 13:56:15.547381 (+ 9795us) op_driver.cc:464] REPLICATION: finished
0925 13:56:15.547726 (+ 345us) write_op.cc:301] APPLY: starting
0925 13:56:15.547774 (+ 48us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:15.547886 (+ 112us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:15.547983 (+ 97us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:15.548208 (+ 225us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:15.548338 (+ 130us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 13:56:15.548380 (+ 42us) write_op.cc:312] APPLY: finished
0925 13:56:15.548537 (+ 157us) log.cc:844] Serialized 29 byte log entry
0925 13:56:15.548669 (+ 132us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:15.548729 (+ 60us) write_op.cc:454] Released schema lock
0925 13:56:15.548762 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0925 13:56:15.919784 (+371022us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:15.919842 (+ 58us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":7,"child_traces":[["op",{"apply.queue_time_us":215,"apply.run_cpu_time_us":1222,"apply.run_wall_time_us":1221,"num_ops":2,"prepare.queue_time_us":408490,"prepare.run_cpu_time_us":2159,"prepare.run_wall_time_us":498301,"raft.queue_time_us":87,"raft.run_cpu_time_us":599,"raft.run_wall_time_us":599,"replication_time_us":10143}]]}
I20260925 13:56:16.264173 19837 raft_consensus.cc:2468] T b36aebfe984f454ea910e84645c4ac7d P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
I20260925 13:56:16.264626 20202 leader_election.cc:290] T f1c6e2137700448a8ebf859a9ca9085d P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:16.265278 19631 leader_election.cc:304] T b36aebfe984f454ea910e84645c4ac7d P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters: 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:16.266096 19836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1c6e2137700448a8ebf859a9ca9085d" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:16.266718 19836 raft_consensus.cc:2468] T f1c6e2137700448a8ebf859a9ca9085d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:16.267858 19490 leader_election.cc:304] T f1c6e2137700448a8ebf859a9ca9085d P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:16.268004 19676 tablet_service.cc:1511] Processing CreateTablet for tablet 7ea4fd6817654bff9c0fe8c6e42ed6b9 (DEFAULT_TABLE table=default.Table_d647554488064f3ea60ad9b397cabe30 [id=b889cefa13cb486f9fb18991d8b6a9ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:16.269170 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1c6e2137700448a8ebf859a9ca9085d" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:16.269397 20202 raft_consensus.cc:2804] T f1c6e2137700448a8ebf859a9ca9085d P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:16.269502 19274 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 4d3dbd9796564472b44a31ccaf56d898
I20260925 13:56:16.269707 20202 raft_consensus.cc:493] T f1c6e2137700448a8ebf859a9ca9085d P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:16.269989 20202 raft_consensus.cc:3060] T f1c6e2137700448a8ebf859a9ca9085d P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:16.271026 19839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bae8c5b6e1014731901bc3f336104784" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
W20260925 13:56:16.272418 19488 leader_election.cc:343] T f1c6e2137700448a8ebf859a9ca9085d P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:16.272454 19631 leader_election.cc:304] T bae8c5b6e1014731901bc3f336104784 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:16.273371 20154 raft_consensus.cc:2804] T b36aebfe984f454ea910e84645c4ac7d P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:16.273641 19534 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d3dbd9796564472b44a31ccaf56d898 with delete_type TABLET_DATA_DELETED (replaced by tablet 0ad6442b2de249119ba0ef85dfc06da9) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:16.273667 19817 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d3dbd9796564472b44a31ccaf56d898 with delete_type TABLET_DATA_DELETED (replaced by tablet 0ad6442b2de249119ba0ef85dfc06da9) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:16.273913 20153 raft_consensus.cc:2749] T bae8c5b6e1014731901bc3f336104784 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260925 13:56:16.274909 19820 tablet_service.cc:1511] Processing CreateTablet for tablet 137c43fb74ec4cc48d30ed3edac81d98 (DEFAULT_TABLE table=default.Table_5321051e9f6e4cd28b9b77619c908dc9 [id=9dac142084c948f9a50cfc9acf24d461]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:16.274675 20154 raft_consensus.cc:697] T b36aebfe984f454ea910e84645c4ac7d P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Becoming Leader. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Running, Role: LEADER
I20260925 13:56:16.275906 19672 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d3dbd9796564472b44a31ccaf56d898 with delete_type TABLET_DATA_DELETED (replaced by tablet 0ad6442b2de249119ba0ef85dfc06da9) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:16.272122 19274 catalog_manager.cc:6934] ReplaceTablet: tablet 4d3dbd9796564472b44a31ccaf56d898 deleted and replaced by tablet 0ad6442b2de249119ba0ef85dfc06da9
I20260925 13:56:16.276810 20154 consensus_queue.cc:237] T b36aebfe984f454ea910e84645c4ac7d P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:16.276968 20202 raft_consensus.cc:515] T f1c6e2137700448a8ebf859a9ca9085d P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:16.279783 19817 tablet_service.cc:1511] Processing CreateTablet for tablet 7ea4fd6817654bff9c0fe8c6e42ed6b9 (DEFAULT_TABLE table=default.Table_d647554488064f3ea60ad9b397cabe30 [id=b889cefa13cb486f9fb18991d8b6a9ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:16.279230 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1c6e2137700448a8ebf859a9ca9085d" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
W20260925 13:56:16.281420 19267 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 436b8d46e1d244deb8bb9dc128ee6279 (table default.Table_079723ff92144ba293733e54ab25d782 [id=4b0bfb9726854123864f8fe743a16a33], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 436b8d46e1d244deb8bb9dc128ee6279 has no consensus state
I20260925 13:56:16.281553 20202 leader_election.cc:290] T f1c6e2137700448a8ebf859a9ca9085d P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
W20260925 13:56:16.281977 19488 leader_election.cc:343] T f1c6e2137700448a8ebf859a9ca9085d P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:16.282775 19817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ea4fd6817654bff9c0fe8c6e42ed6b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:16.291944 19676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ea4fd6817654bff9c0fe8c6e42ed6b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:16.296772 19939 tablet_bootstrap.cc:492] T 8239d0cd3b0d40bdb721e807c0f403d3 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:16.297139 19939 ts_tablet_manager.cc:1403] T 8239d0cd3b0d40bdb721e807c0f403d3 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.058s user 0.011s sys 0.000s
I20260925 13:56:16.297751 19280 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_7746d3f112bf43ffb276928654340d9d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:16.299587 19280 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7746d3f112bf43ffb276928654340d9d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:16.303294 19275 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_50a86db358374d4ea5164480fefdfee5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:16.303556 19939 raft_consensus.cc:359] T 8239d0cd3b0d40bdb721e807c0f403d3 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:16.304153 19939 raft_consensus.cc:385] T 8239d0cd3b0d40bdb721e807c0f403d3 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:16.304390 19939 raft_consensus.cc:740] T 8239d0cd3b0d40bdb721e807c0f403d3 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
W20260925 13:56:16.305260 19275 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_50a86db358374d4ea5164480fefdfee5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:16.304980 19939 consensus_queue.cc:260] T 8239d0cd3b0d40bdb721e807c0f403d3 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:16.311254 19534 tablet_service.cc:1511] Processing CreateTablet for tablet f514fa52e7524cd3ba07f3f65de532d4 (DEFAULT_TABLE table=default.Table_ba97a54a19744b058cffd5638ffb97db [id=7e7ee975d4bb457eb3cc4632e12f3cd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:16.311858 19839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1c6e2137700448a8ebf859a9ca9085d" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:16.312235 19534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f514fa52e7524cd3ba07f3f65de532d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:16.312527 19839 raft_consensus.cc:3060] T f1c6e2137700448a8ebf859a9ca9085d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:16.267259 19838 raft_consensus.cc:2468] T bae8c5b6e1014731901bc3f336104784 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:16.321734 19820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 137c43fb74ec4cc48d30ed3edac81d98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:16.323844 19490 leader_election.cc:304] T bae8c5b6e1014731901bc3f336104784 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:16.324734 20202 raft_consensus.cc:2804] T bae8c5b6e1014731901bc3f336104784 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:16.325100 20202 raft_consensus.cc:697] T bae8c5b6e1014731901bc3f336104784 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Running, Role: LEADER
I20260925 13:56:16.325750 20202 consensus_queue.cc:237] T bae8c5b6e1014731901bc3f336104784 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:16.334561 19939 ts_tablet_manager.cc:1434] T 8239d0cd3b0d40bdb721e807c0f403d3 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.037s user 0.006s sys 0.000s
I20260925 13:56:16.335398 19939 tablet_bootstrap.cc:492] T a88bbe1ebee7468d8f49dfd25637fe83 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:16.347074 20011 ts_tablet_manager.cc:1929] T 7292d04b411c415394e9d0df85aaf579 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:16.350759 19819 tablet_service.cc:1511] Processing CreateTablet for tablet f514fa52e7524cd3ba07f3f65de532d4 (DEFAULT_TABLE table=default.Table_ba97a54a19744b058cffd5638ffb97db [id=7e7ee975d4bb457eb3cc4632e12f3cd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:16.351775 19819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f514fa52e7524cd3ba07f3f65de532d4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:16.355057 19279 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.193:37907 (request call id 25) took 908 ms (client timeout 1000 ms). Trace:
W20260925 13:56:16.355334 19279 rpcz_store.cc:269] 0925 13:56:15.446646 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:15.573640 (+126994us) service_pool.cc:224] Handling call
0925 13:56:15.924547 (+350907us) spinlock_profiling.cc:243] Waited 348 ms on lock 0x7b5400040454. stack: 00007fdf0a30b323 00007fdf0a30b220 00007fdf09eaf574 0000000000516964 00000000005168f9 00007fdf114095c0 00007fdf114a86fa 00007fdf13219d8c 00007fdf1331801e 00007fdf10ebcfb4 00007fdf10ebcf42 00007fdf10ebce71 00007fdf10ebcdec 00007fdf10ebc0a2 00007fdf0f16e09c 00007fdf0f16d4d6
0925 13:56:16.354999 (+430452us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:16.276590 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:16.277872 (+ 1282us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:16.277893 (+ 21us) write_op.cc:435] Acquired schema lock
0925 13:56:16.277912 (+ 19us) tablet.cc:662] Decoding operations
0925 13:56:16.278218 (+ 306us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:16.278260 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:16.278288 (+ 28us) tablet.cc:685] Acquiring locks for 2 operations
0925 13:56:16.278543 (+ 255us) tablet.cc:701] Row locks acquired
0925 13:56:16.278558 (+ 15us) write_op.cc:260] PREPARE: finished
0925 13:56:16.289002 (+ 10444us) write_op.cc:270] Start()
0925 13:56:16.289082 (+ 80us) write_op.cc:276] Timestamp: P: 1790344576289216 usec, L: 0
0925 13:56:16.289096 (+ 14us) op_driver.cc:348] REPLICATION: starting
0925 13:56:16.307324 (+ 18228us) log.cc:844] Serialized 919 byte log entry
0925 13:56:16.308871 (+ 1547us) spinlock_profiling.cc:243] Waited 17.8 ms on lock 0x7b5400040454. stack: 00007fdf0a30b323 00007fdf0a30b220 00007fdf09eaf574 0000000000516964 00000000005168f9 00007fdf11408aa1 00007fdf1149b5e8 00007fdf1192b741 00007fdf1192af18 00007fdf1192fbb1 00007fdf1192fb69 00007fdf1192faf9 00007fdf1192fac1 00007fdf1192eced 00000000005bd4e4 00000000005bd319
0925 13:56:16.342442 (+ 33571us) op_driver.cc:464] REPLICATION: finished
0925 13:56:16.343381 (+ 939us) write_op.cc:301] APPLY: starting
0925 13:56:16.343435 (+ 54us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:16.343564 (+ 129us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:16.343583 (+ 19us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:16.343819 (+ 236us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:16.343961 (+ 142us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 13:56:16.344003 (+ 42us) write_op.cc:312] APPLY: finished
0925 13:56:16.344163 (+ 160us) log.cc:844] Serialized 29 byte log entry
0925 13:56:16.344297 (+ 134us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:16.344362 (+ 65us) write_op.cc:454] Released schema lock
0925 13:56:16.344392 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0925 13:56:16.344623 (+ 231us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:16.344649 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":9,"spinlock_wait_cycles":801310976,"child_traces":[["op",{"apply.queue_time_us":778,"apply.run_cpu_time_us":1232,"apply.run_wall_time_us":1237,"num_ops":2,"prepare.queue_time_us":90037,"prepare.run_cpu_time_us":2528,"prepare.run_wall_time_us":32481,"raft.queue_time_us":10487,"raft.run_cpu_time_us":901,"raft.run_wall_time_us":13749,"replication_time_us":53249,"spinlock_wait_cycles":40897408}]]}
I20260925 13:56:16.364131 19939 tablet_bootstrap.cc:654] T a88bbe1ebee7468d8f49dfd25637fe83 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:16.359833 19273 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_9896a33d94e44d76ada6ec8d00c87ad7" } modify_external_catalogs: true
I20260925 13:56:16.364626 19273 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_9896a33d94e44d76ada6ec8d00c87ad7" } modify_external_catalogs: true
I20260925 13:56:16.387079 19955 ts_tablet_manager.cc:1929] T 00b500182bb04a72b80cfa9103e85fa9 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:16.387513 19955 log.cc:1199] T 00b500182bb04a72b80cfa9103e85fa9 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/00b500182bb04a72b80cfa9103e85fa9
I20260925 13:56:16.388146 19955 ts_tablet_manager.cc:1950] T 00b500182bb04a72b80cfa9103e85fa9 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:16.414094 19271 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_ab121c553eee48ccb03ed0c1e875a43a" } modify_external_catalogs: true
I20260925 13:56:16.414681 19271 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_ab121c553eee48ccb03ed0c1e875a43a" } modify_external_catalogs: true
I20260925 13:56:16.418895 19276 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_abc00b1f4c4c455e8086a46c395f2656" } new_table_name: "default.Table_e814702f1f874529a98bd1631d3b5c84" modify_external_catalogs: true
I20260925 13:56:16.439033 19959 ts_tablet_manager.cc:1929] T a88bbe1ebee7468d8f49dfd25637fe83 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:16.444878 19279 master_service.cc:960] ReplaceTablet: received request to replace tablet dc14c37aa393407a87af0d0eb8b88d82 from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:16.353564 19672 tablet_service.cc:1511] Processing CreateTablet for tablet f514fa52e7524cd3ba07f3f65de532d4 (DEFAULT_TABLE table=default.Table_ba97a54a19744b058cffd5638ffb97db [id=7e7ee975d4bb457eb3cc4632e12f3cd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:16.468447 19672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f514fa52e7524cd3ba07f3f65de532d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:16.387149 19942 tablet_bootstrap.cc:654] T 608bed5b04394bfca1776b5d6c502df1 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
W20260925 13:56:16.411510 19631 consensus_peers.cc:597] T 00b500182bb04a72b80cfa9103e85fa9 P 7c1b52895a14482fb54a6758f5054f8f -> Peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Couldn't send request to peer f790493ba7ad403993b678f2df2b55be. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 00b500182bb04a72b80cfa9103e85fa9. This is attempt 1: this message will repeat every 5th retry.
W20260925 13:56:16.474462 19631 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 63179.8us
I20260925 13:56:16.475869 19939 tablet_bootstrap.cc:492] T a88bbe1ebee7468d8f49dfd25637fe83 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:16.476229 19939 ts_tablet_manager.cc:1403] T a88bbe1ebee7468d8f49dfd25637fe83 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.141s user 0.030s sys 0.021s
I20260925 13:56:16.478282 19939 raft_consensus.cc:359] T a88bbe1ebee7468d8f49dfd25637fe83 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:16.478946 19939 raft_consensus.cc:385] T a88bbe1ebee7468d8f49dfd25637fe83 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:16.479205 19939 raft_consensus.cc:740] T a88bbe1ebee7468d8f49dfd25637fe83 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:16.479833 19939 consensus_queue.cc:260] T a88bbe1ebee7468d8f49dfd25637fe83 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:16.487020 19939 ts_tablet_manager.cc:1434] T a88bbe1ebee7468d8f49dfd25637fe83 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20260925 13:56:16.487808 19939 tablet_bootstrap.cc:492] T c65a8cd3060c405c871270543b4623f7 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:16.492527 20157 raft_consensus.cc:493] T 980d1264c0ea45f3b2b762509e0b2c77 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:16.492919 20157 raft_consensus.cc:515] T 980d1264c0ea45f3b2b762509e0b2c77 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:16.495179 20157 leader_election.cc:290] T 980d1264c0ea45f3b2b762509e0b2c77 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
W20260925 13:56:16.496277 19629 leader_election.cc:343] T 980d1264c0ea45f3b2b762509e0b2c77 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Not found: Tablet not found: 980d1264c0ea45f3b2b762509e0b2c77
I20260925 13:56:16.497493 19939 tablet_bootstrap.cc:654] T c65a8cd3060c405c871270543b4623f7 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
W20260925 13:56:16.498025 19631 leader_election.cc:343] T 980d1264c0ea45f3b2b762509e0b2c77 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 980d1264c0ea45f3b2b762509e0b2c77
I20260925 13:56:16.498394 19631 leader_election.cc:304] T 980d1264c0ea45f3b2b762509e0b2c77 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:16.499680 19280 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_06cdecf307e549bf908eb64143c31996" } new_table_name: "default.Table_547f995b454c497bb5f8d7241ab26876" modify_external_catalogs: true
I20260925 13:56:16.500994 19278 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 18a6d1c4eb7948149e709a380da50d93 (table default.Table_abc00b1f4c4c455e8086a46c395f2656 [id=6c0ec9465714408da93ff6e47827eb48]). Expected version 1 got 0
I20260925 13:56:16.502825 20157 raft_consensus.cc:2749] T 980d1264c0ea45f3b2b762509e0b2c77 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:16.508111 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 59) took 2297 ms. Trace:
I20260925 13:56:16.509847 19955 rpcz_store.cc:276] 0925 13:56:14.210273 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:14.210726 (+ 453us) service_pool.cc:224] Handling call
0925 13:56:16.037786 (+1827060us) ts_tablet_manager.cc:1151] Deleting tablet 00b500182bb04a72b80cfa9103e85fa9
0925 13:56:16.037825 (+ 39us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:16.204404 (+166579us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:16.386949 (+182545us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:16.508060 (+121111us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1826543}
I20260925 13:56:16.518322 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet 00b500182bb04a72b80cfa9103e85fa9 (table default.Table_04d053f331f243ce8e98beb7c00d4159 [id=745c9e0d4f2c4ce58c3a201c34f1f1e3]) successfully deleted
I20260925 13:56:16.519449 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 61) took 2293 ms. Trace:
I20260925 13:56:16.520539 19955 rpcz_store.cc:276] 0925 13:56:14.225794 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:14.225918 (+ 124us) service_pool.cc:224] Handling call
0925 13:56:16.519193 (+2293275us) ts_tablet_manager.cc:1151] Deleting tablet 3b10ecb0e6064d42a2521c4c4e91c028
0925 13:56:16.519234 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:16.519400 (+ 166us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2292799}
I20260925 13:56:16.521354 20011 log.cc:1199] T 7292d04b411c415394e9d0df85aaf579 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/7292d04b411c415394e9d0df85aaf579
I20260925 13:56:16.522367 20011 ts_tablet_manager.cc:1950] T 7292d04b411c415394e9d0df85aaf579 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
W20260925 13:56:16.528088 19264 catalog_manager.cc:5030] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): delete failed for tablet 3b10ecb0e6064d42a2521c4c4e91c028 because the tablet was not found. No further retry: Not found: Tablet not found: 3b10ecb0e6064d42a2521c4c4e91c028
I20260925 13:56:16.529233 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 62) took 2301 ms. Trace:
I20260925 13:56:16.532052 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 7292d04b411c415394e9d0df85aaf579 (table default.Table_06cdecf307e549bf908eb64143c31996 [id=d2bc135460ab4f2fad4740a281bfc6e2]) successfully deleted
I20260925 13:56:16.536221 19955 rpcz_store.cc:276] 0925 13:56:14.228017 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:14.228131 (+ 114us) service_pool.cc:224] Handling call
0925 13:56:16.528963 (+2300832us) ts_tablet_manager.cc:1151] Deleting tablet f0cdc99cde2f4dabb819e3e171c5d8d9
0925 13:56:16.529007 (+ 44us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:16.529176 (+ 169us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2300331}
I20260925 13:56:16.546563 19275 master_service.cc:960] ReplaceTablet: received request to replace tablet 47521fb74f9c40ee999a7f704aa551c9 from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:16.452752 19938 tablet_bootstrap.cc:492] T b12a6c6cff024d70b53cc2c9169dd48b P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:16.319330 19839 raft_consensus.cc:2468] T f1c6e2137700448a8ebf859a9ca9085d P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:16.549331 19938 ts_tablet_manager.cc:1403] T b12a6c6cff024d70b53cc2c9169dd48b P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.603s user 0.053s sys 0.104s
I20260925 13:56:16.550714 19959 log.cc:1199] T a88bbe1ebee7468d8f49dfd25637fe83 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/a88bbe1ebee7468d8f49dfd25637fe83
I20260925 13:56:16.554361 19277 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_83acf26db75e4c02b312cef4df19901f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:16.555775 19277 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_83acf26db75e4c02b312cef4df19901f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:16.550014 19490 leader_election.cc:304] T f1c6e2137700448a8ebf859a9ca9085d P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:16.559293 20202 raft_consensus.cc:2804] T f1c6e2137700448a8ebf859a9ca9085d P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:16.559902 20202 raft_consensus.cc:697] T f1c6e2137700448a8ebf859a9ca9085d P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Running, Role: LEADER
I20260925 13:56:16.560897 20202 consensus_queue.cc:237] T f1c6e2137700448a8ebf859a9ca9085d P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:16.564092 19273 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 1832cff5eb3e4da4b9766ae1591df81a
I20260925 13:56:16.566388 19955 tablet_replica.cc:333] T 8239d0cd3b0d40bdb721e807c0f403d3 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:16.567611 19674 tablet_service.cc:1558] Processing DeleteTablet for tablet 1832cff5eb3e4da4b9766ae1591df81a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:16 UTC) from {username='slave'} at 127.0.0.1:45948
W20260925 13:56:16.568090 19264 catalog_manager.cc:5030] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): delete failed for tablet f0cdc99cde2f4dabb819e3e171c5d8d9 because the tablet was not found. No further retry: Not found: Tablet not found: f0cdc99cde2f4dabb819e3e171c5d8d9
I20260925 13:56:16.551678 19938 raft_consensus.cc:359] T b12a6c6cff024d70b53cc2c9169dd48b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:16.569373 19533 tablet_service.cc:1558] Processing DeleteTablet for tablet 1832cff5eb3e4da4b9766ae1591df81a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:16 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:16.569733 19938 raft_consensus.cc:385] T b12a6c6cff024d70b53cc2c9169dd48b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:16.570024 19938 raft_consensus.cc:740] T b12a6c6cff024d70b53cc2c9169dd48b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:16.570230 19959 ts_tablet_manager.cc:1950] T a88bbe1ebee7468d8f49dfd25637fe83 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:16.570660 19938 consensus_queue.cc:260] T b12a6c6cff024d70b53cc2c9169dd48b P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:16.572494 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 5943795bc8b24bb48b80d6d06d17750b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5943795bc8b24bb48b80d6d06d17750b already in progress: creating tablet
I20260925 13:56:16.572570 19938 ts_tablet_manager.cc:1434] T b12a6c6cff024d70b53cc2c9169dd48b P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.023s user 0.005s sys 0.000s
I20260925 13:56:16.568759 20190 raft_consensus.cc:493] T bbfc4f99a5964e2ba6146e390e8dba44 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:16.573380 20190 raft_consensus.cc:515] T bbfc4f99a5964e2ba6146e390e8dba44 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:16.574716 19942 tablet_bootstrap.cc:492] T 608bed5b04394bfca1776b5d6c502df1 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:16.575197 19942 ts_tablet_manager.cc:1403] T 608bed5b04394bfca1776b5d6c502df1 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.665s user 0.012s sys 0.009s
I20260925 13:56:16.576630 20190 leader_election.cc:290] T bbfc4f99a5964e2ba6146e390e8dba44 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:16.577647 19942 raft_consensus.cc:359] T 608bed5b04394bfca1776b5d6c502df1 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
W20260925 13:56:16.578200 20083 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
I20260925 13:56:16.578286 19942 raft_consensus.cc:385] T 608bed5b04394bfca1776b5d6c502df1 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:16.578275 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfc4f99a5964e2ba6146e390e8dba44" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
W20260925 13:56:16.578612 20083 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
I20260925 13:56:16.578624 19942 raft_consensus.cc:740] T 608bed5b04394bfca1776b5d6c502df1 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
W20260925 13:56:16.578966 20083 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
I20260925 13:56:16.579248 19942 consensus_queue.cc:260] T 608bed5b04394bfca1776b5d6c502df1 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:16.580456 19694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfc4f99a5964e2ba6146e390e8dba44" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:16.581809 19773 leader_election.cc:343] T bbfc4f99a5964e2ba6146e390e8dba44 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:16.582398 19773 leader_election.cc:343] T bbfc4f99a5964e2ba6146e390e8dba44 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:16.582712 19773 leader_election.cc:304] T bbfc4f99a5964e2ba6146e390e8dba44 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:16.583688 20190 raft_consensus.cc:2749] T bbfc4f99a5964e2ba6146e390e8dba44 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:16.590489 19938 tablet_bootstrap.cc:492] T e6e59d6f4f6e4de1be1cf450620cfa59 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:16.569298 19818 tablet_service.cc:1558] Processing DeleteTablet for tablet 1832cff5eb3e4da4b9766ae1591df81a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:16 UTC) from {username='slave'} at 127.0.0.1:51442
W20260925 13:56:16.579344 20083 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:56:16.612574 19271 catalog_manager.cc:6128] Not sending DeleteTablet requests; no consensus state for tablet 6e9452fa9c684756aeae6f87155da70f
I20260925 13:56:16.615739 20011 tablet_replica.cc:333] T f4429db544534f758883f2fae184ea6d P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:16.617550 19939 tablet_bootstrap.cc:492] T c65a8cd3060c405c871270543b4623f7 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:16.617954 19939 ts_tablet_manager.cc:1403] T c65a8cd3060c405c871270543b4623f7 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.130s user 0.027s sys 0.065s
W20260925 13:56:16.620424 19629 consensus_peers.cc:597] T 00b500182bb04a72b80cfa9103e85fa9 P 7c1b52895a14482fb54a6758f5054f8f -> Peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Couldn't send request to peer 8f578869bc084a5d8a9a69f143954eb3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 00b500182bb04a72b80cfa9103e85fa9. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:16.631791 20202 raft_consensus.cc:493] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:16.632274 20202 raft_consensus.cc:515] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:16.634248 20202 leader_election.cc:290] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:16.635890 19694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26644768f516412f8c95d68a76bb99c5" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:16.637934 19488 leader_election.cc:343] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:16.639611 19279 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet dc14c37aa393407a87af0d0eb8b88d82
I20260925 13:56:16.641201 19279 catalog_manager.cc:6934] ReplaceTablet: tablet dc14c37aa393407a87af0d0eb8b88d82 deleted and replaced by tablet 5f8a96ae6eec4f15a221f9aad537c094
I20260925 13:56:16.643970 19274 catalog_manager.cc:5838] T bae8c5b6e1014731901bc3f336104784 P 8f578869bc084a5d8a9a69f143954eb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "8f578869bc084a5d8a9a69f143954eb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: HEALTHY } } }
I20260925 13:56:16.645677 19274 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet bae8c5b6e1014731901bc3f336104784 (table default.Table_99593e98f3394811b4c91e234185cbef [id=0736e722eb8c43e9b6a151a7c2880fe3]). Expected version 1 got 0
I20260925 13:56:16.647053 19533 tablet_service.cc:1558] Processing DeleteTablet for tablet dc14c37aa393407a87af0d0eb8b88d82 with delete_type TABLET_DATA_DELETED (replaced by tablet 5f8a96ae6eec4f15a221f9aad537c094) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:16.653774 19274 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet 26644768f516412f8c95d68a76bb99c5 (table default.Table_bd52a86b8ff14bdbb06fcf88a60b1e82 [id=203024d67e444f839391c8670d7c7d3b]). Expected version 2 got 0
I20260925 13:56:16.655591 19676 tablet_service.cc:1558] Processing DeleteTablet for tablet dc14c37aa393407a87af0d0eb8b88d82 with delete_type TABLET_DATA_DELETED (replaced by tablet 5f8a96ae6eec4f15a221f9aad537c094) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:16.659085 19279 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_954c5af93adb4e8586f870597a63bb74"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:16.661129 19279 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_954c5af93adb4e8586f870597a63bb74 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:16.567234 19955 raft_consensus.cc:2243] T 8239d0cd3b0d40bdb721e807c0f403d3 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:16.665228 19955 raft_consensus.cc:2272] T 8239d0cd3b0d40bdb721e807c0f403d3 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:16.667465 19955 ts_tablet_manager.cc:1916] T 8239d0cd3b0d40bdb721e807c0f403d3 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:16.668172 19939 raft_consensus.cc:359] T c65a8cd3060c405c871270543b4623f7 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:16.668766 19939 raft_consensus.cc:385] T c65a8cd3060c405c871270543b4623f7 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:16.669013 19939 raft_consensus.cc:740] T c65a8cd3060c405c871270543b4623f7 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:16.669621 19939 consensus_queue.cc:260] T c65a8cd3060c405c871270543b4623f7 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:16.687110 20202 raft_consensus.cc:493] T 4d3dbd9796564472b44a31ccaf56d898 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:16.687628 20202 raft_consensus.cc:515] T 4d3dbd9796564472b44a31ccaf56d898 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:16.688349 19939 ts_tablet_manager.cc:1434] T c65a8cd3060c405c871270543b4623f7 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.070s user 0.006s sys 0.001s
I20260925 13:56:16.689854 20202 leader_election.cc:290] T 4d3dbd9796564472b44a31ccaf56d898 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:16.691679 19694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d3dbd9796564472b44a31ccaf56d898" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:16.693387 19488 leader_election.cc:343] T 4d3dbd9796564472b44a31ccaf56d898 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:16.694413 19939 tablet_bootstrap.cc:492] T f5a0bd0e205f4889be93739dfbd2a816 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:16.696898 20011 raft_consensus.cc:2243] T f4429db544534f758883f2fae184ea6d P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:16.697505 20011 raft_consensus.cc:2272] T f4429db544534f758883f2fae184ea6d P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:16.700335 20011 ts_tablet_manager.cc:1916] T f4429db544534f758883f2fae184ea6d P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:16.709257 19271 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_43b8599ad301426687e4035cec94b286"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:16.711798 20157 raft_consensus.cc:493] T 608bed5b04394bfca1776b5d6c502df1 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:16.712232 20157 raft_consensus.cc:515] T 608bed5b04394bfca1776b5d6c502df1 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:16.714237 20157 leader_election.cc:290] T 608bed5b04394bfca1776b5d6c502df1 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:16.716586 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "608bed5b04394bfca1776b5d6c502df1" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:16.717219 19554 raft_consensus.cc:2468] T 608bed5b04394bfca1776b5d6c502df1 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:16.718987 19629 leader_election.cc:304] T 608bed5b04394bfca1776b5d6c502df1 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3; no voters:
I20260925 13:56:16.719462 19939 tablet_bootstrap.cc:654] T f5a0bd0e205f4889be93739dfbd2a816 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:16.720010 20157 raft_consensus.cc:2804] T 608bed5b04394bfca1776b5d6c502df1 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:16.720351 20157 raft_consensus.cc:493] T 608bed5b04394bfca1776b5d6c502df1 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:16.720679 20157 raft_consensus.cc:3060] T 608bed5b04394bfca1776b5d6c502df1 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:16.721369 19955 ts_tablet_manager.cc:1929] T 8239d0cd3b0d40bdb721e807c0f403d3 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:16.722138 19955 log.cc:1199] T 8239d0cd3b0d40bdb721e807c0f403d3 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/8239d0cd3b0d40bdb721e807c0f403d3
I20260925 13:56:16.722707 19955 ts_tablet_manager.cc:1950] T 8239d0cd3b0d40bdb721e807c0f403d3 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
W20260925 13:56:16.723538 19271 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_43b8599ad301426687e4035cec94b286 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:16.724433 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 63) took 2493 ms. Trace:
I20260925 13:56:16.724740 19955 rpcz_store.cc:276] 0925 13:56:14.231434 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:14.231552 (+ 118us) service_pool.cc:224] Handling call
0925 13:56:16.566249 (+2334697us) ts_tablet_manager.cc:1151] Deleting tablet 8239d0cd3b0d40bdb721e807c0f403d3
0925 13:56:16.566288 (+ 39us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:16.684814 (+118526us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:16.721251 (+ 36437us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:16.724379 (+ 3128us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2334223}
I20260925 13:56:16.734664 20157 raft_consensus.cc:515] T 608bed5b04394bfca1776b5d6c502df1 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:16.737604 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet 8239d0cd3b0d40bdb721e807c0f403d3 (table default.Table_374792fc278744f5ad6b9c1818b7a370 [id=7bbf083be05c424fadc6fc2a1df86758]) successfully deleted
I20260925 13:56:16.739053 19955 tablet_replica.cc:333] T c0f05269ea6c43a98d2137eae7a9a401 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:16.739760 19955 raft_consensus.cc:2243] T c0f05269ea6c43a98d2137eae7a9a401 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:16.740720 19817 tablet_service.cc:1558] Processing DeleteTablet for tablet dc14c37aa393407a87af0d0eb8b88d82 with delete_type TABLET_DATA_DELETED (replaced by tablet 5f8a96ae6eec4f15a221f9aad537c094) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:16.740257 19955 raft_consensus.cc:2272] T c0f05269ea6c43a98d2137eae7a9a401 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:16.749372 19955 ts_tablet_manager.cc:1916] T c0f05269ea6c43a98d2137eae7a9a401 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:16.751780 19959 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 81) took 1371 ms. Trace:
I20260925 13:56:16.752220 19959 rpcz_store.cc:276] 0925 13:56:15.380380 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:15.380477 (+ 97us) service_pool.cc:224] Handling call
0925 13:56:15.832809 (+452332us) ts_tablet_manager.cc:1151] Deleting tablet a88bbe1ebee7468d8f49dfd25637fe83
0925 13:56:15.832868 (+ 59us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:16.328118 (+495250us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:16.438907 (+110789us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:16.751712 (+312805us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":111807,"tablet-delete.queue_time_us":451829}
I20260925 13:56:16.754211 19938 tablet_bootstrap.cc:654] T e6e59d6f4f6e4de1be1cf450620cfa59 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:16.756585 19839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26644768f516412f8c95d68a76bb99c5" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:16.757145 19839 raft_consensus.cc:2468] T 26644768f516412f8c95d68a76bb99c5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:16.758625 19839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d3dbd9796564472b44a31ccaf56d898" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:16.759119 19839 raft_consensus.cc:2468] T 4d3dbd9796564472b44a31ccaf56d898 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:16.760310 19490 leader_election.cc:304] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:16.761255 20202 raft_consensus.cc:2804] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:16.761600 20202 raft_consensus.cc:493] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:16.761906 20202 raft_consensus.cc:3060] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:16.764757 19955 ts_tablet_manager.cc:1929] T c0f05269ea6c43a98d2137eae7a9a401 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:56:16.765079 19955 log.cc:1199] T c0f05269ea6c43a98d2137eae7a9a401 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/c0f05269ea6c43a98d2137eae7a9a401
I20260925 13:56:16.765733 19955 ts_tablet_manager.cc:1950] T c0f05269ea6c43a98d2137eae7a9a401 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:16.767477 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 68) took 2299 ms. Trace:
I20260925 13:56:16.767761 19955 rpcz_store.cc:276] 0925 13:56:14.468038 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:14.468163 (+ 125us) service_pool.cc:224] Handling call
0925 13:56:16.738907 (+2270744us) ts_tablet_manager.cc:1151] Deleting tablet c0f05269ea6c43a98d2137eae7a9a401
0925 13:56:16.738946 (+ 39us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:16.758455 (+ 19509us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:16.764661 (+ 6206us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:16.767424 (+ 2763us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2270177}
I20260925 13:56:16.767851 20202 raft_consensus.cc:515] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:16.769030 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet c0f05269ea6c43a98d2137eae7a9a401 (table default.Table_72b4fdbfbca745a286dca8c191dc2286 [id=41786d7cdac8440ca86768cac5b28f49]) successfully deleted
I20260925 13:56:16.771376 19694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26644768f516412f8c95d68a76bb99c5" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:16.771620 19839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26644768f516412f8c95d68a76bb99c5" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:16.772071 19839 raft_consensus.cc:3060] T 26644768f516412f8c95d68a76bb99c5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:16.772140 20157 leader_election.cc:290] T 608bed5b04394bfca1776b5d6c502df1 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:16.740298 19942 ts_tablet_manager.cc:1434] T 608bed5b04394bfca1776b5d6c502df1 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.165s user 0.003s sys 0.002s
I20260925 13:56:16.771612 19490 leader_election.cc:304] T 4d3dbd9796564472b44a31ccaf56d898 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:16.773821 20199 raft_consensus.cc:2804] T 4d3dbd9796564472b44a31ccaf56d898 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:16.774121 20199 raft_consensus.cc:493] T 4d3dbd9796564472b44a31ccaf56d898 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:16.774396 20199 raft_consensus.cc:3060] T 4d3dbd9796564472b44a31ccaf56d898 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:16.775427 19942 tablet_bootstrap.cc:492] T f1c6e2137700448a8ebf859a9ca9085d P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:16.779151 19273 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_29949770026642b09db53126aa0acf63"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:16.780704 19273 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_29949770026642b09db53126aa0acf63 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:16.771907 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "608bed5b04394bfca1776b5d6c502df1" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:16.781268 19554 raft_consensus.cc:3060] T 608bed5b04394bfca1776b5d6c502df1 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:16.783272 19839 raft_consensus.cc:2468] T 26644768f516412f8c95d68a76bb99c5 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:16.785843 20083 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.182.212:43091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-2/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.18.182.254:40825,127.18.182.253:39641,127.18.182.252:36181
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.182.252:36181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.182.252
--webserver_port=42511
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 102683f65af844d269565345d7d8c6ebc55c92c5
build type FASTDEBUG
built by None at 25 Sep 2026 13:43:25 UTC on e7f111948823
build id 13994
TSAN enabled
I20260925 13:56:16.786401 19939 tablet_bootstrap.cc:492] T f5a0bd0e205f4889be93739dfbd2a816 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:16.786765 19939 ts_tablet_manager.cc:1403] T f5a0bd0e205f4889be93739dfbd2a816 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.093s user 0.006s sys 0.003s
I20260925 13:56:16.787628 20083 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:56:16.790143 20083 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260925 13:56:16.772747 19488 leader_election.cc:343] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:16.790971 19939 raft_consensus.cc:359] T f5a0bd0e205f4889be93739dfbd2a816 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:16.791689 19939 raft_consensus.cc:385] T f5a0bd0e205f4889be93739dfbd2a816 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:16.791852 19490 leader_election.cc:304] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:16.792613 20198 raft_consensus.cc:2804] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:16.793066 20198 raft_consensus.cc:697] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Running, Role: LEADER
I20260925 13:56:16.793817 20198 consensus_queue.cc:237] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:16.791968 19939 raft_consensus.cc:740] T f5a0bd0e205f4889be93739dfbd2a816 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:16.795774 19939 consensus_queue.cc:260] T f5a0bd0e205f4889be93739dfbd2a816 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:16.798939 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet a88bbe1ebee7468d8f49dfd25637fe83 (table default.Table_65608937f2cd4986b2588a4831750601 [id=7079cc0c725b4578a1b11c7dd3ef9342]) successfully deleted
I20260925 13:56:16.799890 19939 ts_tablet_manager.cc:1434] T f5a0bd0e205f4889be93739dfbd2a816 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.013s user 0.007s sys 0.000s
I20260925 13:56:16.800649 19939 tablet_bootstrap.cc:492] T bbfc4f99a5964e2ba6146e390e8dba44 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
W20260925 13:56:16.801389 19773 consensus_peers.cc:597] T f4429db544534f758883f2fae184ea6d P f790493ba7ad403993b678f2df2b55be -> Peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Couldn't send request to peer 7c1b52895a14482fb54a6758f5054f8f. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
W20260925 13:56:16.801884 19631 leader_election.cc:343] T 608bed5b04394bfca1776b5d6c502df1 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 608bed5b04394bfca1776b5d6c502df1
W20260925 13:56:16.802703 19631 leader_election.cc:343] T 608bed5b04394bfca1776b5d6c502df1 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 608bed5b04394bfca1776b5d6c502df1
I20260925 13:56:16.803617 19275 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 47521fb74f9c40ee999a7f704aa551c9
I20260925 13:56:16.803715 19959 tablet_replica.cc:333] T 7292d04b411c415394e9d0df85aaf579 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:16.804425 19959 raft_consensus.cc:2243] T 7292d04b411c415394e9d0df85aaf579 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:16.804844 19959 raft_consensus.cc:2272] T 7292d04b411c415394e9d0df85aaf579 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:16.805181 19275 catalog_manager.cc:6934] ReplaceTablet: tablet 47521fb74f9c40ee999a7f704aa551c9 deleted and replaced by tablet 000aad39953c4688b543c7d882a08ec8
I20260925 13:56:16.806030 19275 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_ba97a54a19744b058cffd5638ffb97db" } modify_external_catalogs: true
I20260925 13:56:16.807835 19275 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_ba97a54a19744b058cffd5638ffb97db" } modify_external_catalogs: true
I20260925 13:56:16.810065 19533 tablet_service.cc:1558] Processing DeleteTablet for tablet 47521fb74f9c40ee999a7f704aa551c9 with delete_type TABLET_DATA_DELETED (replaced by tablet 000aad39953c4688b543c7d882a08ec8) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:16.813661 19959 ts_tablet_manager.cc:1916] T 7292d04b411c415394e9d0df85aaf579 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:16.814457 19938 tablet_bootstrap.cc:492] T e6e59d6f4f6e4de1be1cf450620cfa59 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:16.814836 19938 ts_tablet_manager.cc:1403] T e6e59d6f4f6e4de1be1cf450620cfa59 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.225s user 0.014s sys 0.016s
I20260925 13:56:16.817727 19672 tablet_service.cc:1558] Processing DeleteTablet for tablet 47521fb74f9c40ee999a7f704aa551c9 with delete_type TABLET_DATA_DELETED (replaced by tablet 000aad39953c4688b543c7d882a08ec8) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:16.818406 19820 tablet_service.cc:1558] Processing DeleteTablet for tablet 47521fb74f9c40ee999a7f704aa551c9 with delete_type TABLET_DATA_DELETED (replaced by tablet 000aad39953c4688b543c7d882a08ec8) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:16.818567 19938 raft_consensus.cc:359] T e6e59d6f4f6e4de1be1cf450620cfa59 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:16.819195 19938 raft_consensus.cc:385] T e6e59d6f4f6e4de1be1cf450620cfa59 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:16.819465 19938 raft_consensus.cc:740] T e6e59d6f4f6e4de1be1cf450620cfa59 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:16.820108 20199 raft_consensus.cc:515] T 4d3dbd9796564472b44a31ccaf56d898 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:16.820245 19938 consensus_queue.cc:260] T e6e59d6f4f6e4de1be1cf450620cfa59 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:16.821655 19942 tablet_bootstrap.cc:654] T f1c6e2137700448a8ebf859a9ca9085d P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:16.822499 19939 tablet_bootstrap.cc:654] T bbfc4f99a5964e2ba6146e390e8dba44 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:16.822630 19694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d3dbd9796564472b44a31ccaf56d898" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
W20260925 13:56:16.824002 19488 leader_election.cc:343] T 4d3dbd9796564472b44a31ccaf56d898 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:16.829761 19959 ts_tablet_manager.cc:1929] T 7292d04b411c415394e9d0df85aaf579 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:16.830088 19959 log.cc:1199] T 7292d04b411c415394e9d0df85aaf579 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/7292d04b411c415394e9d0df85aaf579
I20260925 13:56:16.830765 19959 ts_tablet_manager.cc:1950] T 7292d04b411c415394e9d0df85aaf579 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:16.835762 19938 ts_tablet_manager.cc:1434] T e6e59d6f4f6e4de1be1cf450620cfa59 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.021s user 0.006s sys 0.000s
I20260925 13:56:16.836694 19277 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_72b4fdbfbca745a286dca8c191dc2286" } modify_external_catalogs: true
I20260925 13:56:16.837399 19277 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_72b4fdbfbca745a286dca8c191dc2286" } modify_external_catalogs: true
I20260925 13:56:16.843546 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 7292d04b411c415394e9d0df85aaf579 (table default.Table_06cdecf307e549bf908eb64143c31996 [id=d2bc135460ab4f2fad4740a281bfc6e2]) successfully deleted
I20260925 13:56:16.848381 19959 tablet_replica.cc:333] T f4429db544534f758883f2fae184ea6d P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:16.864642 19938 tablet_bootstrap.cc:492] T 8dc5ea164d6749b0926c63d6bda9dcc5 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:16.869400 19272 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 18a6d1c4eb7948149e709a380da50d93 (table default.Table_abc00b1f4c4c455e8086a46c395f2656 [id=6c0ec9465714408da93ff6e47827eb48]). Expected version 1 got 0
I20260925 13:56:16.870020 19938 tablet_bootstrap.cc:654] T 8dc5ea164d6749b0926c63d6bda9dcc5 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:16.870432 19272 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet bae8c5b6e1014731901bc3f336104784 (table default.Table_99593e98f3394811b4c91e234185cbef [id=0736e722eb8c43e9b6a151a7c2880fe3]). Expected version 1 got 0
W20260925 13:56:16.875834 19280 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet eb794cef052b43da831531455e2b5f15 has no consensus state
I20260925 13:56:16.883471 19939 tablet_bootstrap.cc:492] T bbfc4f99a5964e2ba6146e390e8dba44 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:16.884361 19820 tablet_service.cc:1558] Processing DeleteTablet for tablet 7292d04b411c415394e9d0df85aaf579 with delete_type TABLET_DATA_DELETED (replaced by tablet eb794cef052b43da831531455e2b5f15) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:16.889802 19938 tablet_bootstrap.cc:492] T 8dc5ea164d6749b0926c63d6bda9dcc5 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:16.890205 19938 ts_tablet_manager.cc:1403] T 8dc5ea164d6749b0926c63d6bda9dcc5 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.026s user 0.008s sys 0.000s
I20260925 13:56:16.893590 19939 ts_tablet_manager.cc:1403] T bbfc4f99a5964e2ba6146e390e8dba44 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.093s user 0.009s sys 0.000s
I20260925 13:56:16.892575 19938 raft_consensus.cc:359] T 8dc5ea164d6749b0926c63d6bda9dcc5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:16.895712 19938 raft_consensus.cc:385] T 8dc5ea164d6749b0926c63d6bda9dcc5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:16.895977 19938 raft_consensus.cc:740] T 8dc5ea164d6749b0926c63d6bda9dcc5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:16.897359 19959 raft_consensus.cc:2243] T f4429db544534f758883f2fae184ea6d P f790493ba7ad403993b678f2df2b55be [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:16.898182 19959 raft_consensus.cc:2272] T f4429db544534f758883f2fae184ea6d P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:16.900067 19959 ts_tablet_manager.cc:1916] T f4429db544534f758883f2fae184ea6d P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:16.901569 20011 ts_tablet_manager.cc:1929] T f4429db544534f758883f2fae184ea6d P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:16.912894 20011 log.cc:1199] T f4429db544534f758883f2fae184ea6d P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/f4429db544534f758883f2fae184ea6d
I20260925 13:56:16.913934 20011 ts_tablet_manager.cc:1950] T f4429db544534f758883f2fae184ea6d P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:16.913177 19938 consensus_queue.cc:260] T 8dc5ea164d6749b0926c63d6bda9dcc5 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:16.911917 19939 raft_consensus.cc:359] T bbfc4f99a5964e2ba6146e390e8dba44 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:16.915428 19939 raft_consensus.cc:385] T bbfc4f99a5964e2ba6146e390e8dba44 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:16.916005 19939 raft_consensus.cc:740] T bbfc4f99a5964e2ba6146e390e8dba44 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:16.917656 19938 ts_tablet_manager.cc:1434] T 8dc5ea164d6749b0926c63d6bda9dcc5 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.027s user 0.006s sys 0.000s
I20260925 13:56:16.904457 19942 tablet_bootstrap.cc:492] T f1c6e2137700448a8ebf859a9ca9085d P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:16.918174 19942 ts_tablet_manager.cc:1403] T f1c6e2137700448a8ebf859a9ca9085d P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.143s user 0.010s sys 0.001s
I20260925 13:56:16.921123 19959 ts_tablet_manager.cc:1929] T f4429db544534f758883f2fae184ea6d P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:16.921252 20177 raft_consensus.cc:493] T e6e59d6f4f6e4de1be1cf450620cfa59 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:16.921453 19959 log.cc:1199] T f4429db544534f758883f2fae184ea6d P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/f4429db544534f758883f2fae184ea6d
I20260925 13:56:16.921743 20177 raft_consensus.cc:515] T e6e59d6f4f6e4de1be1cf450620cfa59 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:16.922142 19959 ts_tablet_manager.cc:1950] T f4429db544534f758883f2fae184ea6d P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:16.924321 19938 tablet_bootstrap.cc:492] T 1832cff5eb3e4da4b9766ae1591df81a P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:16.924779 19939 consensus_queue.cc:260] T bbfc4f99a5964e2ba6146e390e8dba44 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:16.928064 20177 leader_election.cc:290] T e6e59d6f4f6e4de1be1cf450620cfa59 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:16.931756 19694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e59d6f4f6e4de1be1cf450620cfa59" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:16.933284 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e59d6f4f6e4de1be1cf450620cfa59" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
W20260925 13:56:16.934875 19773 leader_election.cc:343] T e6e59d6f4f6e4de1be1cf450620cfa59 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:16.937223 19773 leader_election.cc:343] T e6e59d6f4f6e4de1be1cf450620cfa59 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:16.937575 19773 leader_election.cc:304] T e6e59d6f4f6e4de1be1cf450620cfa59 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:16.937793 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet f4429db544534f758883f2fae184ea6d (table default.Table_731e8683c6714e22a725eae1b86ff4ec [id=e441f4e1a35e400da9cd488ac1d4a754]) successfully deleted
I20260925 13:56:16.938506 20177 raft_consensus.cc:2749] T e6e59d6f4f6e4de1be1cf450620cfa59 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:16.939375 20011 tablet_replica.cc:333] T 00b500182bb04a72b80cfa9103e85fa9 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:16.940026 20011 raft_consensus.cc:2243] T 00b500182bb04a72b80cfa9103e85fa9 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:16.940721 20011 pending_rounds.cc:70] T 00b500182bb04a72b80cfa9103e85fa9 P 7c1b52895a14482fb54a6758f5054f8f: Trying to abort 1 pending ops.
I20260925 13:56:16.940946 20011 pending_rounds.cc:77] T 00b500182bb04a72b80cfa9103e85fa9 P 7c1b52895a14482fb54a6758f5054f8f: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251381320704000 op_type: NO_OP noop_request { }
I20260925 13:56:16.941253 20011 raft_consensus.cc:2889] T 00b500182bb04a72b80cfa9103e85fa9 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:56:16.941553 20011 raft_consensus.cc:2272] T 00b500182bb04a72b80cfa9103e85fa9 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:16.941723 19938 tablet_bootstrap.cc:654] T 1832cff5eb3e4da4b9766ae1591df81a P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:16.946839 19939 ts_tablet_manager.cc:1434] T bbfc4f99a5964e2ba6146e390e8dba44 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.052s user 0.005s sys 0.000s
I20260925 13:56:16.947680 19939 tablet_bootstrap.cc:492] T b12a6c6cff024d70b53cc2c9169dd48b P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:16.920899 19942 raft_consensus.cc:359] T f1c6e2137700448a8ebf859a9ca9085d P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:16.956982 19938 tablet_bootstrap.cc:492] T 1832cff5eb3e4da4b9766ae1591df81a P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:16.957340 19938 ts_tablet_manager.cc:1403] T 1832cff5eb3e4da4b9766ae1591df81a P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.033s user 0.009s sys 0.000s
I20260925 13:56:16.958575 19959 tablet_replica.cc:333] T 8239d0cd3b0d40bdb721e807c0f403d3 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:16.964944 19959 raft_consensus.cc:2243] T 8239d0cd3b0d40bdb721e807c0f403d3 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:16.965435 19959 raft_consensus.cc:2272] T 8239d0cd3b0d40bdb721e807c0f403d3 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:16.967147 19959 ts_tablet_manager.cc:1916] T 8239d0cd3b0d40bdb721e807c0f403d3 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:16.963261 19938 raft_consensus.cc:359] T 1832cff5eb3e4da4b9766ae1591df81a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:16.967800 19938 raft_consensus.cc:385] T 1832cff5eb3e4da4b9766ae1591df81a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:16.968071 19938 raft_consensus.cc:740] T 1832cff5eb3e4da4b9766ae1591df81a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:16.970286 19554 raft_consensus.cc:2468] T 608bed5b04394bfca1776b5d6c502df1 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
I20260925 13:56:16.970423 19938 consensus_queue.cc:260] T 1832cff5eb3e4da4b9766ae1591df81a P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:16.958802 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet f4429db544534f758883f2fae184ea6d (table default.Table_731e8683c6714e22a725eae1b86ff4ec [id=e441f4e1a35e400da9cd488ac1d4a754]) successfully deleted
I20260925 13:56:16.986030 19278 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet e6e59d6f4f6e4de1be1cf450620cfa59 (table default.Table_546097946d054781b0acae5186c5d43d [id=912de4c02216461f8a69b880207d8652]). Expected version 1 got 0
I20260925 13:56:16.986768 19278 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 18a6d1c4eb7948149e709a380da50d93 (table default.Table_e814702f1f874529a98bd1631d3b5c84 [id=6c0ec9465714408da93ff6e47827eb48]). Expected version 2 got 0
I20260925 13:56:16.988997 19534 tablet_service.cc:1558] Processing DeleteTablet for tablet 8239d0cd3b0d40bdb721e807c0f403d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:14 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:16.770687 20202 leader_election.cc:290] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:17.001946 19838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d3dbd9796564472b44a31ccaf56d898" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:17.002540 19838 raft_consensus.cc:3060] T 4d3dbd9796564472b44a31ccaf56d898 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:17.003113 19938 ts_tablet_manager.cc:1434] T 1832cff5eb3e4da4b9766ae1591df81a P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.046s user 0.007s sys 0.001s
I20260925 13:56:17.006433 19938 tablet_bootstrap.cc:492] T 1da6eadb1648479889027fa911f4676b P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:17.010689 19838 raft_consensus.cc:2468] T 4d3dbd9796564472b44a31ccaf56d898 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:16.999383 20198 raft_consensus.cc:493] T a88bbe1ebee7468d8f49dfd25637fe83 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:17.025117 20198 raft_consensus.cc:515] T a88bbe1ebee7468d8f49dfd25637fe83 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.025628 19938 tablet_bootstrap.cc:654] T 1da6eadb1648479889027fa911f4676b P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:16.770287 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 74) took 1651 ms. Trace:
I20260925 13:56:17.027900 19955 rpcz_store.cc:276] 0925 13:56:15.118555 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:15.118688 (+ 133us) service_pool.cc:224] Handling call
0925 13:56:16.769991 (+1651303us) ts_tablet_manager.cc:1151] Deleting tablet 2a1f34dba13143b097a38bab3782c404
0925 13:56:16.770027 (+ 36us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:16.770238 (+ 211us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1601389}
I20260925 13:56:17.029042 19955 tablet_replica.cc:333] T 608bed5b04394bfca1776b5d6c502df1 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:17.000468 20199 leader_election.cc:290] T 4d3dbd9796564472b44a31ccaf56d898 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:16.943184 20011 ts_tablet_manager.cc:1916] T 00b500182bb04a72b80cfa9103e85fa9 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:17.039539 19490 leader_election.cc:304] T 4d3dbd9796564472b44a31ccaf56d898 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:17.040212 19942 raft_consensus.cc:385] T f1c6e2137700448a8ebf859a9ca9085d P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:17.040519 19942 raft_consensus.cc:740] T f1c6e2137700448a8ebf859a9ca9085d P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:17.040704 20199 raft_consensus.cc:2804] T 4d3dbd9796564472b44a31ccaf56d898 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:17.041416 20199 raft_consensus.cc:697] T 4d3dbd9796564472b44a31ccaf56d898 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Running, Role: LEADER
I20260925 13:56:17.041230 19942 consensus_queue.cc:260] T f1c6e2137700448a8ebf859a9ca9085d P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.042243 20199 consensus_queue.cc:237] T 4d3dbd9796564472b44a31ccaf56d898 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:16.984149 19672 tablet_service.cc:1558] Processing DeleteTablet for tablet 7292d04b411c415394e9d0df85aaf579 with delete_type TABLET_DATA_DELETED (replaced by tablet eb794cef052b43da831531455e2b5f15) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:16.985227 19676 tablet_service.cc:1558] Processing DeleteTablet for tablet 980d1264c0ea45f3b2b762509e0b2c77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:13 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:16.985525 19629 leader_election.cc:304] T 608bed5b04394bfca1776b5d6c502df1 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3; no voters: f790493ba7ad403993b678f2df2b55be
I20260925 13:56:17.045987 19938 tablet_bootstrap.cc:492] T 1da6eadb1648479889027fa911f4676b P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:17.046404 19938 ts_tablet_manager.cc:1403] T 1da6eadb1648479889027fa911f4676b P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.040s user 0.008s sys 0.000s
I20260925 13:56:17.047501 19694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a88bbe1ebee7468d8f49dfd25637fe83" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:17.048115 19264 catalog_manager.cc:5043] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): delete failed for tablet 2a1f34dba13143b097a38bab3782c404 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2a1f34dba13143b097a38bab3782c404 already in progress: creating tablet
W20260925 13:56:17.048820 19488 leader_election.cc:343] T a88bbe1ebee7468d8f49dfd25637fe83 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:17.048714 19938 raft_consensus.cc:359] T 1da6eadb1648479889027fa911f4676b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.049307 19938 raft_consensus.cc:385] T 1da6eadb1648479889027fa911f4676b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:17.049554 19938 raft_consensus.cc:740] T 1da6eadb1648479889027fa911f4676b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:17.055532 19939 tablet_bootstrap.cc:654] T b12a6c6cff024d70b53cc2c9169dd48b P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:17.059698 20157 raft_consensus.cc:2804] T 608bed5b04394bfca1776b5d6c502df1 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:17.062167 19942 ts_tablet_manager.cc:1434] T f1c6e2137700448a8ebf859a9ca9085d P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.144s user 0.001s sys 0.009s
I20260925 13:56:17.060395 19938 consensus_queue.cc:260] T 1da6eadb1648479889027fa911f4676b P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.063002 19942 tablet_bootstrap.cc:492] T 4d3dbd9796564472b44a31ccaf56d898 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:17.067958 19938 ts_tablet_manager.cc:1434] T 1da6eadb1648479889027fa911f4676b P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.021s user 0.007s sys 0.000s
W20260925 13:56:17.071152 19629 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.193:38269, user_credentials={real_user=slave}} blocked reactor thread for 85753.2us
I20260925 13:56:17.073331 20198 leader_election.cc:290] T a88bbe1ebee7468d8f49dfd25637fe83 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:17.072870 19938 tablet_bootstrap.cc:492] T 2a1f34dba13143b097a38bab3782c404 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:17.074584 19959 ts_tablet_manager.cc:1929] T 8239d0cd3b0d40bdb721e807c0f403d3 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:17.075728 19959 log.cc:1199] T 8239d0cd3b0d40bdb721e807c0f403d3 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/8239d0cd3b0d40bdb721e807c0f403d3
I20260925 13:56:17.077977 19959 ts_tablet_manager.cc:1950] T 8239d0cd3b0d40bdb721e807c0f403d3 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:17.079609 19938 tablet_bootstrap.cc:654] T 2a1f34dba13143b097a38bab3782c404 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:17.084910 19955 raft_consensus.cc:2243] T 608bed5b04394bfca1776b5d6c502df1 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:17.085489 19955 raft_consensus.cc:2272] T 608bed5b04394bfca1776b5d6c502df1 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:17.086342 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 8239d0cd3b0d40bdb721e807c0f403d3 (table default.Table_374792fc278744f5ad6b9c1818b7a370 [id=7bbf083be05c424fadc6fc2a1df86758]) successfully deleted
I20260925 13:56:17.088481 19942 tablet_bootstrap.cc:654] T 4d3dbd9796564472b44a31ccaf56d898 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:17.088976 19939 tablet_bootstrap.cc:492] T b12a6c6cff024d70b53cc2c9169dd48b P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:17.089468 19939 ts_tablet_manager.cc:1403] T b12a6c6cff024d70b53cc2c9169dd48b P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.142s user 0.009s sys 0.000s
I20260925 13:56:17.094457 19959 tablet_replica.cc:333] T 5943795bc8b24bb48b80d6d06d17750b P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:17.096477 19959 raft_consensus.cc:2243] T 5943795bc8b24bb48b80d6d06d17750b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:17.091892 19939 raft_consensus.cc:359] T b12a6c6cff024d70b53cc2c9169dd48b P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.100312 19939 raft_consensus.cc:385] T b12a6c6cff024d70b53cc2c9169dd48b P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:17.100772 19959 raft_consensus.cc:2272] T 5943795bc8b24bb48b80d6d06d17750b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:17.104043 19939 raft_consensus.cc:740] T b12a6c6cff024d70b53cc2c9169dd48b P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:17.106451 19959 ts_tablet_manager.cc:1916] T 5943795bc8b24bb48b80d6d06d17750b P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:17.107090 19939 consensus_queue.cc:260] T b12a6c6cff024d70b53cc2c9169dd48b P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.113418 20177 raft_consensus.cc:493] T b12a6c6cff024d70b53cc2c9169dd48b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:17.113837 20177 raft_consensus.cc:515] T b12a6c6cff024d70b53cc2c9169dd48b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.115613 20177 leader_election.cc:290] T b12a6c6cff024d70b53cc2c9169dd48b P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:17.117570 19691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12a6c6cff024d70b53cc2c9169dd48b" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:17.118873 19773 leader_election.cc:343] T b12a6c6cff024d70b53cc2c9169dd48b P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:17.123364 19942 tablet_bootstrap.cc:492] T 4d3dbd9796564472b44a31ccaf56d898 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:17.123747 19942 ts_tablet_manager.cc:1403] T 4d3dbd9796564472b44a31ccaf56d898 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.061s user 0.009s sys 0.000s
I20260925 13:56:17.125325 20240 consensus_queue.cc:1048] T dc14c37aa393407a87af0d0eb8b88d82 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:17.126564 19938 tablet_bootstrap.cc:492] T 2a1f34dba13143b097a38bab3782c404 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:17.126962 19938 ts_tablet_manager.cc:1403] T 2a1f34dba13143b097a38bab3782c404 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.054s user 0.013s sys 0.000s
I20260925 13:56:17.129678 19938 raft_consensus.cc:359] T 2a1f34dba13143b097a38bab3782c404 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.130482 19938 raft_consensus.cc:385] T 2a1f34dba13143b097a38bab3782c404 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:17.130767 19938 raft_consensus.cc:740] T 2a1f34dba13143b097a38bab3782c404 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:17.134382 19939 ts_tablet_manager.cc:1434] T b12a6c6cff024d70b53cc2c9169dd48b P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.045s user 0.005s sys 0.000s
I20260925 13:56:17.136126 19942 raft_consensus.cc:359] T 4d3dbd9796564472b44a31ccaf56d898 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.136688 19942 raft_consensus.cc:385] T 4d3dbd9796564472b44a31ccaf56d898 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:17.136919 19942 raft_consensus.cc:740] T 4d3dbd9796564472b44a31ccaf56d898 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:17.137413 19939 tablet_bootstrap.cc:492] T 1da6eadb1648479889027fa911f4676b P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:17.141709 20011 ts_tablet_manager.cc:1929] T 00b500182bb04a72b80cfa9103e85fa9 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:17.141999 19938 consensus_queue.cc:260] T 2a1f34dba13143b097a38bab3782c404 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.143921 20011 log.cc:1199] T 00b500182bb04a72b80cfa9103e85fa9 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/00b500182bb04a72b80cfa9103e85fa9
I20260925 13:56:17.144695 20011 ts_tablet_manager.cc:1950] T 00b500182bb04a72b80cfa9103e85fa9 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:17.144924 19942 consensus_queue.cc:260] T 4d3dbd9796564472b44a31ccaf56d898 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.147955 20011 tablet_replica.cc:333] T dc14c37aa393407a87af0d0eb8b88d82 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:17.148635 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 00b500182bb04a72b80cfa9103e85fa9 (table default.Table_04d053f331f243ce8e98beb7c00d4159 [id=745c9e0d4f2c4ce58c3a201c34f1f1e3]) successfully deleted
W20260925 13:56:17.149545 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 4d3dbd9796564472b44a31ccaf56d898 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4d3dbd9796564472b44a31ccaf56d898 already in progress: creating tablet
W20260925 13:56:17.151046 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 1832cff5eb3e4da4b9766ae1591df81a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1832cff5eb3e4da4b9766ae1591df81a already in progress: creating tablet
I20260925 13:56:17.155099 19275 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet f514fa52e7524cd3ba07f3f65de532d4
I20260925 13:56:17.156180 19938 ts_tablet_manager.cc:1434] T 2a1f34dba13143b097a38bab3782c404 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.029s user 0.006s sys 0.000s
I20260925 13:56:17.158154 19820 tablet_service.cc:1558] Processing DeleteTablet for tablet f514fa52e7524cd3ba07f3f65de532d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:16 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:17.162771 19939 tablet_bootstrap.cc:654] T 1da6eadb1648479889027fa911f4676b P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:17.162966 19533 tablet_service.cc:1558] Processing DeleteTablet for tablet f514fa52e7524cd3ba07f3f65de532d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:16 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:17.167609 19277 catalog_manager.cc:6128] Not sending DeleteTablet requests; no consensus state for tablet 371b5b10fc4948f49c515de7b9049220
I20260925 13:56:17.169687 19938 tablet_bootstrap.cc:492] T f514fa52e7524cd3ba07f3f65de532d4 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:17.142313 20157 raft_consensus.cc:697] T 608bed5b04394bfca1776b5d6c502df1 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Becoming Leader. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Running, Role: LEADER
I20260925 13:56:17.171036 20157 consensus_queue.cc:237] T 608bed5b04394bfca1776b5d6c502df1 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:17.175266 19676 tablet_service.cc:1558] Processing DeleteTablet for tablet f514fa52e7524cd3ba07f3f65de532d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:16 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:17.178458 20177 raft_consensus.cc:493] T 2a1f34dba13143b097a38bab3782c404 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:17.178905 20177 raft_consensus.cc:515] T 2a1f34dba13143b097a38bab3782c404 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.180637 20177 leader_election.cc:290] T 2a1f34dba13143b097a38bab3782c404 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:17.183446 19696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a1f34dba13143b097a38bab3782c404" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:17.183349 20246 consensus_queue.cc:1048] T bae8c5b6e1014731901bc3f336104784 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260925 13:56:17.185660 19773 leader_election.cc:343] T 2a1f34dba13143b097a38bab3782c404 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:17.188977 19938 tablet_bootstrap.cc:654] T f514fa52e7524cd3ba07f3f65de532d4 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:17.190023 19278 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_8a18eca530ed42b7808ab612d9bad5e7" } new_table_name: "default.Table_3a06919b658e46f893318b4e1ec8b52f" modify_external_catalogs: true
I20260925 13:56:17.194464 19273 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_cd0b384bb04c44759472fcffcab11ee3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:17.197031 19939 tablet_bootstrap.cc:492] T 1da6eadb1648479889027fa911f4676b P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:17.197423 19939 ts_tablet_manager.cc:1403] T 1da6eadb1648479889027fa911f4676b P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.060s user 0.008s sys 0.000s
I20260925 13:56:17.197190 19279 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_6f8114f6052845bb85b139bfedbed143"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:17.199738 19939 raft_consensus.cc:359] T 1da6eadb1648479889027fa911f4676b P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.200344 19939 raft_consensus.cc:385] T 1da6eadb1648479889027fa911f4676b P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:17.200572 19939 raft_consensus.cc:740] T 1da6eadb1648479889027fa911f4676b P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:17.201122 19939 consensus_queue.cc:260] T 1da6eadb1648479889027fa911f4676b P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:17.198949 19279 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6f8114f6052845bb85b139bfedbed143 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:17.206391 19938 tablet_bootstrap.cc:492] T f514fa52e7524cd3ba07f3f65de532d4 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:17.206758 19938 ts_tablet_manager.cc:1403] T f514fa52e7524cd3ba07f3f65de532d4 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.037s user 0.010s sys 0.000s
I20260925 13:56:17.201790 19280 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_50d8afbc179c4db0a81fff129311fa10"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:17.208194 19280 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_50d8afbc179c4db0a81fff129311fa10 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:17.209051 19938 raft_consensus.cc:359] T f514fa52e7524cd3ba07f3f65de532d4 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.185655 19959 ts_tablet_manager.cc:1929] T 5943795bc8b24bb48b80d6d06d17750b P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:17.209674 19938 raft_consensus.cc:385] T f514fa52e7524cd3ba07f3f65de532d4 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:17.209991 19959 log.cc:1199] T 5943795bc8b24bb48b80d6d06d17750b P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/5943795bc8b24bb48b80d6d06d17750b
I20260925 13:56:17.210623 19959 ts_tablet_manager.cc:1950] T 5943795bc8b24bb48b80d6d06d17750b P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:17.202907 19276 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_b42a6e2de4e74cb7a6f01dba22f3b94c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:17.212419 19276 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b42a6e2de4e74cb7a6f01dba22f3b94c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:17.215538 19939 ts_tablet_manager.cc:1434] T 1da6eadb1648479889027fa911f4676b P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.018s user 0.005s sys 0.000s
I20260925 13:56:17.199561 19271 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_c232f8cfe77b43e6ad45e9979d8e3364" } new_table_name: "default.Table_83f9b6c4585f4edbb857f16696339ff3" modify_external_catalogs: true
I20260925 13:56:17.216315 19939 tablet_bootstrap.cc:492] T 1832cff5eb3e4da4b9766ae1591df81a P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:17.209996 19938 raft_consensus.cc:740] T f514fa52e7524cd3ba07f3f65de532d4 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:17.218617 19938 consensus_queue.cc:260] T f514fa52e7524cd3ba07f3f65de532d4 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:17.203819 19273 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cd0b384bb04c44759472fcffcab11ee3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:17.220291 19959 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 86) took 1051 ms. Trace:
I20260925 13:56:17.220590 19959 rpcz_store.cc:276] 0925 13:56:16.168909 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:16.232066 (+ 63157us) service_pool.cc:224] Handling call
0925 13:56:17.094302 (+862236us) ts_tablet_manager.cc:1151] Deleting tablet 5943795bc8b24bb48b80d6d06d17750b
0925 13:56:17.094344 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:17.165945 (+ 71601us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:17.185517 (+ 19572us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:17.220237 (+ 34720us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":861700}
I20260925 13:56:17.221746 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 5943795bc8b24bb48b80d6d06d17750b (table default.Table_36b4dd2097aa4e2dbd58d98277e90281 [id=90bd44e8847549f2964a32f926ef1d99]) successfully deleted
I20260925 13:56:17.229156 19938 ts_tablet_manager.cc:1434] T f514fa52e7524cd3ba07f3f65de532d4 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.022s user 0.005s sys 0.000s
I20260925 13:56:17.229884 19938 tablet_bootstrap.cc:492] T 3e6c8b9bbf4549b3a8685f886973ed9c P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:17.232933 19959 tablet_replica.cc:333] T 4d3dbd9796564472b44a31ccaf56d898 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:17.235091 20256 raft_consensus.cc:493] T 4d3dbd9796564472b44a31ccaf56d898 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:17.235546 20256 raft_consensus.cc:515] T 4d3dbd9796564472b44a31ccaf56d898 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.237433 20256 leader_election.cc:290] T 4d3dbd9796564472b44a31ccaf56d898 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:17.244170 19938 tablet_bootstrap.cc:654] T 3e6c8b9bbf4549b3a8685f886973ed9c P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:17.248697 19939 tablet_bootstrap.cc:654] T 1832cff5eb3e4da4b9766ae1591df81a P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:17.266009 20177 raft_consensus.cc:493] T 8dc5ea164d6749b0926c63d6bda9dcc5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:17.266564 20177 raft_consensus.cc:515] T 8dc5ea164d6749b0926c63d6bda9dcc5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.268676 20177 leader_election.cc:290] T 8dc5ea164d6749b0926c63d6bda9dcc5 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:17.270533 19939 tablet_bootstrap.cc:492] T 1832cff5eb3e4da4b9766ae1591df81a P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:17.270973 19939 ts_tablet_manager.cc:1403] T 1832cff5eb3e4da4b9766ae1591df81a P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.055s user 0.008s sys 0.003s
I20260925 13:56:17.270802 19696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dc5ea164d6749b0926c63d6bda9dcc5" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:17.273964 19939 raft_consensus.cc:359] T 1832cff5eb3e4da4b9766ae1591df81a P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.279973 19939 raft_consensus.cc:385] T 1832cff5eb3e4da4b9766ae1591df81a P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:17.280354 19939 raft_consensus.cc:740] T 1832cff5eb3e4da4b9766ae1591df81a P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:17.281984 19939 consensus_queue.cc:260] T 1832cff5eb3e4da4b9766ae1591df81a P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:17.285945 19773 leader_election.cc:343] T 8dc5ea164d6749b0926c63d6bda9dcc5 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:17.289345 19938 tablet_bootstrap.cc:492] T 3e6c8b9bbf4549b3a8685f886973ed9c P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:17.294441 19938 ts_tablet_manager.cc:1403] T 3e6c8b9bbf4549b3a8685f886973ed9c P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.065s user 0.008s sys 0.008s
W20260925 13:56:17.294821 19278 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5f8a96ae6eec4f15a221f9aad537c094 has no consensus state
I20260925 13:56:17.295786 19939 ts_tablet_manager.cc:1434] T 1832cff5eb3e4da4b9766ae1591df81a P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.025s user 0.008s sys 0.000s
I20260925 13:56:17.296818 19939 tablet_bootstrap.cc:492] T c2d333f892f84a258f8a148a90ce3b06 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:17.303869 20177 raft_consensus.cc:493] T 1832cff5eb3e4da4b9766ae1591df81a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:17.304267 20177 raft_consensus.cc:515] T 1832cff5eb3e4da4b9766ae1591df81a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.306169 20177 leader_election.cc:290] T 1832cff5eb3e4da4b9766ae1591df81a P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:17.298949 19938 raft_consensus.cc:359] T 3e6c8b9bbf4549b3a8685f886973ed9c P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.310397 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12a6c6cff024d70b53cc2c9169dd48b" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:17.310738 19938 raft_consensus.cc:385] T 3e6c8b9bbf4549b3a8685f886973ed9c P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:17.311002 19938 raft_consensus.cc:740] T 3e6c8b9bbf4549b3a8685f886973ed9c P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:17.311117 19554 raft_consensus.cc:2468] T b12a6c6cff024d70b53cc2c9169dd48b P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 0.
I20260925 13:56:17.311791 19939 tablet_bootstrap.cc:654] T c2d333f892f84a258f8a148a90ce3b06 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
W20260925 13:56:17.313359 19267 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 5f8a96ae6eec4f15a221f9aad537c094 (table default.Table_3a06919b658e46f893318b4e1ec8b52f [id=e6f2a872536b4e55b071231020903581], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 5f8a96ae6eec4f15a221f9aad537c094 has no consensus state
I20260925 13:56:17.314548 19938 consensus_queue.cc:260] T 3e6c8b9bbf4549b3a8685f886973ed9c P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.318444 19938 ts_tablet_manager.cc:1434] T 3e6c8b9bbf4549b3a8685f886973ed9c P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.024s user 0.005s sys 0.000s
I20260925 13:56:17.319203 19938 tablet_bootstrap.cc:492] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:17.323334 19694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1832cff5eb3e4da4b9766ae1591df81a" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:17.324420 19279 master_service.cc:960] ReplaceTablet: received request to replace tablet 4d3dbd9796564472b44a31ccaf56d898 from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:17.333946 20202 raft_consensus.cc:493] T c65a8cd3060c405c871270543b4623f7 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:17.334311 20202 raft_consensus.cc:515] T c65a8cd3060c405c871270543b4623f7 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.337664 20202 leader_election.cc:290] T c65a8cd3060c405c871270543b4623f7 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:17.337855 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65a8cd3060c405c871270543b4623f7" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:17.339099 19488 leader_election.cc:343] T c65a8cd3060c405c871270543b4623f7 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:17.339943 19773 leader_election.cc:343] T 1832cff5eb3e4da4b9766ae1591df81a P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:17.341086 19939 tablet_bootstrap.cc:492] T c2d333f892f84a258f8a148a90ce3b06 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:17.341480 19939 ts_tablet_manager.cc:1403] T c2d333f892f84a258f8a148a90ce3b06 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.045s user 0.004s sys 0.005s
I20260925 13:56:17.342855 20246 raft_consensus.cc:493] T bbfc4f99a5964e2ba6146e390e8dba44 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:17.343268 20246 raft_consensus.cc:515] T bbfc4f99a5964e2ba6146e390e8dba44 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.343634 19939 raft_consensus.cc:359] T c2d333f892f84a258f8a148a90ce3b06 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.344331 19939 raft_consensus.cc:385] T c2d333f892f84a258f8a148a90ce3b06 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:17.344599 19939 raft_consensus.cc:740] T c2d333f892f84a258f8a148a90ce3b06 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:17.344980 20246 leader_election.cc:290] T bbfc4f99a5964e2ba6146e390e8dba44 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:17.345300 19938 tablet_bootstrap.cc:654] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:17.345148 19939 consensus_queue.cc:260] T c2d333f892f84a258f8a148a90ce3b06 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.348297 19838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d3dbd9796564472b44a31ccaf56d898" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:17.348663 19939 ts_tablet_manager.cc:1434] T c2d333f892f84a258f8a148a90ce3b06 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.007s user 0.004s sys 0.001s
I20260925 13:56:17.349181 19838 raft_consensus.cc:2393] T 4d3dbd9796564472b44a31ccaf56d898 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7c1b52895a14482fb54a6758f5054f8f in current term 1: Already voted for candidate 8f578869bc084a5d8a9a69f143954eb3 in this term.
I20260925 13:56:17.349409 19939 tablet_bootstrap.cc:492] T 5943795bc8b24bb48b80d6d06d17750b P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:17.351742 19959 raft_consensus.cc:2243] T 4d3dbd9796564472b44a31ccaf56d898 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:17.352293 19959 raft_consensus.cc:2272] T 4d3dbd9796564472b44a31ccaf56d898 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:17.354189 19959 ts_tablet_manager.cc:1916] T 4d3dbd9796564472b44a31ccaf56d898 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:17.361006 20246 raft_consensus.cc:493] T 1da6eadb1648479889027fa911f4676b P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:17.361382 20246 raft_consensus.cc:515] T 1da6eadb1648479889027fa911f4676b P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.362928 20246 leader_election.cc:290] T 1da6eadb1648479889027fa911f4676b P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:17.360281 20267 tablet.cc:1751] T 18a6d1c4eb7948149e709a380da50d93 P 7c1b52895a14482fb54a6758f5054f8f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 13:56:17.365756 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da6eadb1648479889027fa911f4676b" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:17.367147 19488 leader_election.cc:343] T 1da6eadb1648479889027fa911f4676b P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:17.369272 19694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfc4f99a5964e2ba6146e390e8dba44" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:17.371930 19278 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_475efa7896f84e658eb81238ceb58d8f" } modify_external_catalogs: true
I20260925 13:56:17.372756 19278 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_475efa7896f84e658eb81238ceb58d8f" } modify_external_catalogs: true
W20260925 13:56:17.373023 19488 leader_election.cc:343] T bbfc4f99a5964e2ba6146e390e8dba44 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:17.375025 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a1f34dba13143b097a38bab3782c404" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:17.377244 19277 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_d3801e388b7b4b4d95188dc60386cce9" } modify_external_catalogs: true
I20260925 13:56:17.377867 19277 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_d3801e388b7b4b4d95188dc60386cce9" } modify_external_catalogs: true
I20260925 13:56:17.388469 19955 ts_tablet_manager.cc:1916] T 608bed5b04394bfca1776b5d6c502df1 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:17.400465 19959 ts_tablet_manager.cc:1929] T 4d3dbd9796564472b44a31ccaf56d898 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:17.400640 19942 ts_tablet_manager.cc:1434] T 4d3dbd9796564472b44a31ccaf56d898 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.277s user 0.021s sys 0.013s
I20260925 13:56:17.400810 19959 log.cc:1199] T 4d3dbd9796564472b44a31ccaf56d898 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/4d3dbd9796564472b44a31ccaf56d898
I20260925 13:56:17.401676 19942 tablet_bootstrap.cc:492] T 8239d0cd3b0d40bdb721e807c0f403d3 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:17.402664 20011 raft_consensus.cc:2243] T dc14c37aa393407a87af0d0eb8b88d82 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:17.403730 20011 pending_rounds.cc:70] T dc14c37aa393407a87af0d0eb8b88d82 P 7c1b52895a14482fb54a6758f5054f8f: Trying to abort 1 pending ops.
I20260925 13:56:17.403928 20011 pending_rounds.cc:77] T dc14c37aa393407a87af0d0eb8b88d82 P 7c1b52895a14482fb54a6758f5054f8f: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251383963111424 op_type: NO_OP noop_request { }
I20260925 13:56:17.404232 20011 raft_consensus.cc:2889] T dc14c37aa393407a87af0d0eb8b88d82 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:56:17.404541 20011 raft_consensus.cc:2272] T dc14c37aa393407a87af0d0eb8b88d82 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:17.402235 19959 ts_tablet_manager.cc:1950] T 4d3dbd9796564472b44a31ccaf56d898 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:17.405470 20250 raft_consensus.cc:493] T f1c6e2137700448a8ebf859a9ca9085d P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:17.405893 20250 raft_consensus.cc:515] T f1c6e2137700448a8ebf859a9ca9085d P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.415212 19942 tablet_bootstrap.cc:654] T 8239d0cd3b0d40bdb721e807c0f403d3 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
W20260925 13:56:17.418498 20083 thread.cc:641] kernel-watcher (kernel-watchdog) Time spent creating pthread: real 0.607s user 0.015s sys 0.131s
W20260925 13:56:17.418793 20083 thread.cc:608] kernel-watcher (kernel-watchdog) Time spent starting thread: real 0.607s user 0.015s sys 0.131s
I20260925 13:56:17.428925 19959 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 89) took 1156 ms. Trace:
I20260925 13:56:17.429229 19959 rpcz_store.cc:276] 0925 13:56:16.272717 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:16.273508 (+ 791us) service_pool.cc:224] Handling call
0925 13:56:17.232751 (+959243us) ts_tablet_manager.cc:1151] Deleting tablet 4d3dbd9796564472b44a31ccaf56d898
0925 13:56:17.232801 (+ 50us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:17.368772 (+135971us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:17.400348 (+ 31576us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:17.428867 (+ 28519us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":117814,"tablet-delete.queue_time_us":958669}
I20260925 13:56:17.430611 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 4d3dbd9796564472b44a31ccaf56d898 (table default.Table_1aa8069da6504812a85380a42ca07583 [id=a6d1215ba3be4fca903af9b17566b4a8]) successfully deleted
I20260925 13:56:17.431537 19959 tablet_replica.cc:333] T 1832cff5eb3e4da4b9766ae1591df81a P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:17.432269 19959 raft_consensus.cc:2243] T 1832cff5eb3e4da4b9766ae1591df81a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:17.432761 19959 raft_consensus.cc:2272] T 1832cff5eb3e4da4b9766ae1591df81a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:17.434600 19938 tablet_bootstrap.cc:492] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:17.435117 19938 ts_tablet_manager.cc:1403] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.116s user 0.014s sys 0.000s
I20260925 13:56:17.437451 19938 raft_consensus.cc:359] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.438048 19938 raft_consensus.cc:385] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:17.438282 19938 raft_consensus.cc:740] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:17.440073 20250 leader_election.cc:290] T f1c6e2137700448a8ebf859a9ca9085d P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:17.440291 19838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1c6e2137700448a8ebf859a9ca9085d" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:17.441105 19838 raft_consensus.cc:2393] T f1c6e2137700448a8ebf859a9ca9085d P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7c1b52895a14482fb54a6758f5054f8f in current term 1: Already voted for candidate 8f578869bc084a5d8a9a69f143954eb3 in this term.
W20260925 13:56:17.443461 20265 consensus_queue.cc:1175] T dc14c37aa393407a87af0d0eb8b88d82 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7c1b52895a14482fb54a6758f5054f8f" status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
I20260925 13:56:17.445494 19942 tablet_bootstrap.cc:492] T 8239d0cd3b0d40bdb721e807c0f403d3 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:17.445894 19942 ts_tablet_manager.cc:1403] T 8239d0cd3b0d40bdb721e807c0f403d3 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.044s user 0.010s sys 0.000s
I20260925 13:56:17.447821 19938 consensus_queue.cc:260] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.448096 19942 raft_consensus.cc:359] T 8239d0cd3b0d40bdb721e807c0f403d3 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.448675 19942 raft_consensus.cc:385] T 8239d0cd3b0d40bdb721e807c0f403d3 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:17.448920 19942 raft_consensus.cc:740] T 8239d0cd3b0d40bdb721e807c0f403d3 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:17.449730 19942 consensus_queue.cc:260] T 8239d0cd3b0d40bdb721e807c0f403d3 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:17.451987 19490 leader_election.cc:343] T a88bbe1ebee7468d8f49dfd25637fe83 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: a88bbe1ebee7468d8f49dfd25637fe83
I20260925 13:56:17.452492 19942 ts_tablet_manager.cc:1434] T 8239d0cd3b0d40bdb721e807c0f403d3 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260925 13:56:17.452358 19490 leader_election.cc:304] T a88bbe1ebee7468d8f49dfd25637fe83 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3; no voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:17.453202 19942 tablet_bootstrap.cc:492] T f5a0bd0e205f4889be93739dfbd2a816 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:17.454367 19938 ts_tablet_manager.cc:1434] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.019s user 0.005s sys 0.000s
I20260925 13:56:17.455866 19938 tablet_bootstrap.cc:492] T 137c43fb74ec4cc48d30ed3edac81d98 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:17.456904 19939 tablet_bootstrap.cc:654] T 5943795bc8b24bb48b80d6d06d17750b P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:17.458314 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1832cff5eb3e4da4b9766ae1591df81a" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:17.458873 19555 raft_consensus.cc:2468] T 1832cff5eb3e4da4b9766ae1591df81a P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 0.
I20260925 13:56:17.391702 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dc5ea164d6749b0926c63d6bda9dcc5" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:17.469898 19691 raft_consensus.cc:3060] T f1c6e2137700448a8ebf859a9ca9085d P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:17.470090 19938 tablet_bootstrap.cc:654] T 137c43fb74ec4cc48d30ed3edac81d98 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:17.471031 20266 tablet.cc:1751] T 18a6d1c4eb7948149e709a380da50d93 P 8f578869bc084a5d8a9a69f143954eb3: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260925 13:56:17.472426 20083 thread.cc:641] AWS (cloud detector) Time spent creating pthread: real 0.661s user 0.026s sys 0.133s
W20260925 13:56:17.472687 20083 thread.cc:608] AWS (cloud detector) Time spent starting thread: real 0.661s user 0.026s sys 0.133s
W20260925 13:56:17.493427 20274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:56:17.498819 19279 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_899d461298254c7aa8b6c4abbb123d6f" } new_table_name: "default.Table_535f425cc9574fd5b746db1d7707e35d" modify_external_catalogs: true
I20260925 13:56:17.499341 20199 raft_consensus.cc:2749] T a88bbe1ebee7468d8f49dfd25637fe83 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:17.499462 20201 raft_consensus.cc:493] T 1832cff5eb3e4da4b9766ae1591df81a P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:17.501199 19773 leader_election.cc:304] T b12a6c6cff024d70b53cc2c9169dd48b P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
W20260925 13:56:17.502218 19773 leader_election.cc:343] T 2a1f34dba13143b097a38bab3782c404 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:17.502599 19773 leader_election.cc:304] T 2a1f34dba13143b097a38bab3782c404 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:17.503412 20177 raft_consensus.cc:2804] T b12a6c6cff024d70b53cc2c9169dd48b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:17.503742 20177 raft_consensus.cc:493] T b12a6c6cff024d70b53cc2c9169dd48b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:17.504061 20177 raft_consensus.cc:3060] T b12a6c6cff024d70b53cc2c9169dd48b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:17.504092 19836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65a8cd3060c405c871270543b4623f7" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:17.504673 19836 raft_consensus.cc:2468] T c65a8cd3060c405c871270543b4623f7 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:17.505196 19275 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_67306a96449b4a7bbb4b96498a396cea"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:17.506547 19836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfc4f99a5964e2ba6146e390e8dba44" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
W20260925 13:56:17.506664 19275 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_67306a96449b4a7bbb4b96498a396cea in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:17.507020 19836 raft_consensus.cc:2468] T bbfc4f99a5964e2ba6146e390e8dba44 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:17.507864 19840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da6eadb1648479889027fa911f4676b" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:17.500393 20201 raft_consensus.cc:515] T 1832cff5eb3e4da4b9766ae1591df81a P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.508411 19840 raft_consensus.cc:2468] T 1da6eadb1648479889027fa911f4676b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:17.510318 20201 leader_election.cc:290] T 1832cff5eb3e4da4b9766ae1591df81a P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:17.511232 20177 raft_consensus.cc:515] T b12a6c6cff024d70b53cc2c9169dd48b P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.518848 20177 leader_election.cc:290] T b12a6c6cff024d70b53cc2c9169dd48b P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:17.519140 19553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d3dbd9796564472b44a31ccaf56d898" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:17.519332 20177 raft_consensus.cc:2749] T 2a1f34dba13143b097a38bab3782c404 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:17.521588 19553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1c6e2137700448a8ebf859a9ca9085d" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:17.532140 20199 consensus_queue.cc:1048] T bae8c5b6e1014731901bc3f336104784 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:17.533190 20201 raft_consensus.cc:493] T f5a0bd0e205f4889be93739dfbd2a816 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:17.533604 20201 raft_consensus.cc:515] T f5a0bd0e205f4889be93739dfbd2a816 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:17.535490 20201 leader_election.cc:290] T f5a0bd0e205f4889be93739dfbd2a816 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:17.539191 20199 consensus_queue.cc:1048] T dc14c37aa393407a87af0d0eb8b88d82 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 13:56:17.540908 19490 consensus_peers.cc:597] T 7292d04b411c415394e9d0df85aaf579 P 8f578869bc084a5d8a9a69f143954eb3 -> Peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Couldn't send request to peer f790493ba7ad403993b678f2df2b55be. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7292d04b411c415394e9d0df85aaf579. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:17.542630 19490 leader_election.cc:304] T c65a8cd3060c405c871270543b4623f7 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:17.551990 19490 leader_election.cc:304] T bbfc4f99a5964e2ba6146e390e8dba44 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:17.558110 20199 raft_consensus.cc:2804] T bbfc4f99a5964e2ba6146e390e8dba44 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:17.558465 20199 raft_consensus.cc:493] T bbfc4f99a5964e2ba6146e390e8dba44 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:17.558836 20199 raft_consensus.cc:3060] T bbfc4f99a5964e2ba6146e390e8dba44 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:17.565246 20199 raft_consensus.cc:515] T bbfc4f99a5964e2ba6146e390e8dba44 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.552508 20202 raft_consensus.cc:2804] T c65a8cd3060c405c871270543b4623f7 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:17.567448 20202 raft_consensus.cc:493] T c65a8cd3060c405c871270543b4623f7 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:17.567853 20202 raft_consensus.cc:3060] T c65a8cd3060c405c871270543b4623f7 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:17.572320 19490 leader_election.cc:304] T 1da6eadb1648479889027fa911f4676b P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:17.572996 20201 raft_consensus.cc:2804] T 1da6eadb1648479889027fa911f4676b P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:17.573370 20201 raft_consensus.cc:493] T 1da6eadb1648479889027fa911f4676b P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:17.573725 20201 raft_consensus.cc:3060] T 1da6eadb1648479889027fa911f4676b P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:17.588548 20201 raft_consensus.cc:515] T 1da6eadb1648479889027fa911f4676b P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.591809 19942 tablet_bootstrap.cc:654] T f5a0bd0e205f4889be93739dfbd2a816 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:17.595539 20272 tablet.cc:1751] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260925 13:56:17.602505 19279 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f50ea68e3b3c4459b3f5e5b0df724724 has no consensus state
I20260925 13:56:17.606956 20011 ts_tablet_manager.cc:1916] T dc14c37aa393407a87af0d0eb8b88d82 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:17.609048 19939 tablet_bootstrap.cc:492] T 5943795bc8b24bb48b80d6d06d17750b P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:17.609558 19939 ts_tablet_manager.cc:1403] T 5943795bc8b24bb48b80d6d06d17750b P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.260s user 0.003s sys 0.012s
I20260925 13:56:17.618572 20242 consensus_queue.cc:1048] T b36aebfe984f454ea910e84645c4ac7d P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 13:56:17.621111 19694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1832cff5eb3e4da4b9766ae1591df81a" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:17.622680 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a0bd0e205f4889be93739dfbd2a816" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:17.623270 19696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfc4f99a5964e2ba6146e390e8dba44" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:17.623873 19688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da6eadb1648479889027fa911f4676b" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:17.626221 19629 leader_election.cc:304] T 4d3dbd9796564472b44a31ccaf56d898 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:17.629595 19629 leader_election.cc:304] T f1c6e2137700448a8ebf859a9ca9085d P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:17.630692 20252 raft_consensus.cc:3060] T 4d3dbd9796564472b44a31ccaf56d898 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:17.633961 19275 master_service.cc:960] ReplaceTablet: received request to replace tablet c2d333f892f84a258f8a148a90ce3b06 from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:17.618664 20267 tablet.cc:1751] T 1513c0048bd244028d3dfa8d118bc14e P 7c1b52895a14482fb54a6758f5054f8f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:17.644670 20252 raft_consensus.cc:2749] T 4d3dbd9796564472b44a31ccaf56d898 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:56:17.651026 19773 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.193:38269, user_credentials={real_user=slave}} blocked reactor thread for 148734us
I20260925 13:56:17.612757 19939 raft_consensus.cc:359] T 5943795bc8b24bb48b80d6d06d17750b P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.652575 19939 raft_consensus.cc:385] T 5943795bc8b24bb48b80d6d06d17750b P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:17.652384 19773 leader_election.cc:304] T 1832cff5eb3e4da4b9766ae1591df81a P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:17.652889 19939 raft_consensus.cc:740] T 5943795bc8b24bb48b80d6d06d17750b P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
W20260925 13:56:17.653353 19773 raft_consensus.cc:2686] T 1832cff5eb3e4da4b9766ae1591df81a P f790493ba7ad403993b678f2df2b55be: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260925 13:56:17.654584 19773 leader_election.cc:343] T 8dc5ea164d6749b0926c63d6bda9dcc5 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:17.655045 19773 leader_election.cc:304] T 8dc5ea164d6749b0926c63d6bda9dcc5 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:17.657855 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12a6c6cff024d70b53cc2c9169dd48b" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
W20260925 13:56:17.659912 19773 leader_election.cc:343] T b12a6c6cff024d70b53cc2c9169dd48b P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:17.662211 19831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1da6eadb1648479889027fa911f4676b" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:17.663008 19831 raft_consensus.cc:3060] T 1da6eadb1648479889027fa911f4676b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:17.614614 20199 leader_election.cc:290] T bbfc4f99a5964e2ba6146e390e8dba44 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:17.664983 20250 raft_consensus.cc:2749] T f1c6e2137700448a8ebf859a9ca9085d P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:17.668027 20202 raft_consensus.cc:515] T c65a8cd3060c405c871270543b4623f7 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.671103 19831 raft_consensus.cc:2468] T 1da6eadb1648479889027fa911f4676b P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:17.639787 20201 leader_election.cc:290] T 1da6eadb1648479889027fa911f4676b P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:17.672700 19942 tablet_bootstrap.cc:492] T f5a0bd0e205f4889be93739dfbd2a816 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:17.672857 19832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfc4f99a5964e2ba6146e390e8dba44" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:17.673130 19942 ts_tablet_manager.cc:1403] T f5a0bd0e205f4889be93739dfbd2a816 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.220s user 0.012s sys 0.005s
I20260925 13:56:17.673485 19832 raft_consensus.cc:3060] T bbfc4f99a5964e2ba6146e390e8dba44 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:17.675531 19942 raft_consensus.cc:359] T f5a0bd0e205f4889be93739dfbd2a816 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
W20260925 13:56:17.676369 19488 consensus_peers.cc:597] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3 -> Peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Couldn't send request to peer 7c1b52895a14482fb54a6758f5054f8f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260925 13:56:17.676925 19488 leader_election.cc:343] T 1832cff5eb3e4da4b9766ae1591df81a P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:17.677490 19488 leader_election.cc:343] T f5a0bd0e205f4889be93739dfbd2a816 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:17.677866 19942 raft_consensus.cc:385] T f5a0bd0e205f4889be93739dfbd2a816 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260925 13:56:17.678052 19488 leader_election.cc:343] T bbfc4f99a5964e2ba6146e390e8dba44 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:17.678399 19942 raft_consensus.cc:740] T f5a0bd0e205f4889be93739dfbd2a816 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
W20260925 13:56:17.678592 19488 leader_election.cc:343] T 1da6eadb1648479889027fa911f4676b P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:17.681295 19832 raft_consensus.cc:2468] T bbfc4f99a5964e2ba6146e390e8dba44 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:17.684321 19942 consensus_queue.cc:260] T f5a0bd0e205f4889be93739dfbd2a816 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:17.691567 19939 consensus_queue.cc:260] T 5943795bc8b24bb48b80d6d06d17750b P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.692794 19942 ts_tablet_manager.cc:1434] T f5a0bd0e205f4889be93739dfbd2a816 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.019s user 0.005s sys 0.000s
I20260925 13:56:17.694386 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12a6c6cff024d70b53cc2c9169dd48b" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:17.694923 19554 raft_consensus.cc:3060] T b12a6c6cff024d70b53cc2c9169dd48b P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:17.683610 20011 ts_tablet_manager.cc:1929] T dc14c37aa393407a87af0d0eb8b88d82 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:17.695866 20011 log.cc:1199] T dc14c37aa393407a87af0d0eb8b88d82 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/dc14c37aa393407a87af0d0eb8b88d82
I20260925 13:56:17.702397 19955 ts_tablet_manager.cc:1929] T 608bed5b04394bfca1776b5d6c502df1 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:17.702744 19955 log.cc:1199] T 608bed5b04394bfca1776b5d6c502df1 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/608bed5b04394bfca1776b5d6c502df1
I20260925 13:56:17.703408 20011 ts_tablet_manager.cc:1950] T dc14c37aa393407a87af0d0eb8b88d82 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:17.707230 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 84) took 1065 ms. Trace:
I20260925 13:56:17.707290 19277 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 1da6eadb1648479889027fa911f4676b
W20260925 13:56:17.679209 19488 consensus_peers.cc:597] T dc14c37aa393407a87af0d0eb8b88d82 P 8f578869bc084a5d8a9a69f143954eb3 -> Peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Couldn't send request to peer 7c1b52895a14482fb54a6758f5054f8f. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:17.708597 19490 leader_election.cc:304] T 1da6eadb1648479889027fa911f4676b P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:17.710505 19533 tablet_service.cc:1558] Processing DeleteTablet for tablet 1da6eadb1648479889027fa911f4676b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:17 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:17.711002 20011 rpcz_store.cc:276] 0925 13:56:16.641770 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:16.655438 (+ 13668us) service_pool.cc:224] Handling call
0925 13:56:17.147838 (+492400us) ts_tablet_manager.cc:1151] Deleting tablet dc14c37aa393407a87af0d0eb8b88d82
0925 13:56:17.147868 (+ 30us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:17.403342 (+255474us) spinlock_profiling.cc:243] Waited 232 ms on lock 0x7b5400061154. stack: 00007f1ff9889323 00007f1ff9889220 00007f1ff942d574 0000000000516964 00000000005168f9 00007f2000986aa1 00007f2000a25b22 00007f2000e6650c 00007f2001acfc3a 00007f2001b00960 00007f2001ae6e6d 00007f2001ae6e19 00007f2001ae6da9 00007f2001ae6d71 00007f2001ae5f9d 00000000005bd4e4
0925 13:56:17.661768 (+258426us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:17.683408 (+ 21640us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:17.707175 (+ 23767us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":534671744,"tablet-delete.queue_time_us":491880}
I20260925 13:56:17.712365 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet dc14c37aa393407a87af0d0eb8b88d82 (table default.Table_3a06919b658e46f893318b4e1ec8b52f [id=e6f2a872536b4e55b071231020903581]) successfully deleted
I20260925 13:56:17.703672 19942 tablet_bootstrap.cc:492] T 26644768f516412f8c95d68a76bb99c5 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:17.709257 19676 tablet_service.cc:1558] Processing DeleteTablet for tablet 1da6eadb1648479889027fa911f4676b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:17 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:17.713346 20201 consensus_queue.cc:1048] T f1c6e2137700448a8ebf859a9ca9085d P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:17.715700 20202 leader_election.cc:290] T c65a8cd3060c405c871270543b4623f7 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:17.717259 19691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65a8cd3060c405c871270543b4623f7" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:17.718637 20011 tablet_replica.cc:333] T 47521fb74f9c40ee999a7f704aa551c9 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:17.719609 19554 raft_consensus.cc:2468] T b12a6c6cff024d70b53cc2c9169dd48b P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 1.
I20260925 13:56:17.720518 20011 raft_consensus.cc:2243] T 47521fb74f9c40ee999a7f704aa551c9 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:17.720846 19939 ts_tablet_manager.cc:1434] T 5943795bc8b24bb48b80d6d06d17750b P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.111s user 0.006s sys 0.000s
I20260925 13:56:17.724625 19490 leader_election.cc:304] T bbfc4f99a5964e2ba6146e390e8dba44 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:17.725422 19272 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 18a6d1c4eb7948149e709a380da50d93 (table default.Table_e814702f1f874529a98bd1631d3b5c84 [id=6c0ec9465714408da93ff6e47827eb48]). Expected version 2 got 0
I20260925 13:56:17.726292 19272 catalog_manager.cc:5838] T b36aebfe984f454ea910e84645c4ac7d P 7c1b52895a14482fb54a6758f5054f8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194). New cstate: current_term: 1 leader_uuid: "7c1b52895a14482fb54a6758f5054f8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:17.728372 20011 raft_consensus.cc:2272] T 47521fb74f9c40ee999a7f704aa551c9 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:17.736178 19955 ts_tablet_manager.cc:1950] T 608bed5b04394bfca1776b5d6c502df1 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:17.737046 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 76) took 2501 ms. Trace:
I20260925 13:56:17.736532 19271 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_b1985cfbe41c4240a1c15abfd5741f90"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:17.737320 19955 rpcz_store.cc:276] 0925 13:56:15.235311 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:15.236121 (+ 810us) service_pool.cc:224] Handling call
0925 13:56:17.028860 (+1792739us) ts_tablet_manager.cc:1151] Deleting tablet 608bed5b04394bfca1776b5d6c502df1
0925 13:56:17.028908 (+ 48us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:17.581074 (+552166us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:17.702281 (+121207us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:17.736991 (+ 34710us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1792186}
I20260925 13:56:17.738762 20201 raft_consensus.cc:2804] T 1da6eadb1648479889027fa911f4676b P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election won for term 1
W20260925 13:56:17.738718 19271 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b1985cfbe41c4240a1c15abfd5741f90 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:17.739259 20201 raft_consensus.cc:697] T 1da6eadb1648479889027fa911f4676b P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Running, Role: LEADER
I20260925 13:56:17.740314 20201 consensus_queue.cc:237] T 1da6eadb1648479889027fa911f4676b P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.742494 20199 raft_consensus.cc:2804] T bbfc4f99a5964e2ba6146e390e8dba44 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:17.743065 20199 raft_consensus.cc:697] T bbfc4f99a5964e2ba6146e390e8dba44 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Running, Role: LEADER
I20260925 13:56:17.743726 19942 tablet_bootstrap.cc:654] T 26644768f516412f8c95d68a76bb99c5 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:17.743909 20199 consensus_queue.cc:237] T bbfc4f99a5964e2ba6146e390e8dba44 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.745270 19274 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_9cdc814c021f49dea82875834691ba9e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:17.752120 20011 ts_tablet_manager.cc:1916] T 47521fb74f9c40ee999a7f704aa551c9 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:17.752952 20198 raft_consensus.cc:493] T c2d333f892f84a258f8a148a90ce3b06 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260925 13:56:17.747144 19274 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9cdc814c021f49dea82875834691ba9e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:17.753351 20198 raft_consensus.cc:515] T c2d333f892f84a258f8a148a90ce3b06 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.755060 20198 leader_election.cc:290] T c2d333f892f84a258f8a148a90ce3b06 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:17.756668 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d333f892f84a258f8a148a90ce3b06" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:17.758147 19488 leader_election.cc:343] T c65a8cd3060c405c871270543b4623f7 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:17.758771 19488 leader_election.cc:343] T c2d333f892f84a258f8a148a90ce3b06 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:17.721666 19939 tablet_bootstrap.cc:492] T 2a1f34dba13143b097a38bab3782c404 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:17.781127 19942 tablet_bootstrap.cc:492] T 26644768f516412f8c95d68a76bb99c5 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:17.781559 19942 ts_tablet_manager.cc:1403] T 26644768f516412f8c95d68a76bb99c5 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.088s user 0.009s sys 0.000s
I20260925 13:56:17.790210 19676 tablet_service.cc:1558] Processing DeleteTablet for tablet 00b500182bb04a72b80cfa9103e85fa9 with delete_type TABLET_DATA_DELETED (replaced by tablet f5a0bd0e205f4889be93739dfbd2a816) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:17.791301 19955 tablet_replica.cc:333] T a88bbe1ebee7468d8f49dfd25637fe83 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:17.792073 19955 raft_consensus.cc:2243] T a88bbe1ebee7468d8f49dfd25637fe83 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:17.792083 19676 tablet_service.cc:1558] Processing DeleteTablet for tablet 608bed5b04394bfca1776b5d6c502df1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:14 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:17.792495 19955 raft_consensus.cc:2272] T a88bbe1ebee7468d8f49dfd25637fe83 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:17.791342 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet 608bed5b04394bfca1776b5d6c502df1 (table default.Table_3b49fb893abb48ac8646283360e8e681 [id=9621507077d04649a58715b250fcef05]) successfully deleted
I20260925 13:56:17.791834 19942 raft_consensus.cc:359] T 26644768f516412f8c95d68a76bb99c5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.799387 19942 raft_consensus.cc:385] T 26644768f516412f8c95d68a76bb99c5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:17.803450 19942 raft_consensus.cc:740] T 26644768f516412f8c95d68a76bb99c5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:17.804147 19942 consensus_queue.cc:260] T 26644768f516412f8c95d68a76bb99c5 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.806771 19273 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_535f425cc9574fd5b746db1d7707e35d" } new_table_name: "default.Table_6d0ced912e4e4c188553d0839f794603" modify_external_catalogs: true
I20260925 13:56:17.809685 19942 ts_tablet_manager.cc:1434] T 26644768f516412f8c95d68a76bb99c5 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.028s user 0.007s sys 0.000s
I20260925 13:56:17.803854 19277 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_261023ab15a845df82deea31f0eac8c8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:17.811590 19277 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_261023ab15a845df82deea31f0eac8c8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:17.816780 19942 tablet_bootstrap.cc:492] T c65a8cd3060c405c871270543b4623f7 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:17.434640 19959 ts_tablet_manager.cc:1916] T 1832cff5eb3e4da4b9766ae1591df81a P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:17.833387 19836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1832cff5eb3e4da4b9766ae1591df81a" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:17.834093 20011 ts_tablet_manager.cc:1929] T 47521fb74f9c40ee999a7f704aa551c9 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:17.834394 20011 log.cc:1199] T 47521fb74f9c40ee999a7f704aa551c9 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/47521fb74f9c40ee999a7f704aa551c9
I20260925 13:56:17.835508 19955 ts_tablet_manager.cc:1916] T a88bbe1ebee7468d8f49dfd25637fe83 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:17.839775 19835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a0bd0e205f4889be93739dfbd2a816" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:17.840508 19835 raft_consensus.cc:2468] T f5a0bd0e205f4889be93739dfbd2a816 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:17.841374 20177 raft_consensus.cc:2749] T 8dc5ea164d6749b0926c63d6bda9dcc5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:17.842144 20011 ts_tablet_manager.cc:1950] T 47521fb74f9c40ee999a7f704aa551c9 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:17.843928 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 85) took 1027 ms. Trace:
I20260925 13:56:17.845345 19942 tablet_bootstrap.cc:654] T c65a8cd3060c405c871270543b4623f7 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:17.847304 20242 raft_consensus.cc:493] T 8239d0cd3b0d40bdb721e807c0f403d3 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:17.847756 20242 raft_consensus.cc:515] T 8239d0cd3b0d40bdb721e807c0f403d3 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.849591 20242 leader_election.cc:290] T 8239d0cd3b0d40bdb721e807c0f403d3 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:17.851303 19773 leader_election.cc:304] T b12a6c6cff024d70b53cc2c9169dd48b P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:17.851253 20242 consensus_queue.cc:1048] T b36aebfe984f454ea910e84645c4ac7d P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:17.852586 19820 tablet_service.cc:1558] Processing DeleteTablet for tablet 1da6eadb1648479889027fa911f4676b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:17 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:17.853389 19831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65a8cd3060c405c871270543b4623f7" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:17.853860 19831 raft_consensus.cc:3060] T c65a8cd3060c405c871270543b4623f7 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:17.855249 19276 master_service.cc:960] ReplaceTablet: received request to replace tablet 7ea4fd6817654bff9c0fe8c6e42ed6b9 from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:17.844219 20011 rpcz_store.cc:276] 0925 13:56:16.816460 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:16.817573 (+ 1113us) service_pool.cc:224] Handling call
0925 13:56:17.718471 (+900898us) ts_tablet_manager.cc:1151] Deleting tablet 47521fb74f9c40ee999a7f704aa551c9
0925 13:56:17.718529 (+ 58us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:17.793250 (+ 74721us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:17.833986 (+ 40736us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:17.843878 (+ 9892us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":900276}
I20260925 13:56:17.858114 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 47521fb74f9c40ee999a7f704aa551c9 (table default.Table_3a3e48a0641a43a1a783c6ba6374a708 [id=e64fe44a237f48859a288a87a565ee5e]) successfully deleted
W20260925 13:56:17.858903 19264 catalog_manager.cc:5030] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 7292d04b411c415394e9d0df85aaf579 because the tablet was not found. No further retry: Not found: Tablet not found: 7292d04b411c415394e9d0df85aaf579
I20260925 13:56:17.859692 20011 tablet_replica.cc:333] T 980d1264c0ea45f3b2b762509e0b2c77 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:17.860338 20011 raft_consensus.cc:2243] T 980d1264c0ea45f3b2b762509e0b2c77 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:17.860703 20011 raft_consensus.cc:2272] T 980d1264c0ea45f3b2b762509e0b2c77 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:17.862284 20011 ts_tablet_manager.cc:1916] T 980d1264c0ea45f3b2b762509e0b2c77 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 13:56:17.870186 19490 leader_election.cc:343] T 1832cff5eb3e4da4b9766ae1591df81a P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Illegal state: Tablet not RUNNING: STOPPED
I20260925 13:56:17.870683 20267 tablet.cc:1751] T 6b52ed579b19479686247d3ecba9489a P 7c1b52895a14482fb54a6758f5054f8f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:17.872948 19279 catalog_manager.cc:5838] T f1c6e2137700448a8ebf859a9ca9085d P 8f578869bc084a5d8a9a69f143954eb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "8f578869bc084a5d8a9a69f143954eb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:17.874501 19279 catalog_manager.cc:5838] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "8f578869bc084a5d8a9a69f143954eb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:17.875723 19279 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet 26644768f516412f8c95d68a76bb99c5 (table default.Table_bd52a86b8ff14bdbb06fcf88a60b1e82 [id=203024d67e444f839391c8670d7c7d3b]). Expected version 2 got 0
I20260925 13:56:17.876607 19939 tablet_bootstrap.cc:654] T 2a1f34dba13143b097a38bab3782c404 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:17.870572 19490 leader_election.cc:304] T 1832cff5eb3e4da4b9766ae1591df81a P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3; no voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:17.876866 19279 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet 18a6d1c4eb7948149e709a380da50d93 (table default.Table_e814702f1f874529a98bd1631d3b5c84 [id=6c0ec9465714408da93ff6e47827eb48]). Expected version 2 got 0
I20260925 13:56:17.877084 20201 raft_consensus.cc:2749] T 1832cff5eb3e4da4b9766ae1591df81a P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:17.878197 19490 leader_election.cc:304] T f5a0bd0e205f4889be93739dfbd2a816 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:17.879917 20201 raft_consensus.cc:2804] T f5a0bd0e205f4889be93739dfbd2a816 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:17.880226 20201 raft_consensus.cc:493] T f5a0bd0e205f4889be93739dfbd2a816 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:17.880522 20201 raft_consensus.cc:3060] T f5a0bd0e205f4889be93739dfbd2a816 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:17.880388 20281 raft_consensus.cc:493] T 3e6c8b9bbf4549b3a8685f886973ed9c P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:17.880807 20177 raft_consensus.cc:493] T f514fa52e7524cd3ba07f3f65de532d4 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:17.884487 20177 raft_consensus.cc:515] T f514fa52e7524cd3ba07f3f65de532d4 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.886565 20201 raft_consensus.cc:515] T f5a0bd0e205f4889be93739dfbd2a816 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:17.887768 19840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d333f892f84a258f8a148a90ce3b06" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:17.888418 19840 raft_consensus.cc:2468] T c2d333f892f84a258f8a148a90ce3b06 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:17.883460 20281 raft_consensus.cc:515] T 3e6c8b9bbf4549b3a8685f886973ed9c P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:17.890355 19631 leader_election.cc:343] T 8239d0cd3b0d40bdb721e807c0f403d3 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 8239d0cd3b0d40bdb721e807c0f403d3
I20260925 13:56:17.887089 20177 leader_election.cc:290] T f514fa52e7524cd3ba07f3f65de532d4 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:17.891881 20177 raft_consensus.cc:2804] T b12a6c6cff024d70b53cc2c9169dd48b P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:17.892210 19278 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet c161ae3edfed4f8488c770ecc6332607
I20260925 13:56:17.892293 20177 raft_consensus.cc:697] T b12a6c6cff024d70b53cc2c9169dd48b P f790493ba7ad403993b678f2df2b55be [term 1 LEADER]: Becoming Leader. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Running, Role: LEADER
I20260925 13:56:17.894963 19820 tablet_service.cc:1558] Processing DeleteTablet for tablet c161ae3edfed4f8488c770ecc6332607 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:17 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:17.896919 19533 tablet_service.cc:1558] Processing DeleteTablet for tablet c161ae3edfed4f8488c770ecc6332607 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:17 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:17.891665 19691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f514fa52e7524cd3ba07f3f65de532d4" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:17.899430 19533 tablet_service.cc:1511] Processing CreateTablet for tablet 28219970809145bc9d4e2a3289a9479a (DEFAULT_TABLE table=default.Table_7746d3f112bf43ffb276928654340d9d [id=b0113116ae7f49afb23499437e1c1246]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:17.900455 19533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28219970809145bc9d4e2a3289a9479a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:17.900635 19676 tablet_service.cc:1558] Processing DeleteTablet for tablet c161ae3edfed4f8488c770ecc6332607 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:17 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:17.901398 19274 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_3751c59d6c7347db843dd4f021a6c620"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:17.904091 19672 tablet_service.cc:1511] Processing CreateTablet for tablet 28219970809145bc9d4e2a3289a9479a (DEFAULT_TABLE table=default.Table_7746d3f112bf43ffb276928654340d9d [id=b0113116ae7f49afb23499437e1c1246]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:17.905113 19672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28219970809145bc9d4e2a3289a9479a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:17.907001 19274 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3751c59d6c7347db843dd4f021a6c620 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 13:56:17.900418 19272 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.194:34125 (request call id 25) took 908 ms (client timeout 1000 ms). Trace:
I20260925 13:56:17.909235 19676 tablet_service.cc:1511] Processing CreateTablet for tablet 12984db09f34472ba365fdf63a33b720 (DEFAULT_TABLE table=default.Table_10868b99c6ec43b2a38a3a4a5bdcf960 [id=46b6ad1a631c4c08a05ef5a1b5f62ab9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:17.910084 19676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12984db09f34472ba365fdf63a33b720. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:17.917048 19942 tablet_bootstrap.cc:492] T c65a8cd3060c405c871270543b4623f7 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:17.917438 19942 ts_tablet_manager.cc:1403] T c65a8cd3060c405c871270543b4623f7 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.101s user 0.006s sys 0.005s
I20260925 13:56:17.920456 19942 raft_consensus.cc:359] T c65a8cd3060c405c871270543b4623f7 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.921046 19942 raft_consensus.cc:385] T c65a8cd3060c405c871270543b4623f7 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:17.921276 19942 raft_consensus.cc:740] T c65a8cd3060c405c871270543b4623f7 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:17.923889 19831 raft_consensus.cc:2468] T c65a8cd3060c405c871270543b4623f7 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
W20260925 13:56:17.925726 19773 leader_election.cc:343] T f514fa52e7524cd3ba07f3f65de532d4 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:17.926016 19534 tablet_service.cc:1511] Processing CreateTablet for tablet 12984db09f34472ba365fdf63a33b720 (DEFAULT_TABLE table=default.Table_10868b99c6ec43b2a38a3a4a5bdcf960 [id=46b6ad1a631c4c08a05ef5a1b5f62ab9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:17.926913 19534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12984db09f34472ba365fdf63a33b720. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:17.927034 19273 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f50ea68e3b3c4459b3f5e5b0df724724 consensus state has no leader
W20260925 13:56:17.908957 19272 rpcz_store.cc:269] 0925 13:56:16.991908 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:16.992051 (+ 143us) service_pool.cc:224] Handling call
0925 13:56:17.196202 (+204151us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b5400040454. stack: 00007fdf0a30b323 00007fdf0a30b220 00007fdf09eaf574 0000000000516964 00000000005168f9 00007fdf114095c0 00007fdf114a86fa 00007fdf13219d8c 00007fdf1331801e 00007fdf10ebcfb4 00007fdf10ebcf42 00007fdf10ebce71 00007fdf10ebcdec 00007fdf10ebc0a2 00007fdf0f16e09c 00007fdf0f16d4d6
0925 13:56:17.900363 (+704161us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:17.748426 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:17.748895 (+ 469us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:17.748925 (+ 30us) write_op.cc:435] Acquired schema lock
0925 13:56:17.748943 (+ 18us) tablet.cc:662] Decoding operations
0925 13:56:17.749692 (+ 749us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:17.749740 (+ 48us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:17.749769 (+ 29us) tablet.cc:685] Acquiring locks for 1 operations
0925 13:56:17.749929 (+ 160us) tablet.cc:701] Row locks acquired
0925 13:56:17.749946 (+ 17us) write_op.cc:260] PREPARE: finished
0925 13:56:17.750061 (+ 115us) write_op.cc:270] Start()
0925 13:56:17.750138 (+ 77us) write_op.cc:276] Timestamp: P: 1790344577750131 usec, L: 0
0925 13:56:17.750159 (+ 21us) op_driver.cc:348] REPLICATION: starting
0925 13:56:17.750708 (+ 549us) log.cc:844] Serialized 536 byte log entry
0925 13:56:17.766635 (+ 15927us) op_driver.cc:464] REPLICATION: finished
0925 13:56:17.767972 (+ 1337us) write_op.cc:301] APPLY: starting
0925 13:56:17.768020 (+ 48us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:17.768118 (+ 98us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:17.768198 (+ 80us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:17.768358 (+ 160us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:17.768483 (+ 125us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 13:56:17.768524 (+ 41us) write_op.cc:312] APPLY: finished
0925 13:56:17.768687 (+ 163us) log.cc:844] Serialized 23 byte log entry
0925 13:56:17.768819 (+ 132us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:17.768871 (+ 52us) write_op.cc:454] Released schema lock
0925 13:56:17.768906 (+ 35us) write_op.cc:341] FINISH: Updating metrics
0925 13:56:17.769112 (+ 206us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:17.769136 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":6,"spinlock_wait_cycles":462148864,"child_traces":[["op",{"apply.queue_time_us":1185,"apply.run_cpu_time_us":1108,"apply.run_wall_time_us":1108,"num_ops":1,"prepare.queue_time_us":120,"prepare.run_cpu_time_us":2208,"prepare.run_wall_time_us":3689,"raft.queue_time_us":362,"raft.run_cpu_time_us":794,"raft.run_wall_time_us":795,"replication_time_us":16364}]]}
I20260925 13:56:17.928028 19942 consensus_queue.cc:260] T c65a8cd3060c405c871270543b4623f7 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.933851 19674 tablet_service.cc:1511] Processing CreateTablet for tablet 0de8f2d2766c4ff197b552a1b5ed3d6b (DEFAULT_TABLE table=default.Table_052e4e4ee53e4e408b18cd0ce8063d31 [id=993a34f0c22c4555834fb5cf42da8579]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:17.934582 19942 ts_tablet_manager.cc:1434] T c65a8cd3060c405c871270543b4623f7 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.017s user 0.005s sys 0.000s
I20260925 13:56:17.934837 19674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0de8f2d2766c4ff197b552a1b5ed3d6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:17.946798 20011 ts_tablet_manager.cc:1929] T 980d1264c0ea45f3b2b762509e0b2c77 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:17.947132 20011 log.cc:1199] T 980d1264c0ea45f3b2b762509e0b2c77 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/980d1264c0ea45f3b2b762509e0b2c77
I20260925 13:56:17.948228 19942 tablet_bootstrap.cc:492] T a88bbe1ebee7468d8f49dfd25637fe83 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:17.953202 20272 tablet.cc:1751] T 6b52ed579b19479686247d3ecba9489a P 8f578869bc084a5d8a9a69f143954eb3: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:17.955567 20281 leader_election.cc:290] T 3e6c8b9bbf4549b3a8685f886973ed9c P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:17.959259 19278 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_50d8afbc179c4db0a81fff129311fa10" } modify_external_catalogs: true
I20260925 13:56:17.959790 19278 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_50d8afbc179c4db0a81fff129311fa10" } modify_external_catalogs: true
I20260925 13:56:17.961599 19939 tablet_bootstrap.cc:492] T 2a1f34dba13143b097a38bab3782c404 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:17.961997 19939 ts_tablet_manager.cc:1403] T 2a1f34dba13143b097a38bab3782c404 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.240s user 0.010s sys 0.004s
I20260925 13:56:17.962244 19820 tablet_service.cc:1511] Processing CreateTablet for tablet 28219970809145bc9d4e2a3289a9479a (DEFAULT_TABLE table=default.Table_7746d3f112bf43ffb276928654340d9d [id=b0113116ae7f49afb23499437e1c1246]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:17.964285 19939 raft_consensus.cc:359] T 2a1f34dba13143b097a38bab3782c404 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.964875 19939 raft_consensus.cc:385] T 2a1f34dba13143b097a38bab3782c404 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:17.965134 19939 raft_consensus.cc:740] T 2a1f34dba13143b097a38bab3782c404 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:17.965814 19939 consensus_queue.cc:260] T 2a1f34dba13143b097a38bab3782c404 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:17.966048 19277 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_9b29863ad2ca4d698aebe4b655b19e50" } new_table_name: "default.Table_e0f7b807e8884431898298a1c635c784" modify_external_catalogs: true
I20260925 13:56:17.967693 19939 ts_tablet_manager.cc:1434] T 2a1f34dba13143b097a38bab3782c404 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260925 13:56:17.973166 20177 consensus_queue.cc:237] T b12a6c6cff024d70b53cc2c9169dd48b P f790493ba7ad403993b678f2df2b55be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:17.974390 19629 leader_election.cc:343] T 8239d0cd3b0d40bdb721e807c0f403d3 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Not found: Tablet not found: 8239d0cd3b0d40bdb721e807c0f403d3
I20260925 13:56:17.974828 19629 leader_election.cc:304] T 8239d0cd3b0d40bdb721e807c0f403d3 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:17.976208 20252 raft_consensus.cc:2749] T 8239d0cd3b0d40bdb721e807c0f403d3 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:17.976104 19817 tablet_service.cc:1511] Processing CreateTablet for tablet 12984db09f34472ba365fdf63a33b720 (DEFAULT_TABLE table=default.Table_10868b99c6ec43b2a38a3a4a5bdcf960 [id=46b6ad1a631c4c08a05ef5a1b5f62ab9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:17.977509 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f514fa52e7524cd3ba07f3f65de532d4" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:17.977643 19818 tablet_service.cc:1511] Processing CreateTablet for tablet 0de8f2d2766c4ff197b552a1b5ed3d6b (DEFAULT_TABLE table=default.Table_052e4e4ee53e4e408b18cd0ce8063d31 [id=993a34f0c22c4555834fb5cf42da8579]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:17.979967 19691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e6c8b9bbf4549b3a8685f886973ed9c" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:17.980562 19535 tablet_service.cc:1511] Processing CreateTablet for tablet 0de8f2d2766c4ff197b552a1b5ed3d6b (DEFAULT_TABLE table=default.Table_052e4e4ee53e4e408b18cd0ce8063d31 [id=993a34f0c22c4555834fb5cf42da8579]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:17.981438 19535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0de8f2d2766c4ff197b552a1b5ed3d6b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:17.981407 19773 leader_election.cc:343] T 3e6c8b9bbf4549b3a8685f886973ed9c P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:17.981503 19676 tablet_service.cc:1511] Processing CreateTablet for tablet 436b8d46e1d244deb8bb9dc128ee6279 (DEFAULT_TABLE table=default.Table_079723ff92144ba293733e54ab25d782 [id=4b0bfb9726854123864f8fe743a16a33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:17.982342 19676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 436b8d46e1d244deb8bb9dc128ee6279. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:17.985033 19819 tablet_service.cc:1511] Processing CreateTablet for tablet 436b8d46e1d244deb8bb9dc128ee6279 (DEFAULT_TABLE table=default.Table_079723ff92144ba293733e54ab25d782 [id=4b0bfb9726854123864f8fe743a16a33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:17.985778 19938 tablet_bootstrap.cc:492] T 137c43fb74ec4cc48d30ed3edac81d98 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:17.985980 19820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28219970809145bc9d4e2a3289a9479a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:17.986325 19938 ts_tablet_manager.cc:1403] T 137c43fb74ec4cc48d30ed3edac81d98 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.531s user 0.037s sys 0.040s
I20260925 13:56:17.986590 19815 tablet_service.cc:1511] Processing CreateTablet for tablet e3cf9676160c4af08f1f62ae952b4c57 (DEFAULT_TABLE table=default.Table_4d6d747ea4034b3ab48710e10ff4544f [id=d6e6dd0772bf4901b696e11b10e41a48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:17.998059 19531 tablet_service.cc:1511] Processing CreateTablet for tablet 436b8d46e1d244deb8bb9dc128ee6279 (DEFAULT_TABLE table=default.Table_079723ff92144ba293733e54ab25d782 [id=4b0bfb9726854123864f8fe743a16a33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:17.999094 19531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 436b8d46e1d244deb8bb9dc128ee6279. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.006078 19819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 436b8d46e1d244deb8bb9dc128ee6279. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.007647 19818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0de8f2d2766c4ff197b552a1b5ed3d6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.009057 19817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12984db09f34472ba365fdf63a33b720. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.013697 19816 tablet_service.cc:1511] Processing CreateTablet for tablet 2ef3efe644364e93a089ddeed328389d (DEFAULT_TABLE table=default.Table_dfe9d9863d364dfc8df4c405e87dd635 [id=7b4bc216530346b6ab44c1ce25fe2112]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.016878 19673 tablet_service.cc:1511] Processing CreateTablet for tablet e3cf9676160c4af08f1f62ae952b4c57 (DEFAULT_TABLE table=default.Table_4d6d747ea4034b3ab48710e10ff4544f [id=d6e6dd0772bf4901b696e11b10e41a48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.017903 19673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3cf9676160c4af08f1f62ae952b4c57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.025126 19530 tablet_service.cc:1511] Processing CreateTablet for tablet e3cf9676160c4af08f1f62ae952b4c57 (DEFAULT_TABLE table=default.Table_4d6d747ea4034b3ab48710e10ff4544f [id=d6e6dd0772bf4901b696e11b10e41a48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.026487 19530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3cf9676160c4af08f1f62ae952b4c57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.033215 19942 tablet_bootstrap.cc:654] T a88bbe1ebee7468d8f49dfd25637fe83 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:18.035260 19815 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3cf9676160c4af08f1f62ae952b4c57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.037163 20202 consensus_queue.cc:1048] T 4d3dbd9796564472b44a31ccaf56d898 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:18.042819 19816 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ef3efe644364e93a089ddeed328389d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.037659 20242 raft_consensus.cc:493] T 26644768f516412f8c95d68a76bb99c5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:18.059329 20242 raft_consensus.cc:515] T 26644768f516412f8c95d68a76bb99c5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:18.059794 19675 tablet_service.cc:1511] Processing CreateTablet for tablet 2ef3efe644364e93a089ddeed328389d (DEFAULT_TABLE table=default.Table_dfe9d9863d364dfc8df4c405e87dd635 [id=7b4bc216530346b6ab44c1ce25fe2112]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.060743 19675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ef3efe644364e93a089ddeed328389d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.061275 20242 leader_election.cc:290] T 26644768f516412f8c95d68a76bb99c5 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:18.061446 20202 consensus_queue.cc:1048] T f1c6e2137700448a8ebf859a9ca9085d P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:18.064688 20011 ts_tablet_manager.cc:1950] T 980d1264c0ea45f3b2b762509e0b2c77 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:18.066829 20242 raft_consensus.cc:493] T c65a8cd3060c405c871270543b4623f7 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:18.067572 19275 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet c2d333f892f84a258f8a148a90ce3b06
I20260925 13:56:18.067255 20242 raft_consensus.cc:515] T c65a8cd3060c405c871270543b4623f7 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:18.068965 19275 catalog_manager.cc:6934] ReplaceTablet: tablet c2d333f892f84a258f8a148a90ce3b06 deleted and replaced by tablet 908df914cfec490cb7058dbe7b501c97
I20260925 13:56:18.071087 20242 leader_election.cc:290] T c65a8cd3060c405c871270543b4623f7 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:18.072697 19490 leader_election.cc:304] T c2d333f892f84a258f8a148a90ce3b06 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:18.072933 19276 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 7ea4fd6817654bff9c0fe8c6e42ed6b9
I20260925 13:56:18.073871 20202 raft_consensus.cc:2804] T c2d333f892f84a258f8a148a90ce3b06 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:18.074237 19276 catalog_manager.cc:6934] ReplaceTablet: tablet 7ea4fd6817654bff9c0fe8c6e42ed6b9 deleted and replaced by tablet 321c740ba8eb4cbb8974305c5f8702af
I20260925 13:56:18.074214 20202 raft_consensus.cc:493] T c2d333f892f84a258f8a148a90ce3b06 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:18.074517 20202 raft_consensus.cc:3060] T c2d333f892f84a258f8a148a90ce3b06 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:18.082086 19671 tablet_service.cc:1511] Processing CreateTablet for tablet 70a86ea84d1d49d1b0950b4ce3a3d770 (DEFAULT_TABLE table=default.Table_9d043ec520e1422995a293b09996a713 [id=6496cd8160f9458eb2ef4dcdd2fb67cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.082355 19535 tablet_service.cc:1511] Processing CreateTablet for tablet 2ef3efe644364e93a089ddeed328389d (DEFAULT_TABLE table=default.Table_dfe9d9863d364dfc8df4c405e87dd635 [id=7b4bc216530346b6ab44c1ce25fe2112]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.083184 19671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70a86ea84d1d49d1b0950b4ce3a3d770. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.083570 19535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ef3efe644364e93a089ddeed328389d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.085817 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 87) took 1100 ms. Trace:
I20260925 13:56:18.089231 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a0bd0e205f4889be93739dfbd2a816" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:18.089805 19695 raft_consensus.cc:3060] T f5a0bd0e205f4889be93739dfbd2a816 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:18.086184 20011 rpcz_store.cc:276] 0925 13:56:16.984947 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:16.985068 (+ 121us) service_pool.cc:224] Handling call
0925 13:56:17.859558 (+874490us) ts_tablet_manager.cc:1151] Deleting tablet 980d1264c0ea45f3b2b762509e0b2c77
0925 13:56:17.859596 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:17.940914 (+ 81318us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:17.946697 (+ 5783us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:18.085761 (+139064us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":815462}
I20260925 13:56:18.091209 19939 tablet_bootstrap.cc:492] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:18.092212 19814 tablet_service.cc:1511] Processing CreateTablet for tablet 70a86ea84d1d49d1b0950b4ce3a3d770 (DEFAULT_TABLE table=default.Table_9d043ec520e1422995a293b09996a713 [id=6496cd8160f9458eb2ef4dcdd2fb67cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.095541 19814 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70a86ea84d1d49d1b0950b4ce3a3d770. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.097888 20011 tablet_replica.cc:333] T 608bed5b04394bfca1776b5d6c502df1 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:18.097544 20201 leader_election.cc:290] T f5a0bd0e205f4889be93739dfbd2a816 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:18.108253 20202 raft_consensus.cc:515] T c2d333f892f84a258f8a148a90ce3b06 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:18.116580 20011 raft_consensus.cc:2243] T 608bed5b04394bfca1776b5d6c502df1 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:18.127576 20011 pending_rounds.cc:70] T 608bed5b04394bfca1776b5d6c502df1 P 7c1b52895a14482fb54a6758f5054f8f: Trying to abort 1 pending ops.
I20260925 13:56:18.127898 20011 pending_rounds.cc:77] T 608bed5b04394bfca1776b5d6c502df1 P 7c1b52895a14482fb54a6758f5054f8f: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251388104572928 op_type: NO_OP noop_request { }
I20260925 13:56:18.128350 20011 raft_consensus.cc:2889] T 608bed5b04394bfca1776b5d6c502df1 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:56:18.132732 20011 raft_consensus.cc:2272] T 608bed5b04394bfca1776b5d6c502df1 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:18.125874 19695 raft_consensus.cc:2468] T f5a0bd0e205f4889be93739dfbd2a816 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:18.134311 19488 leader_election.cc:304] T f5a0bd0e205f4889be93739dfbd2a816 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3; no voters:
I20260925 13:56:18.135947 20198 raft_consensus.cc:2804] T f5a0bd0e205f4889be93739dfbd2a816 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:18.136380 20198 raft_consensus.cc:697] T f5a0bd0e205f4889be93739dfbd2a816 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Running, Role: LEADER
I20260925 13:56:18.137084 20198 consensus_queue.cc:237] T f5a0bd0e205f4889be93739dfbd2a816 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:18.138865 19273 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 26644768f516412f8c95d68a76bb99c5 (table default.Table_bd52a86b8ff14bdbb06fcf88a60b1e82 [id=203024d67e444f839391c8670d7c7d3b]). Expected version 2 got 0
I20260925 13:56:18.144979 19273 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet bae8c5b6e1014731901bc3f336104784 (table default.Table_99593e98f3394811b4c91e234185cbef [id=0736e722eb8c43e9b6a151a7c2880fe3]). Expected version 1 got 0
I20260925 13:56:18.154340 19942 tablet_bootstrap.cc:492] T a88bbe1ebee7468d8f49dfd25637fe83 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:18.154726 19942 ts_tablet_manager.cc:1403] T a88bbe1ebee7468d8f49dfd25637fe83 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.207s user 0.006s sys 0.003s
I20260925 13:56:18.156162 19838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65a8cd3060c405c871270543b4623f7" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:18.156971 19838 raft_consensus.cc:2393] T c65a8cd3060c405c871270543b4623f7 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7c1b52895a14482fb54a6758f5054f8f in current term 1: Already voted for candidate 8f578869bc084a5d8a9a69f143954eb3 in this term.
I20260925 13:56:18.157986 19942 raft_consensus.cc:359] T a88bbe1ebee7468d8f49dfd25637fe83 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:18.158631 19942 raft_consensus.cc:385] T a88bbe1ebee7468d8f49dfd25637fe83 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:18.158911 19942 raft_consensus.cc:740] T a88bbe1ebee7468d8f49dfd25637fe83 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:18.159622 19942 consensus_queue.cc:260] T a88bbe1ebee7468d8f49dfd25637fe83 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:18.162947 19942 ts_tablet_manager.cc:1434] T a88bbe1ebee7468d8f49dfd25637fe83 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
W20260925 13:56:18.170118 20289 log.cc:927] Time spent T dc14c37aa393407a87af0d0eb8b88d82 P f790493ba7ad403993b678f2df2b55be: Append to log took a long time: real 0.065s user 0.000s sys 0.000s
I20260925 13:56:18.168628 19832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26644768f516412f8c95d68a76bb99c5" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:18.171965 19832 raft_consensus.cc:2393] T 26644768f516412f8c95d68a76bb99c5 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7c1b52895a14482fb54a6758f5054f8f in current term 1: Already voted for candidate 8f578869bc084a5d8a9a69f143954eb3 in this term.
I20260925 13:56:18.173177 19938 raft_consensus.cc:359] T 137c43fb74ec4cc48d30ed3edac81d98 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:18.173771 19938 raft_consensus.cc:385] T 137c43fb74ec4cc48d30ed3edac81d98 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:18.174043 19938 raft_consensus.cc:740] T 137c43fb74ec4cc48d30ed3edac81d98 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:18.175282 19675 tablet_service.cc:1511] Processing CreateTablet for tablet 5788849b7336496488e5401b4ac91191 (DEFAULT_TABLE table=default.Table_16c91b8d55be449899b5e3a1884d81b4 [id=72cd050d1582435e8a7e8c101df3e117]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.183033 19532 tablet_service.cc:1511] Processing CreateTablet for tablet 5788849b7336496488e5401b4ac91191 (DEFAULT_TABLE table=default.Table_16c91b8d55be449899b5e3a1884d81b4 [id=72cd050d1582435e8a7e8c101df3e117]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.184000 19532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5788849b7336496488e5401b4ac91191. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.185388 19278 catalog_manager.cc:6128] Not sending DeleteTablet requests; no consensus state for tablet 5a995eaa23ad4ce39341c4dd3af75c3f
I20260925 13:56:18.186290 19820 tablet_service.cc:1511] Processing CreateTablet for tablet 5788849b7336496488e5401b4ac91191 (DEFAULT_TABLE table=default.Table_16c91b8d55be449899b5e3a1884d81b4 [id=72cd050d1582435e8a7e8c101df3e117]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 13:56:18.196151 19277 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8dc5ea164d6749b0926c63d6bda9dcc5 consensus state has no leader
I20260925 13:56:18.192704 19812 tablet_service.cc:1511] Processing CreateTablet for tablet c161ae3edfed4f8488c770ecc6332607 (DEFAULT_TABLE table=default.Table_475efa7896f84e658eb81238ceb58d8f [id=5e975bd1dd6e487e97377bc2896e6a3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.197853 19273 catalog_manager.cc:6391] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: default.Table_e814702f1f874529a98bd1631d3b5c84 [id=6c0ec9465714408da93ff6e47827eb48] alter complete (version 2)
I20260925 13:56:18.200239 19820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5788849b7336496488e5401b4ac91191. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.198004 19938 consensus_queue.cc:260] T 137c43fb74ec4cc48d30ed3edac81d98 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:18.201016 19813 tablet_service.cc:1511] Processing CreateTablet for tablet 0ad6442b2de249119ba0ef85dfc06da9 (DEFAULT_TABLE table=default.Table_1aa8069da6504812a85380a42ca07583 [id=a6d1215ba3be4fca903af9b17566b4a8]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 13:56:18.206539 20303 log.cc:927] Time spent T 6b52ed579b19479686247d3ecba9489a P f790493ba7ad403993b678f2df2b55be: Append to log took a long time: real 0.051s user 0.000s sys 0.000s
I20260925 13:56:18.215196 19670 tablet_service.cc:1511] Processing CreateTablet for tablet 0ad6442b2de249119ba0ef85dfc06da9 (DEFAULT_TABLE table=default.Table_1aa8069da6504812a85380a42ca07583 [id=a6d1215ba3be4fca903af9b17566b4a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.217206 19535 tablet_service.cc:1511] Processing CreateTablet for tablet 0ad6442b2de249119ba0ef85dfc06da9 (DEFAULT_TABLE table=default.Table_1aa8069da6504812a85380a42ca07583 [id=a6d1215ba3be4fca903af9b17566b4a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.218001 19669 tablet_service.cc:1511] Processing CreateTablet for tablet c161ae3edfed4f8488c770ecc6332607 (DEFAULT_TABLE table=default.Table_475efa7896f84e658eb81238ceb58d8f [id=5e975bd1dd6e487e97377bc2896e6a3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.218367 19535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ad6442b2de249119ba0ef85dfc06da9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.219929 19529 tablet_service.cc:1511] Processing CreateTablet for tablet c161ae3edfed4f8488c770ecc6332607 (DEFAULT_TABLE table=default.Table_475efa7896f84e658eb81238ceb58d8f [id=5e975bd1dd6e487e97377bc2896e6a3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.221007 19529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c161ae3edfed4f8488c770ecc6332607. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.227183 19273 catalog_manager.cc:6391] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: default.Table_74b6076b22674a31bc58fef71cc9bfb1 [id=d81cf07e5f9a49d4bbf5aceda3da5a25] alter complete (version 1)
I20260925 13:56:18.232908 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26644768f516412f8c95d68a76bb99c5" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:18.234284 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65a8cd3060c405c871270543b4623f7" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:18.234959 19554 raft_consensus.cc:2393] T c65a8cd3060c405c871270543b4623f7 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7c1b52895a14482fb54a6758f5054f8f in current term 1: Already voted for candidate 8f578869bc084a5d8a9a69f143954eb3 in this term.
I20260925 13:56:18.236047 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e6c8b9bbf4549b3a8685f886973ed9c" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:18.248592 19942 tablet_bootstrap.cc:492] T e6e59d6f4f6e4de1be1cf450620cfa59 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:18.248800 19675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5788849b7336496488e5401b4ac91191. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.250651 19273 catalog_manager.cc:6391] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: default.Table_83f9b6c4585f4edbb857f16696339ff3 [id=d804d75fe5004ea4a1dc3245acb39ae7] alter complete (version 1)
I20260925 13:56:18.253806 19670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ad6442b2de249119ba0ef85dfc06da9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.254441 19942 tablet_bootstrap.cc:654] T e6e59d6f4f6e4de1be1cf450620cfa59 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:18.257268 19669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c161ae3edfed4f8488c770ecc6332607. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.274389 19942 tablet_bootstrap.cc:492] T e6e59d6f4f6e4de1be1cf450620cfa59 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:18.274874 19942 ts_tablet_manager.cc:1403] T e6e59d6f4f6e4de1be1cf450620cfa59 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.026s user 0.010s sys 0.001s
I20260925 13:56:18.277869 19942 raft_consensus.cc:359] T e6e59d6f4f6e4de1be1cf450620cfa59 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:18.287451 19942 raft_consensus.cc:385] T e6e59d6f4f6e4de1be1cf450620cfa59 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:18.287751 19942 raft_consensus.cc:740] T e6e59d6f4f6e4de1be1cf450620cfa59 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:18.288456 19942 consensus_queue.cc:260] T e6e59d6f4f6e4de1be1cf450620cfa59 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:18.289350 19938 ts_tablet_manager.cc:1434] T 137c43fb74ec4cc48d30ed3edac81d98 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.303s user 0.006s sys 0.002s
I20260925 13:56:18.293551 20011 ts_tablet_manager.cc:1916] T 608bed5b04394bfca1776b5d6c502df1 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:18.297792 19808 tablet_service.cc:1558] Processing DeleteTablet for tablet c2d333f892f84a258f8a148a90ce3b06 with delete_type TABLET_DATA_DELETED (replaced by tablet 908df914cfec490cb7058dbe7b501c97) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:18.297748 19534 tablet_service.cc:1511] Processing CreateTablet for tablet 70a86ea84d1d49d1b0950b4ce3a3d770 (DEFAULT_TABLE table=default.Table_9d043ec520e1422995a293b09996a713 [id=6496cd8160f9458eb2ef4dcdd2fb67cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.299403 19534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70a86ea84d1d49d1b0950b4ce3a3d770. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.302640 20297 raft_consensus.cc:493] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:18.303745 19668 tablet_service.cc:1511] Processing CreateTablet for tablet a26f5a02311c435a8c2434fd030c80a6 (DEFAULT_TABLE table=default.Table_65608937f2cd4986b2588a4831750601 [id=7079cc0c725b4578a1b11c7dd3ef9342]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.304472 19942 ts_tablet_manager.cc:1434] T e6e59d6f4f6e4de1be1cf450620cfa59 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.029s user 0.004s sys 0.000s
I20260925 13:56:18.304826 19668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a26f5a02311c435a8c2434fd030c80a6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:18.307147 19773 leader_election.cc:343] T f514fa52e7524cd3ba07f3f65de532d4 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:18.307668 19773 leader_election.cc:304] T f514fa52e7524cd3ba07f3f65de532d4 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:18.308426 19835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5a0bd0e205f4889be93739dfbd2a816" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:18.308976 19835 raft_consensus.cc:3060] T f5a0bd0e205f4889be93739dfbd2a816 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:18.299396 19959 ts_tablet_manager.cc:1929] T 1832cff5eb3e4da4b9766ae1591df81a P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:18.310380 19959 log.cc:1199] T 1832cff5eb3e4da4b9766ae1591df81a P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/1832cff5eb3e4da4b9766ae1591df81a
I20260925 13:56:18.314399 19942 tablet_bootstrap.cc:492] T 1da6eadb1648479889027fa911f4676b P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:18.303109 20297 raft_consensus.cc:515] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:18.322077 19267 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet f50ea68e3b3c4459b3f5e5b0df724724 (table default.Table_6d0ced912e4e4c188553d0839f794603 [id=f1353f53d28748c8a8a4a2c6259581bf], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet f50ea68e3b3c4459b3f5e5b0df724724 consensus state has no leader
I20260925 13:56:18.322265 19938 tablet_bootstrap.cc:492] T 28219970809145bc9d4e2a3289a9479a P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:18.311749 19812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c161ae3edfed4f8488c770ecc6332607. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.342538 19811 tablet_service.cc:1511] Processing CreateTablet for tablet a26f5a02311c435a8c2434fd030c80a6 (DEFAULT_TABLE table=default.Table_65608937f2cd4986b2588a4831750601 [id=7079cc0c725b4578a1b11c7dd3ef9342]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.345124 19818 tablet_service.cc:1511] Processing CreateTablet for tablet f50ea68e3b3c4459b3f5e5b0df724724 (DEFAULT_TABLE table=default.Table_6d0ced912e4e4c188553d0839f794603 [id=f1353f53d28748c8a8a4a2c6259581bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.347713 19807 tablet_service.cc:1558] Processing DeleteTablet for tablet 7ea4fd6817654bff9c0fe8c6e42ed6b9 with delete_type TABLET_DATA_DELETED (replaced by tablet 321c740ba8eb4cbb8974305c5f8702af) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:18.314939 20281 raft_consensus.cc:2749] T f514fa52e7524cd3ba07f3f65de532d4 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:56:18.353183 19773 leader_election.cc:343] T 3e6c8b9bbf4549b3a8685f886973ed9c P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:18.353549 19773 leader_election.cc:304] T 3e6c8b9bbf4549b3a8685f886973ed9c P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:18.354598 19817 tablet_service.cc:1511] Processing CreateTablet for tablet eb794cef052b43da831531455e2b5f15 (DEFAULT_TABLE table=default.Table_547f995b454c497bb5f8d7241ab26876 [id=d2bc135460ab4f2fad4740a281bfc6e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.340744 19535 tablet_service.cc:1511] Processing CreateTablet for tablet a26f5a02311c435a8c2434fd030c80a6 (DEFAULT_TABLE table=default.Table_65608937f2cd4986b2588a4831750601 [id=7079cc0c725b4578a1b11c7dd3ef9342]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.374977 19535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a26f5a02311c435a8c2434fd030c80a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.387135 19939 tablet_bootstrap.cc:654] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:18.395661 19531 tablet_service.cc:1558] Processing DeleteTablet for tablet c2d333f892f84a258f8a148a90ce3b06 with delete_type TABLET_DATA_DELETED (replaced by tablet 908df914cfec490cb7058dbe7b501c97) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:18.400210 19942 tablet_bootstrap.cc:654] T 1da6eadb1648479889027fa911f4676b P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:18.402330 19809 tablet_service.cc:1511] Processing CreateTablet for tablet f16050752ad44720ba8703ef10bbb359 (DEFAULT_TABLE table=default.Table_07d4243785b14434b731eb80cf4f242c [id=b954461264164e969a40af810627fcb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.403754 19959 ts_tablet_manager.cc:1950] T 1832cff5eb3e4da4b9766ae1591df81a P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:18.406139 19959 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 92) took 1839 ms. Trace:
I20260925 13:56:18.407480 20281 raft_consensus.cc:493] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:18.407891 20281 raft_consensus.cc:515] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:18.410292 20281 leader_election.cc:290] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:18.410873 20281 raft_consensus.cc:2749] T 3e6c8b9bbf4549b3a8685f886973ed9c P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:18.362291 19813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ad6442b2de249119ba0ef85dfc06da9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.406603 19959 rpcz_store.cc:276] 0925 13:56:16.566353 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:16.569178 (+ 2825us) service_pool.cc:224] Handling call
0925 13:56:17.431388 (+862210us) ts_tablet_manager.cc:1151] Deleting tablet 1832cff5eb3e4da4b9766ae1591df81a
0925 13:56:17.431433 (+ 45us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:18.081892 (+650459us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:18.165348 (+ 83456us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:18.406053 (+240705us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":840213}
I20260925 13:56:18.411520 19810 tablet_service.cc:1511] Processing CreateTablet for tablet ec7fe5c834be4a51b627b8b1c3ebe888 (DEFAULT_TABLE table=default.Table_86e7099f24054da589b5a6f7de9ab131 [id=191bb6f9e3b14882aabe499934d8eb57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.346813 19955 ts_tablet_manager.cc:1929] T a88bbe1ebee7468d8f49dfd25637fe83 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:18.349826 19528 tablet_service.cc:1511] Processing CreateTablet for tablet f16050752ad44720ba8703ef10bbb359 (DEFAULT_TABLE table=default.Table_07d4243785b14434b731eb80cf4f242c [id=b954461264164e969a40af810627fcb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.416198 19528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f16050752ad44720ba8703ef10bbb359. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.351747 19533 tablet_service.cc:1511] Processing CreateTablet for tablet ec7fe5c834be4a51b627b8b1c3ebe888 (DEFAULT_TABLE table=default.Table_86e7099f24054da589b5a6f7de9ab131 [id=191bb6f9e3b14882aabe499934d8eb57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.354509 20240 raft_consensus.cc:493] T 5943795bc8b24bb48b80d6d06d17750b P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:18.427742 20240 raft_consensus.cc:515] T 5943795bc8b24bb48b80d6d06d17750b P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:18.430187 20240 leader_election.cc:290] T 5943795bc8b24bb48b80d6d06d17750b P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:18.431501 19533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec7fe5c834be4a51b627b8b1c3ebe888. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.430691 19819 tablet_service.cc:1511] Processing CreateTablet for tablet 1215d875209a40d38cc527e2795d6c75 (DEFAULT_TABLE table=default.Table_34fbf7824415492a8d02f09e2eb91bc6 [id=ef6bdff197af4006b3502ba17dd82eb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.357431 19490 leader_election.cc:304] T c65a8cd3060c405c871270543b4623f7 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:18.435382 20240 raft_consensus.cc:2804] T c65a8cd3060c405c871270543b4623f7 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:18.436156 20240 raft_consensus.cc:697] T c65a8cd3060c405c871270543b4623f7 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Running, Role: LEADER
I20260925 13:56:18.436911 20240 consensus_queue.cc:237] T c65a8cd3060c405c871270543b4623f7 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:18.360790 20202 leader_election.cc:290] T c2d333f892f84a258f8a148a90ce3b06 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:18.374054 19527 tablet_service.cc:1558] Processing DeleteTablet for tablet 7ea4fd6817654bff9c0fe8c6e42ed6b9 with delete_type TABLET_DATA_DELETED (replaced by tablet 321c740ba8eb4cbb8974305c5f8702af) from {username='slave'} at 127.0.0.1:40018
W20260925 13:56:18.441715 19490 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 84371.1us
I20260925 13:56:18.442757 20198 consensus_queue.cc:1048] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:18.448149 19955 log.cc:1199] T a88bbe1ebee7468d8f49dfd25637fe83 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/a88bbe1ebee7468d8f49dfd25637fe83
I20260925 13:56:18.449088 19955 ts_tablet_manager.cc:1950] T a88bbe1ebee7468d8f49dfd25637fe83 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:18.453011 19629 leader_election.cc:304] T 26644768f516412f8c95d68a76bb99c5 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:18.462534 19959 tablet_replica.cc:333] T dc14c37aa393407a87af0d0eb8b88d82 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:18.423892 19808 tablet_service.cc:1511] Processing CreateTablet for tablet 9059655f2c3d4e86b3b389b5b9a68891 (DEFAULT_TABLE table=default.Table_1930b257c6824b38ab3c8cba088f9c42 [id=0c13bfd171f2464b887f15d085439fa3]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 13:56:18.465095 19773 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.193:38269, user_credentials={real_user=slave}} blocked reactor thread for 111972us
I20260925 13:56:18.467331 19675 tablet_service.cc:1511] Processing CreateTablet for tablet ec7fe5c834be4a51b627b8b1c3ebe888 (DEFAULT_TABLE table=default.Table_86e7099f24054da589b5a6f7de9ab131 [id=191bb6f9e3b14882aabe499934d8eb57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.475749 19629 leader_election.cc:304] T c65a8cd3060c405c871270543b4623f7 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:18.474046 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 1832cff5eb3e4da4b9766ae1591df81a (table default.Table_9896a33d94e44d76ada6ec8d00c87ad7 [id=a5549f9e40c644afa1fafcc16d649f68]) successfully deleted
I20260925 13:56:18.476867 20252 raft_consensus.cc:3060] T c65a8cd3060c405c871270543b4623f7 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:18.477861 19667 tablet_service.cc:1558] Processing DeleteTablet for tablet 7ea4fd6817654bff9c0fe8c6e42ed6b9 with delete_type TABLET_DATA_DELETED (replaced by tablet 321c740ba8eb4cbb8974305c5f8702af) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:18.478587 19691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea4fd6817654bff9c0fe8c6e42ed6b9" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:18.471969 20242 raft_consensus.cc:3060] T 26644768f516412f8c95d68a76bb99c5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:18.481479 19675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec7fe5c834be4a51b627b8b1c3ebe888. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.482012 19694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d333f892f84a258f8a148a90ce3b06" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:18.489357 19674 tablet_service.cc:1511] Processing CreateTablet for tablet f16050752ad44720ba8703ef10bbb359 (DEFAULT_TABLE table=default.Table_07d4243785b14434b731eb80cf4f242c [id=b954461264164e969a40af810627fcb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.490496 19674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f16050752ad44720ba8703ef10bbb359. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.500869 19694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5943795bc8b24bb48b80d6d06d17750b" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:18.472743 19810 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec7fe5c834be4a51b627b8b1c3ebe888. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.506304 20252 raft_consensus.cc:2749] T c65a8cd3060c405c871270543b4623f7 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:18.508263 19672 tablet_service.cc:1558] Processing DeleteTablet for tablet c2d333f892f84a258f8a148a90ce3b06 with delete_type TABLET_DATA_DELETED (replaced by tablet 908df914cfec490cb7058dbe7b501c97) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:18.510465 20242 raft_consensus.cc:2749] T 26644768f516412f8c95d68a76bb99c5 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:18.516166 19809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f16050752ad44720ba8703ef10bbb359. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.494408 19959 raft_consensus.cc:2243] T dc14c37aa393407a87af0d0eb8b88d82 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:18.523766 19959 raft_consensus.cc:2272] T dc14c37aa393407a87af0d0eb8b88d82 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:18.526414 19942 tablet_bootstrap.cc:492] T 1da6eadb1648479889027fa911f4676b P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:18.526890 19942 ts_tablet_manager.cc:1403] T 1da6eadb1648479889027fa911f4676b P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.213s user 0.011s sys 0.003s
W20260925 13:56:18.527040 19266 consensus_peers.cc:597] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 -> Peer fc3953a2ba984a658630c423ec9b9728 (127.18.182.252:36181): Couldn't send request to peer fc3953a2ba984a658630c423ec9b9728. Status: Network error: Client connection negotiation failed: client connection to 127.18.182.252:36181: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260925 13:56:18.529383 19942 raft_consensus.cc:359] T 1da6eadb1648479889027fa911f4676b P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:18.530073 19942 raft_consensus.cc:385] T 1da6eadb1648479889027fa911f4676b P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:18.530364 19942 raft_consensus.cc:740] T 1da6eadb1648479889027fa911f4676b P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:18.530897 19811 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a26f5a02311c435a8c2434fd030c80a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.531054 19942 consensus_queue.cc:260] T 1da6eadb1648479889027fa911f4676b P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:18.532793 19808 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9059655f2c3d4e86b3b389b5b9a68891. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.501508 20297 leader_election.cc:290] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
W20260925 13:56:18.540843 19264 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.18.182.193:38269, user_credentials={real_user=slave}} blocked reactor thread for 141552us
I20260925 13:56:18.482173 19835 raft_consensus.cc:2468] T f5a0bd0e205f4889be93739dfbd2a816 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:18.543469 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 980d1264c0ea45f3b2b762509e0b2c77 (table default.Table_8290acbcb9d84a48aad169c54251bc19 [id=90312b1b0c254ff9ad5693f561114a7f]) successfully deleted
W20260925 13:56:18.544427 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet f514fa52e7524cd3ba07f3f65de532d4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f514fa52e7524cd3ba07f3f65de532d4 already in progress: creating tablet
W20260925 13:56:18.545356 19490 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 97434.6us
W20260925 13:56:18.546576 19488 leader_election.cc:343] T c2d333f892f84a258f8a148a90ce3b06 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:18.551730 19817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb794cef052b43da831531455e2b5f15. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:18.552037 19488 leader_election.cc:343] T 5943795bc8b24bb48b80d6d06d17750b P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:18.554723 19490 consensus_peers.cc:597] T 4d3dbd9796564472b44a31ccaf56d898 P 8f578869bc084a5d8a9a69f143954eb3 -> Peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Couldn't send request to peer f790493ba7ad403993b678f2df2b55be. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4d3dbd9796564472b44a31ccaf56d898. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:18.569008 19939 tablet_bootstrap.cc:492] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:18.569542 19939 ts_tablet_manager.cc:1403] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.479s user 0.010s sys 0.000s
I20260925 13:56:18.571486 19959 ts_tablet_manager.cc:1916] T dc14c37aa393407a87af0d0eb8b88d82 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:18.588112 19818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f50ea68e3b3c4459b3f5e5b0df724724. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.588666 19819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1215d875209a40d38cc527e2795d6c75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.620823 19695 raft_consensus.cc:3060] T 1da6eadb1648479889027fa911f4676b P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
W20260925 13:56:18.625836 19773 leader_election.cc:343] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:18.626389 19773 leader_election.cc:343] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Not found: Tablet not found: 47521fb74f9c40ee999a7f704aa551c9
I20260925 13:56:18.628923 20240 consensus_queue.cc:1048] T 1da6eadb1648479889027fa911f4676b P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:18.633857 19816 tablet_service.cc:1511] Processing CreateTablet for tablet 0e8aa12e95dc434a98a426499593d9a5 (DEFAULT_TABLE table=default.Table_b42a6e2de4e74cb7a6f01dba22f3b94c [id=03497b028f664f1794f4ff6b53912124]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.634865 19816 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e8aa12e95dc434a98a426499593d9a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.643374 19815 tablet_service.cc:1511] Processing CreateTablet for tablet 3aaf29a2310a41d0971eded7ef51fdc9 (DEFAULT_TABLE table=default.Table_954c5af93adb4e8586f870597a63bb74 [id=f4beec04dea942f98007b1f4cf33c9ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.644379 19815 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3aaf29a2310a41d0971eded7ef51fdc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.648106 19942 ts_tablet_manager.cc:1434] T 1da6eadb1648479889027fa911f4676b P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.121s user 0.005s sys 0.000s
W20260925 13:56:18.648396 19490 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 93123.7us
I20260925 13:56:18.649899 19835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d333f892f84a258f8a148a90ce3b06" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:18.650406 19835 raft_consensus.cc:3060] T c2d333f892f84a258f8a148a90ce3b06 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:18.651626 20240 raft_consensus.cc:493] T 2a1f34dba13143b097a38bab3782c404 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:18.652485 20240 raft_consensus.cc:515] T 2a1f34dba13143b097a38bab3782c404 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:18.655176 20240 leader_election.cc:290] T 2a1f34dba13143b097a38bab3782c404 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:18.655840 19814 tablet_service.cc:1511] Processing CreateTablet for tablet 953d41f26f5f4027acfd374c6ec4ee75 (DEFAULT_TABLE table=default.Table_9cdc814c021f49dea82875834691ba9e [id=e3fd45d69e384f9eb2ebce81bf195e5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.656842 19814 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 953d41f26f5f4027acfd374c6ec4ee75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.550238 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 79) took 3175 ms. Trace:
I20260925 13:56:18.662348 20318 tablet.cc:1751] T 6b52ed579b19479686247d3ecba9489a P f790493ba7ad403993b678f2df2b55be: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:18.659399 20315 raft_consensus.cc:493] T 137c43fb74ec4cc48d30ed3edac81d98 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:18.664009 20315 raft_consensus.cc:515] T 137c43fb74ec4cc48d30ed3edac81d98 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:18.659418 19955 rpcz_store.cc:276] 0925 13:56:15.374611 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:15.374710 (+ 99us) service_pool.cc:224] Handling call
0925 13:56:17.791159 (+2416449us) ts_tablet_manager.cc:1151] Deleting tablet a88bbe1ebee7468d8f49dfd25637fe83
0925 13:56:17.791198 (+ 39us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:17.957623 (+166425us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:18.346698 (+389075us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:18.550180 (+203482us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2415929}
I20260925 13:56:18.668676 19955 tablet_replica.cc:333] T 7292d04b411c415394e9d0df85aaf579 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:18.669350 19955 raft_consensus.cc:2243] T 7292d04b411c415394e9d0df85aaf579 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:18.668278 19807 tablet_service.cc:1511] Processing CreateTablet for tablet 2711e7a6973546b1822d93f91095dae2 (DEFAULT_TABLE table=default.Table_83acf26db75e4c02b312cef4df19901f [id=7ec3d38b986747ca99d6a49e6a7dbe1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.670081 19955 raft_consensus.cc:2272] T 7292d04b411c415394e9d0df85aaf579 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:18.671269 19807 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2711e7a6973546b1822d93f91095dae2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.672724 19955 ts_tablet_manager.cc:1916] T 7292d04b411c415394e9d0df85aaf579 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:18.677914 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47521fb74f9c40ee999a7f704aa551c9" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:18.678463 19554 raft_consensus.cc:2468] T 47521fb74f9c40ee999a7f704aa551c9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 1.
I20260925 13:56:18.679733 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea4fd6817654bff9c0fe8c6e42ed6b9" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:18.687587 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a1f34dba13143b097a38bab3782c404" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:18.696571 19488 leader_election.cc:343] T 2a1f34dba13143b097a38bab3782c404 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:18.698020 19488 consensus_peers.cc:597] T bbfc4f99a5964e2ba6146e390e8dba44 P 8f578869bc084a5d8a9a69f143954eb3 -> Peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Couldn't send request to peer 7c1b52895a14482fb54a6758f5054f8f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:18.699833 19942 tablet_bootstrap.cc:492] T 5943795bc8b24bb48b80d6d06d17750b P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
W20260925 13:56:18.713706 19264 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.18.182.194:38641, user_credentials={real_user=slave}} blocked reactor thread for 169356us
W20260925 13:56:18.714651 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 1da6eadb1648479889027fa911f4676b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1da6eadb1648479889027fa911f4676b already in progress: creating tablet
I20260925 13:56:18.719671 19806 tablet_service.cc:1511] Processing CreateTablet for tablet a8a679a453e0430b866edc707fe7c234 (DEFAULT_TABLE table=default.Table_50a86db358374d4ea5164480fefdfee5 [id=7556e78d915f446fb0f67f55fcaa2bed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.728420 19939 raft_consensus.cc:359] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:18.729252 19939 raft_consensus.cc:385] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:18.729557 19939 raft_consensus.cc:740] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:18.730188 19939 consensus_queue.cc:260] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:18.733351 19806 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8a679a453e0430b866edc707fe7c234. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.741480 20326 tablet.cc:1751] T 1513c0048bd244028d3dfa8d118bc14e P f790493ba7ad403993b678f2df2b55be: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:18.547147 19274 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_6f3fcafe361344c6920c158efa979438"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:18.547749 19272 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_93aa193a56f246aeaa089b2bfc0e9d8c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:18.764948 19274 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6f3fcafe361344c6920c158efa979438 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 13:56:18.765029 19272 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_93aa193a56f246aeaa089b2bfc0e9d8c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:18.548830 19276 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_052e4e4ee53e4e408b18cd0ce8063d31" } modify_external_catalogs: true
I20260925 13:56:18.765591 19276 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_052e4e4ee53e4e408b18cd0ce8063d31" } modify_external_catalogs: true
I20260925 13:56:18.549707 19279 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_883c781744824c2a9e6e86917bf2143e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:18.766891 19264 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.18.182.194:38641, user_credentials={real_user=slave}} blocked reactor thread for 52289.6us
I20260925 13:56:18.594648 19278 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet e6e59d6f4f6e4de1be1cf450620cfa59 (table default.Table_546097946d054781b0acae5186c5d43d [id=912de4c02216461f8a69b880207d8652]). Expected version 1 got 0
I20260925 13:56:18.544997 19275 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_4559d216bb6041bdba82905a01b2412a" } new_table_name: "default.Table_4a822937f96346c1bd348b842fe60dfc" modify_external_catalogs: true
W20260925 13:56:18.768042 19279 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_883c781744824c2a9e6e86917bf2143e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:18.592487 19273 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet bae8c5b6e1014731901bc3f336104784 (table default.Table_99593e98f3394811b4c91e234185cbef [id=0736e722eb8c43e9b6a151a7c2880fe3]). Expected version 1 got 0
I20260925 13:56:18.771335 19273 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 3e6c8b9bbf4549b3a8685f886973ed9c (table default.Table_d0dadd9b31b143fd8dd5adcb7f752c5f [id=beee0ac537e44859b0f7c76d6808b0e0]). Expected version 1 got 0
I20260925 13:56:18.784538 19277 catalog_manager.cc:5838] T bbfc4f99a5964e2ba6146e390e8dba44 P 8f578869bc084a5d8a9a69f143954eb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "8f578869bc084a5d8a9a69f143954eb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:18.791993 19277 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet bae8c5b6e1014731901bc3f336104784 (table default.Table_99593e98f3394811b4c91e234185cbef [id=0736e722eb8c43e9b6a151a7c2880fe3]). Expected version 1 got 0
I20260925 13:56:18.792730 19277 catalog_manager.cc:5838] T f5a0bd0e205f4889be93739dfbd2a816 P 8f578869bc084a5d8a9a69f143954eb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "8f578869bc084a5d8a9a69f143954eb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: HEALTHY } } }
I20260925 13:56:18.797849 19939 ts_tablet_manager.cc:1434] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.228s user 0.003s sys 0.002s
I20260925 13:56:18.801851 20011 ts_tablet_manager.cc:1929] T 608bed5b04394bfca1776b5d6c502df1 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:18.803315 19820 tablet_service.cc:1511] Processing CreateTablet for tablet 0fdb45b31f93466c8bed1b848c6cdf15 (DEFAULT_TABLE table=default.Table_cd0b384bb04c44759472fcffcab11ee3 [id=9eab89f862ca4be88e4131feeb777864]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.807945 19820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fdb45b31f93466c8bed1b848c6cdf15. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:18.809976 19264 catalog_manager.cc:5030] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 00b500182bb04a72b80cfa9103e85fa9 because the tablet was not found. No further retry: Not found: Tablet not found: 00b500182bb04a72b80cfa9103e85fa9
I20260925 13:56:18.815322 19528 tablet_service.cc:1511] Processing CreateTablet for tablet 9059655f2c3d4e86b3b389b5b9a68891 (DEFAULT_TABLE table=default.Table_1930b257c6824b38ab3c8cba088f9c42 [id=0c13bfd171f2464b887f15d085439fa3]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 13:56:18.759994 19490 leader_election.cc:343] T 5943795bc8b24bb48b80d6d06d17750b P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 5943795bc8b24bb48b80d6d06d17750b
I20260925 13:56:18.819554 19490 leader_election.cc:304] T 5943795bc8b24bb48b80d6d06d17750b P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3; no voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:18.820339 20198 raft_consensus.cc:2749] T 5943795bc8b24bb48b80d6d06d17750b P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:56:18.821305 19490 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 61365.7us
I20260925 13:56:18.824100 20198 consensus_queue.cc:1048] T bbfc4f99a5964e2ba6146e390e8dba44 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:18.824807 19818 tablet_service.cc:1511] Processing CreateTablet for tablet 5a9b3dfaa7474611a1824d63eb906b43 (DEFAULT_TABLE table=default.Table_67306a96449b4a7bbb4b96498a396cea [id=bcf9d7c5e4f44267b9204bc3c2d90584]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.828831 19818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a9b3dfaa7474611a1824d63eb906b43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.809773 19834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a1f34dba13143b097a38bab3782c404" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:18.829566 19834 raft_consensus.cc:2468] T 2a1f34dba13143b097a38bab3782c404 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:18.839156 19273 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 26644768f516412f8c95d68a76bb99c5 (table default.Table_bd52a86b8ff14bdbb06fcf88a60b1e82 [id=203024d67e444f839391c8670d7c7d3b]). Expected version 2 got 0
I20260925 13:56:18.840430 19835 raft_consensus.cc:2468] T c2d333f892f84a258f8a148a90ce3b06 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:18.844831 20325 tablet.cc:1751] T 18a6d1c4eb7948149e709a380da50d93 P f790493ba7ad403993b678f2df2b55be: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 13:56:18.846306 19529 tablet_service.cc:1511] Processing CreateTablet for tablet eb794cef052b43da831531455e2b5f15 (DEFAULT_TABLE table=default.Table_547f995b454c497bb5f8d7241ab26876 [id=d2bc135460ab4f2fad4740a281bfc6e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.849738 19527 tablet_service.cc:1511] Processing CreateTablet for tablet 1215d875209a40d38cc527e2795d6c75 (DEFAULT_TABLE table=default.Table_34fbf7824415492a8d02f09e2eb91bc6 [id=ef6bdff197af4006b3502ba17dd82eb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.852336 19530 tablet_service.cc:1511] Processing CreateTablet for tablet f50ea68e3b3c4459b3f5e5b0df724724 (DEFAULT_TABLE table=default.Table_6d0ced912e4e4c188553d0839f794603 [id=f1353f53d28748c8a8a4a2c6259581bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.853534 19959 ts_tablet_manager.cc:1929] T dc14c37aa393407a87af0d0eb8b88d82 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:18.831846 20315 leader_election.cc:290] T 137c43fb74ec4cc48d30ed3edac81d98 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:18.860090 20011 log.cc:1199] T 608bed5b04394bfca1776b5d6c502df1 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/608bed5b04394bfca1776b5d6c502df1
I20260925 13:56:18.860962 20011 ts_tablet_manager.cc:1950] T 608bed5b04394bfca1776b5d6c502df1 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:18.804852 19942 tablet_bootstrap.cc:654] T 5943795bc8b24bb48b80d6d06d17750b P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:18.813663 19676 tablet_service.cc:1511] Processing CreateTablet for tablet 9059655f2c3d4e86b3b389b5b9a68891 (DEFAULT_TABLE table=default.Table_1930b257c6824b38ab3c8cba088f9c42 [id=0c13bfd171f2464b887f15d085439fa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.848987 19675 tablet_service.cc:1511] Processing CreateTablet for tablet 1215d875209a40d38cc527e2795d6c75 (DEFAULT_TABLE table=default.Table_34fbf7824415492a8d02f09e2eb91bc6 [id=ef6bdff197af4006b3502ba17dd82eb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.864358 19675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1215d875209a40d38cc527e2795d6c75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.851779 19668 tablet_service.cc:1511] Processing CreateTablet for tablet f50ea68e3b3c4459b3f5e5b0df724724 (DEFAULT_TABLE table=default.Table_6d0ced912e4e4c188553d0839f794603 [id=f1353f53d28748c8a8a4a2c6259581bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.868008 19668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f50ea68e3b3c4459b3f5e5b0df724724. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.869405 19490 leader_election.cc:304] T 2a1f34dba13143b097a38bab3782c404 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:18.870105 20198 raft_consensus.cc:2804] T 2a1f34dba13143b097a38bab3782c404 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:18.870432 20198 raft_consensus.cc:493] T 2a1f34dba13143b097a38bab3782c404 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:18.870741 20198 raft_consensus.cc:3060] T 2a1f34dba13143b097a38bab3782c404 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:18.856294 19671 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d3dbd9796564472b44a31ccaf56d898 with delete_type TABLET_DATA_DELETED (replaced by tablet 0ad6442b2de249119ba0ef85dfc06da9) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:18.856936 19670 tablet_service.cc:1558] Processing DeleteTablet for tablet 8239d0cd3b0d40bdb721e807c0f403d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:14 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:18.873440 19676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9059655f2c3d4e86b3b389b5b9a68891. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.878187 20198 raft_consensus.cc:515] T 2a1f34dba13143b097a38bab3782c404 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:18.880805 19689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a1f34dba13143b097a38bab3782c404" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:18.880982 20198 leader_election.cc:290] T 2a1f34dba13143b097a38bab3782c404 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:18.882184 19530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f50ea68e3b3c4459b3f5e5b0df724724. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.883495 19805 tablet_service.cc:1511] Processing CreateTablet for tablet 908df914cfec490cb7058dbe7b501c97 (DEFAULT_TABLE table=default.Table_c90b2c8024944acb9ffe76b506fff5a2 [id=d06e00bf26bd4c47a9e27f7eb35835df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.889454 19805 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 908df914cfec490cb7058dbe7b501c97. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:18.888931 19488 leader_election.cc:343] T 2a1f34dba13143b097a38bab3782c404 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:18.888254 19528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9059655f2c3d4e86b3b389b5b9a68891. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.892012 19939 tablet_bootstrap.cc:492] T e6e59d6f4f6e4de1be1cf450620cfa59 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:18.857844 19674 tablet_service.cc:1511] Processing CreateTablet for tablet eb794cef052b43da831531455e2b5f15 (DEFAULT_TABLE table=default.Table_547f995b454c497bb5f8d7241ab26876 [id=d2bc135460ab4f2fad4740a281bfc6e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.904995 19529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb794cef052b43da831531455e2b5f15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.853878 19959 log.cc:1199] T dc14c37aa393407a87af0d0eb8b88d82 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/dc14c37aa393407a87af0d0eb8b88d82
I20260925 13:56:18.905808 19674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb794cef052b43da831531455e2b5f15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.906160 19803 tablet_service.cc:1511] Processing CreateTablet for tablet 000aad39953c4688b543c7d882a08ec8 (DEFAULT_TABLE table=default.Table_3a3e48a0641a43a1a783c6ba6374a708 [id=e64fe44a237f48859a288a87a565ee5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.906556 19959 ts_tablet_manager.cc:1950] T dc14c37aa393407a87af0d0eb8b88d82 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:18.907294 19803 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 000aad39953c4688b543c7d882a08ec8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.907330 19959 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 93) took 2260 ms. Trace:
I20260925 13:56:18.907963 19527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1215d875209a40d38cc527e2795d6c75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.859580 19669 tablet_service.cc:1511] Processing CreateTablet for tablet 953d41f26f5f4027acfd374c6ec4ee75 (DEFAULT_TABLE table=default.Table_9cdc814c021f49dea82875834691ba9e [id=e3fd45d69e384f9eb2ebce81bf195e5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.918476 19669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 953d41f26f5f4027acfd374c6ec4ee75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.917228 19804 tablet_service.cc:1511] Processing CreateTablet for tablet e2e44b2ca25c4bbcb91288424b3e18b9 (DEFAULT_TABLE table=default.Table_261023ab15a845df82deea31f0eac8c8 [id=f03b9baa231c40999b38848e9059553e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.921609 19942 tablet_bootstrap.cc:492] T 5943795bc8b24bb48b80d6d06d17750b P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:18.922010 19942 ts_tablet_manager.cc:1403] T 5943795bc8b24bb48b80d6d06d17750b P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.222s user 0.005s sys 0.004s
I20260925 13:56:18.922873 19804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2e44b2ca25c4bbcb91288424b3e18b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.934106 20332 raft_consensus.cc:493] T e6e59d6f4f6e4de1be1cf450620cfa59 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:18.907673 19959 rpcz_store.cc:276] 0925 13:56:16.647017 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:16.740558 (+ 93541us) service_pool.cc:224] Handling call
0925 13:56:18.434619 (+1694061us) ts_tablet_manager.cc:1151] Deleting tablet dc14c37aa393407a87af0d0eb8b88d82
0925 13:56:18.462395 (+ 27776us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:18.611670 (+149275us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:18.853429 (+241759us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:18.907266 (+ 53837us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1693469}
I20260925 13:56:18.935789 20332 raft_consensus.cc:515] T e6e59d6f4f6e4de1be1cf450620cfa59 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:18.936383 19959 tablet_replica.cc:333] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:18.942022 20332 leader_election.cc:290] T e6e59d6f4f6e4de1be1cf450620cfa59 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:18.947541 19942 raft_consensus.cc:359] T 5943795bc8b24bb48b80d6d06d17750b P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:18.950635 19942 raft_consensus.cc:385] T 5943795bc8b24bb48b80d6d06d17750b P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:18.950906 19942 raft_consensus.cc:740] T 5943795bc8b24bb48b80d6d06d17750b P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:18.953358 19942 consensus_queue.cc:260] T 5943795bc8b24bb48b80d6d06d17750b P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:18.959339 19942 ts_tablet_manager.cc:1434] T 5943795bc8b24bb48b80d6d06d17750b P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.037s user 0.007s sys 0.000s
I20260925 13:56:18.974663 19939 tablet_bootstrap.cc:654] T e6e59d6f4f6e4de1be1cf450620cfa59 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:18.974676 19802 tablet_service.cc:1511] Processing CreateTablet for tablet f3b46e4fac694084b06b7086bcc596b9 (DEFAULT_TABLE table=default.Table_6f8114f6052845bb85b139bfedbed143 [id=c31f813d79a244afa726df0238376aba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.975863 19802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3b46e4fac694084b06b7086bcc596b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.979902 19817 tablet_service.cc:1511] Processing CreateTablet for tablet 1d641f09429546d683f2884c02152f73 (DEFAULT_TABLE table=default.Table_b1985cfbe41c4240a1c15abfd5741f90 [id=6d63400819ed4ce395ab408cb58407b9]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 13:56:18.980410 19490 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 111117us
I20260925 13:56:18.981515 19490 leader_election.cc:304] T c2d333f892f84a258f8a148a90ce3b06 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:18.863039 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 92) took 1071 ms. Trace:
I20260925 13:56:18.982877 20011 rpcz_store.cc:276] 0925 13:56:17.791795 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:17.791943 (+ 148us) service_pool.cc:224] Handling call
0925 13:56:18.097750 (+305807us) ts_tablet_manager.cc:1151] Deleting tablet 608bed5b04394bfca1776b5d6c502df1
0925 13:56:18.097784 (+ 34us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:18.558255 (+460471us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:18.801742 (+243487us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:18.862984 (+ 61242us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":305205}
I20260925 13:56:18.984616 19668 tablet_service.cc:1511] Processing CreateTablet for tablet 5a9b3dfaa7474611a1824d63eb906b43 (DEFAULT_TABLE table=default.Table_67306a96449b4a7bbb4b96498a396cea [id=bcf9d7c5e4f44267b9204bc3c2d90584]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.985720 19668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a9b3dfaa7474611a1824d63eb906b43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.986258 19676 tablet_service.cc:1511] Processing CreateTablet for tablet 0e8aa12e95dc434a98a426499593d9a5 (DEFAULT_TABLE table=default.Table_b42a6e2de4e74cb7a6f01dba22f3b94c [id=03497b028f664f1794f4ff6b53912124]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.987120 19676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e8aa12e95dc434a98a426499593d9a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.989244 19817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d641f09429546d683f2884c02152f73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.997069 19675 tablet_service.cc:1511] Processing CreateTablet for tablet 3aaf29a2310a41d0971eded7ef51fdc9 (DEFAULT_TABLE table=default.Table_954c5af93adb4e8586f870597a63bb74 [id=f4beec04dea942f98007b1f4cf33c9ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:18.997591 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 93) took 1097 ms. Trace:
I20260925 13:56:18.998179 19675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3aaf29a2310a41d0971eded7ef51fdc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.001035 19670 tablet_service.cc:1511] Processing CreateTablet for tablet 2711e7a6973546b1822d93f91095dae2 (DEFAULT_TABLE table=default.Table_83acf26db75e4c02b312cef4df19901f [id=7ec3d38b986747ca99d6a49e6a7dbe1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.002007 19670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2711e7a6973546b1822d93f91095dae2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.011817 19277 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet bae8c5b6e1014731901bc3f336104784 (table default.Table_99593e98f3394811b4c91e234185cbef [id=0736e722eb8c43e9b6a151a7c2880fe3]). Expected version 1 got 0
I20260925 13:56:19.012583 19277 catalog_manager.cc:5838] T c65a8cd3060c405c871270543b4623f7 P 8f578869bc084a5d8a9a69f143954eb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "8f578869bc084a5d8a9a69f143954eb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:19.014114 19277 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet 26644768f516412f8c95d68a76bb99c5 (table default.Table_bd52a86b8ff14bdbb06fcf88a60b1e82 [id=203024d67e444f839391c8670d7c7d3b]). Expected version 2 got 0
I20260925 13:56:19.014731 19277 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet 8dc5ea164d6749b0926c63d6bda9dcc5 (table default.Table_e0f7b807e8884431898298a1c635c784 [id=e49e015b173c48c1b6ef8112f60b33e4]). Expected version 1 got 0
I20260925 13:56:19.016850 20198 consensus_queue.cc:1048] T 1da6eadb1648479889027fa911f4676b P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:19.018842 20199 raft_consensus.cc:2804] T c2d333f892f84a258f8a148a90ce3b06 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:19.019294 20199 raft_consensus.cc:697] T c2d333f892f84a258f8a148a90ce3b06 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Running, Role: LEADER
I20260925 13:56:19.020117 20199 consensus_queue.cc:237] T c2d333f892f84a258f8a148a90ce3b06 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:19.021380 19535 tablet_service.cc:1511] Processing CreateTablet for tablet 3aaf29a2310a41d0971eded7ef51fdc9 (DEFAULT_TABLE table=default.Table_954c5af93adb4e8586f870597a63bb74 [id=f4beec04dea942f98007b1f4cf33c9ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.022346 19535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3aaf29a2310a41d0971eded7ef51fdc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.023689 19531 tablet_service.cc:1511] Processing CreateTablet for tablet 0e8aa12e95dc434a98a426499593d9a5 (DEFAULT_TABLE table=default.Table_b42a6e2de4e74cb7a6f01dba22f3b94c [id=03497b028f664f1794f4ff6b53912124]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.024688 19531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e8aa12e95dc434a98a426499593d9a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.032051 19533 tablet_service.cc:1511] Processing CreateTablet for tablet 953d41f26f5f4027acfd374c6ec4ee75 (DEFAULT_TABLE table=default.Table_9cdc814c021f49dea82875834691ba9e [id=e3fd45d69e384f9eb2ebce81bf195e5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.033105 19533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 953d41f26f5f4027acfd374c6ec4ee75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.036085 19534 tablet_service.cc:1511] Processing CreateTablet for tablet 5a9b3dfaa7474611a1824d63eb906b43 (DEFAULT_TABLE table=default.Table_67306a96449b4a7bbb4b96498a396cea [id=bcf9d7c5e4f44267b9204bc3c2d90584]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.036973 19534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a9b3dfaa7474611a1824d63eb906b43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.997884 20011 rpcz_store.cc:276] 0925 13:56:17.900385 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:17.900505 (+ 120us) service_pool.cc:224] Handling call
0925 13:56:18.997261 (+1096756us) ts_tablet_manager.cc:1151] Deleting tablet c161ae3edfed4f8488c770ecc6332607
0925 13:56:18.997303 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:18.997537 (+ 234us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1096224}
I20260925 13:56:19.046154 19671 tablet_service.cc:1511] Processing CreateTablet for tablet a8a679a453e0430b866edc707fe7c234 (DEFAULT_TABLE table=default.Table_50a86db358374d4ea5164480fefdfee5 [id=7556e78d915f446fb0f67f55fcaa2bed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.046386 19673 tablet_service.cc:1511] Processing CreateTablet for tablet 908df914cfec490cb7058dbe7b501c97 (DEFAULT_TABLE table=default.Table_c90b2c8024944acb9ffe76b506fff5a2 [id=d06e00bf26bd4c47a9e27f7eb35835df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.047161 19671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8a679a453e0430b866edc707fe7c234. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.047717 19673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 908df914cfec490cb7058dbe7b501c97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.055697 20011 tablet_replica.cc:333] T 4d3dbd9796564472b44a31ccaf56d898 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:19.056519 20011 raft_consensus.cc:2243] T 4d3dbd9796564472b44a31ccaf56d898 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:19.057000 20011 raft_consensus.cc:2272] T 4d3dbd9796564472b44a31ccaf56d898 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:19.058911 19959 raft_consensus.cc:2243] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus shutting down.
W20260925 13:56:19.063982 19490 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 82552.9us
I20260925 13:56:19.067535 19959 raft_consensus.cc:2272] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:19.068099 19801 tablet_service.cc:1511] Processing CreateTablet for tablet 321c740ba8eb4cbb8974305c5f8702af (DEFAULT_TABLE table=default.Table_d647554488064f3ea60ad9b397cabe30 [id=b889cefa13cb486f9fb18991d8b6a9ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.069229 19801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 321c740ba8eb4cbb8974305c5f8702af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:18.960407 19942 tablet_bootstrap.cc:492] T bbfc4f99a5964e2ba6146e390e8dba44 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:19.070751 19532 tablet_service.cc:1511] Processing CreateTablet for tablet 2711e7a6973546b1822d93f91095dae2 (DEFAULT_TABLE table=default.Table_83acf26db75e4c02b312cef4df19901f [id=7ec3d38b986747ca99d6a49e6a7dbe1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.072146 19532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2711e7a6973546b1822d93f91095dae2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.077224 19955 ts_tablet_manager.cc:1929] T 7292d04b411c415394e9d0df85aaf579 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:19.077617 19955 log.cc:1199] T 7292d04b411c415394e9d0df85aaf579 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/7292d04b411c415394e9d0df85aaf579
I20260925 13:56:19.078308 19955 ts_tablet_manager.cc:1950] T 7292d04b411c415394e9d0df85aaf579 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:19.079109 19672 tablet_service.cc:1511] Processing CreateTablet for tablet 0fdb45b31f93466c8bed1b848c6cdf15 (DEFAULT_TABLE table=default.Table_cd0b384bb04c44759472fcffcab11ee3 [id=9eab89f862ca4be88e4131feeb777864]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.080157 19672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fdb45b31f93466c8bed1b848c6cdf15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.080561 19819 tablet_service.cc:1511] Processing CreateTablet for tablet 5f8a96ae6eec4f15a221f9aad537c094 (DEFAULT_TABLE table=default.Table_3a06919b658e46f893318b4e1ec8b52f [id=e6f2a872536b4e55b071231020903581]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.081430 19819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f8a96ae6eec4f15a221f9aad537c094. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.088882 20011 ts_tablet_manager.cc:1916] T 4d3dbd9796564472b44a31ccaf56d898 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:19.093338 19806 tablet_service.cc:1558] Processing DeleteTablet for tablet 2a1f34dba13143b097a38bab3782c404 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:14 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:19.096091 19674 tablet_service.cc:1511] Processing CreateTablet for tablet e2e44b2ca25c4bbcb91288424b3e18b9 (DEFAULT_TABLE table=default.Table_261023ab15a845df82deea31f0eac8c8 [id=f03b9baa231c40999b38848e9059553e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.097496 19674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2e44b2ca25c4bbcb91288424b3e18b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.098621 19959 ts_tablet_manager.cc:1916] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:19.102713 19671 tablet_service.cc:1511] Processing CreateTablet for tablet 000aad39953c4688b543c7d882a08ec8 (DEFAULT_TABLE table=default.Table_3a3e48a0641a43a1a783c6ba6374a708 [id=e64fe44a237f48859a288a87a565ee5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.104936 19671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 000aad39953c4688b543c7d882a08ec8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.105034 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet dc14c37aa393407a87af0d0eb8b88d82 (table default.Table_3a06919b658e46f893318b4e1ec8b52f [id=e6f2a872536b4e55b071231020903581]) successfully deleted
I20260925 13:56:19.105921 19689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "137c43fb74ec4cc48d30ed3edac81d98" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:19.107990 19667 tablet_service.cc:1511] Processing CreateTablet for tablet 1d641f09429546d683f2884c02152f73 (DEFAULT_TABLE table=default.Table_b1985cfbe41c4240a1c15abfd5741f90 [id=6d63400819ed4ce395ab408cb58407b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.109241 19667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d641f09429546d683f2884c02152f73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.112515 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 84) took 3008 ms. Trace:
I20260925 13:56:19.112840 19955 rpcz_store.cc:276] 0925 13:56:16.104482 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:16.105104 (+ 622us) service_pool.cc:224] Handling call
0925 13:56:18.668518 (+2563414us) ts_tablet_manager.cc:1151] Deleting tablet 7292d04b411c415394e9d0df85aaf579
0925 13:56:18.668561 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:18.833696 (+165135us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:19.077095 (+243399us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:19.112454 (+ 35359us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2562228}
I20260925 13:56:19.113622 19818 tablet_service.cc:1511] Processing CreateTablet for tablet 31049bccd170436893512a7e9ef435cc (DEFAULT_TABLE table=default.Table_3751c59d6c7347db843dd4f021a6c620 [id=0213b487e92a47eb819dd3a868411f9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.059813 19813 tablet_service.cc:1511] Processing CreateTablet for tablet 82972fa4868d4c59b3b9bb1240300938 (DEFAULT_TABLE table=default.Table_43b8599ad301426687e4035cec94b286 [id=4232cecee9104120ac2521eb250bbd0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.121271 19773 leader_election.cc:304] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:19.123117 19938 tablet_bootstrap.cc:654] T 28219970809145bc9d4e2a3289a9479a P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
W20260925 13:56:19.126485 19773 raft_consensus.cc:2686] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260925 13:56:19.127142 19773 leader_election.cc:343] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:19.127631 20332 raft_consensus.cc:493] T a88bbe1ebee7468d8f49dfd25637fe83 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:19.127521 19773 leader_election.cc:304] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:19.128125 20332 raft_consensus.cc:515] T a88bbe1ebee7468d8f49dfd25637fe83 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:19.129282 19773 leader_election.cc:343] T 137c43fb74ec4cc48d30ed3edac81d98 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:19.131057 20332 leader_election.cc:290] T a88bbe1ebee7468d8f49dfd25637fe83 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:19.134289 19836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e59d6f4f6e4de1be1cf450620cfa59" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:19.134903 19836 raft_consensus.cc:2468] T e6e59d6f4f6e4de1be1cf450620cfa59 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:19.137367 20011 ts_tablet_manager.cc:1929] T 4d3dbd9796564472b44a31ccaf56d898 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:19.138124 20011 log.cc:1199] T 4d3dbd9796564472b44a31ccaf56d898 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/4d3dbd9796564472b44a31ccaf56d898
I20260925 13:56:19.139235 20011 ts_tablet_manager.cc:1950] T 4d3dbd9796564472b44a31ccaf56d898 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:19.141808 20011 tablet_replica.cc:333] T 8239d0cd3b0d40bdb721e807c0f403d3 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
W20260925 13:56:19.142328 19809 consensus_meta.cc:287] T f16050752ad44720ba8703ef10bbb359 P f790493ba7ad403993b678f2df2b55be: Time spent flushing consensus metadata: real 0.531s user 0.008s sys 0.000s
I20260925 13:56:19.142544 20011 raft_consensus.cc:2243] T 8239d0cd3b0d40bdb721e807c0f403d3 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:19.125726 19812 tablet_service.cc:1511] Processing CreateTablet for tablet 089dc4dfeb5243788dc03061c8743a81 (DEFAULT_TABLE table=default.Table_29949770026642b09db53126aa0acf63 [id=fa30ce5cdc804d5f9d9f77e15bdb9ed9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.143116 20011 raft_consensus.cc:2272] T 8239d0cd3b0d40bdb721e807c0f403d3 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus is shut down!
W20260925 13:56:19.144315 19808 consensus_meta.cc:287] T 9059655f2c3d4e86b3b389b5b9a68891 P f790493ba7ad403993b678f2df2b55be: Time spent flushing consensus metadata: real 0.523s user 0.008s sys 0.000s
I20260925 13:56:19.144940 20011 ts_tablet_manager.cc:1916] T 8239d0cd3b0d40bdb721e807c0f403d3 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:19.148713 19668 tablet_service.cc:1511] Processing CreateTablet for tablet 82972fa4868d4c59b3b9bb1240300938 (DEFAULT_TABLE table=default.Table_43b8599ad301426687e4035cec94b286 [id=4232cecee9104120ac2521eb250bbd0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.149699 19668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82972fa4868d4c59b3b9bb1240300938. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.149781 20297 raft_consensus.cc:2749] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:19.151831 19276 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 0de8f2d2766c4ff197b552a1b5ed3d6b
I20260925 13:56:19.171900 20332 raft_consensus.cc:493] T 5943795bc8b24bb48b80d6d06d17750b P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:19.172324 20332 raft_consensus.cc:515] T 5943795bc8b24bb48b80d6d06d17750b P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:19.174122 20332 leader_election.cc:290] T 5943795bc8b24bb48b80d6d06d17750b P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
W20260925 13:56:19.174813 19773 consensus_peers.cc:597] T b12a6c6cff024d70b53cc2c9169dd48b P f790493ba7ad403993b678f2df2b55be -> Peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Couldn't send request to peer 7c1b52895a14482fb54a6758f5054f8f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260925 13:56:19.175143 19773 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.194:38641, user_credentials={real_user=slave}} blocked reactor thread for 45073.8us
I20260925 13:56:19.175865 19631 leader_election.cc:304] T e6e59d6f4f6e4de1be1cf450620cfa59 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:19.176541 20332 raft_consensus.cc:2804] T e6e59d6f4f6e4de1be1cf450620cfa59 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:19.176846 20332 raft_consensus.cc:493] T e6e59d6f4f6e4de1be1cf450620cfa59 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:19.177157 20332 raft_consensus.cc:3060] T e6e59d6f4f6e4de1be1cf450620cfa59 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:19.178843 20011 ts_tablet_manager.cc:1929] T 8239d0cd3b0d40bdb721e807c0f403d3 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:19.179216 20011 log.cc:1199] T 8239d0cd3b0d40bdb721e807c0f403d3 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/8239d0cd3b0d40bdb721e807c0f403d3
I20260925 13:56:19.179863 20011 ts_tablet_manager.cc:1950] T 8239d0cd3b0d40bdb721e807c0f403d3 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:19.183985 20332 raft_consensus.cc:515] T e6e59d6f4f6e4de1be1cf450620cfa59 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:19.188990 19676 tablet_service.cc:1511] Processing CreateTablet for tablet f3b46e4fac694084b06b7086bcc596b9 (DEFAULT_TABLE table=default.Table_6f8114f6052845bb85b139bfedbed143 [id=c31f813d79a244afa726df0238376aba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.190088 19676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3b46e4fac694084b06b7086bcc596b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.197638 19669 tablet_service.cc:1511] Processing CreateTablet for tablet 321c740ba8eb4cbb8974305c5f8702af (DEFAULT_TABLE table=default.Table_d647554488064f3ea60ad9b397cabe30 [id=b889cefa13cb486f9fb18991d8b6a9ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.198908 19669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 321c740ba8eb4cbb8974305c5f8702af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.200871 20332 leader_election.cc:290] T e6e59d6f4f6e4de1be1cf450620cfa59 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:19.110886 19942 tablet_bootstrap.cc:654] T bbfc4f99a5964e2ba6146e390e8dba44 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:19.222934 19812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 089dc4dfeb5243788dc03061c8743a81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.226671 19810 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:51442 (request call id 110) took 1019 ms. Trace:
I20260925 13:56:19.226917 19811 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:51442 (request call id 109) took 1053 ms. Trace:
I20260925 13:56:19.241112 19668 tablet_service.cc:1511] Processing CreateTablet for tablet 089dc4dfeb5243788dc03061c8743a81 (DEFAULT_TABLE table=default.Table_29949770026642b09db53126aa0acf63 [id=fa30ce5cdc804d5f9d9f77e15bdb9ed9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.242311 19668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 089dc4dfeb5243788dc03061c8743a81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.242940 19942 tablet_bootstrap.cc:492] T bbfc4f99a5964e2ba6146e390e8dba44 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:19.228977 19818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31049bccd170436893512a7e9ef435cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.246409 19811 rpcz_store.cc:276] 0925 13:56:18.173998 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:18.212566 (+ 38568us) service_pool.cc:224] Handling call
0925 13:56:18.343197 (+130631us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0925 13:56:18.343279 (+ 82us) ts_tablet_manager.cc:682] Creating new metadata...
0925 13:56:18.660408 (+317129us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:19.178659 (+518251us) tablet_replica.cc:206] Creating consensus instance
0925 13:56:19.226828 (+ 48169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21939}
I20260925 13:56:19.251868 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet a88bbe1ebee7468d8f49dfd25637fe83 (table default.Table_65608937f2cd4986b2588a4831750601 [id=7079cc0c725b4578a1b11c7dd3ef9342]) successfully deleted
I20260925 13:56:19.253610 19276 master_service.cc:960] ReplaceTablet: received request to replace tablet 26644768f516412f8c95d68a76bb99c5 from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:19.253867 19273 master_service.cc:960] ReplaceTablet: received request to replace tablet 7ea4fd6817654bff9c0fe8c6e42ed6b9 from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:19.243558 19806 tablet_service.cc:1558] Processing DeleteTablet for tablet 0de8f2d2766c4ff197b552a1b5ed3d6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:18 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:19.269819 19813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82972fa4868d4c59b3b9bb1240300938. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.274030 19276 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 26644768f516412f8c95d68a76bb99c5
I20260925 13:56:19.275815 19809 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:51442 (request call id 111) took 1060 ms. Trace:
I20260925 13:56:19.276161 19809 rpcz_store.cc:276] 0925 13:56:18.207930 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:18.208122 (+ 192us) service_pool.cc:224] Handling call
0925 13:56:18.403001 (+194879us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0925 13:56:18.403071 (+ 70us) ts_tablet_manager.cc:682] Creating new metadata...
0925 13:56:18.563243 (+160172us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:19.182193 (+618950us) tablet_replica.cc:206] Creating consensus instance
0925 13:56:19.268385 (+ 86192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70569}
I20260925 13:56:19.235996 19832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e59d6f4f6e4de1be1cf450620cfa59" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:19.277464 19832 raft_consensus.cc:3060] T e6e59d6f4f6e4de1be1cf450620cfa59 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:19.226994 19810 rpcz_store.cc:276] 0925 13:56:18.207565 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:18.286924 (+ 79359us) service_pool.cc:224] Handling call
0925 13:56:18.414072 (+127148us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0925 13:56:18.414175 (+ 103us) ts_tablet_manager.cc:682] Creating new metadata...
0925 13:56:18.674229 (+260054us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:19.116051 (+441822us) tablet_replica.cc:206] Creating consensus instance
0925 13:56:19.226621 (+110570us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15173}
I20260925 13:56:19.277558 19276 catalog_manager.cc:6934] ReplaceTablet: tablet 26644768f516412f8c95d68a76bb99c5 deleted and replaced by tablet b3172d99afee4a4c878e71b645037917
W20260925 13:56:19.291522 19490 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 218350us
I20260925 13:56:19.293965 19276 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_74b6076b22674a31bc58fef71cc9bfb1" } modify_external_catalogs: true
I20260925 13:56:19.294507 19276 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_74b6076b22674a31bc58fef71cc9bfb1" } modify_external_catalogs: true
I20260925 13:56:19.299755 19273 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_f1d49a17b4c9418a9872e05be0d39488" } new_table_name: "default.Table_b2d4cd63d7124850a1070083a1c003c8" modify_external_catalogs: true
I20260925 13:56:19.308796 19832 raft_consensus.cc:2468] T e6e59d6f4f6e4de1be1cf450620cfa59 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
I20260925 13:56:19.326213 19521 tablet_service.cc:1511] Processing CreateTablet for tablet 1d641f09429546d683f2884c02152f73 (DEFAULT_TABLE table=default.Table_b1985cfbe41c4240a1c15abfd5741f90 [id=6d63400819ed4ce395ab408cb58407b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.327418 19521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d641f09429546d683f2884c02152f73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.332613 19524 tablet_service.cc:1511] Processing CreateTablet for tablet 0fdb45b31f93466c8bed1b848c6cdf15 (DEFAULT_TABLE table=default.Table_cd0b384bb04c44759472fcffcab11ee3 [id=9eab89f862ca4be88e4131feeb777864]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.334293 19524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fdb45b31f93466c8bed1b848c6cdf15. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:19.337427 19264 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 8dc5ea164d6749b0926c63d6bda9dcc5 (table default.Table_e0f7b807e8884431898298a1c635c784 [id=e49e015b173c48c1b6ef8112f60b33e4], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 8dc5ea164d6749b0926c63d6bda9dcc5 consensus state has no leader
I20260925 13:56:19.338986 19938 tablet_bootstrap.cc:492] T 28219970809145bc9d4e2a3289a9479a P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:19.339054 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 608bed5b04394bfca1776b5d6c502df1 (table default.Table_3b49fb893abb48ac8646283360e8e681 [id=9621507077d04649a58715b250fcef05]) successfully deleted
I20260925 13:56:19.339406 19938 ts_tablet_manager.cc:1403] T 28219970809145bc9d4e2a3289a9479a P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 1.026s user 0.024s sys 0.012s
W20260925 13:56:19.339668 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet c161ae3edfed4f8488c770ecc6332607 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c161ae3edfed4f8488c770ecc6332607 already in progress: creating tablet
W20260925 13:56:19.340204 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 7ea4fd6817654bff9c0fe8c6e42ed6b9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7ea4fd6817654bff9c0fe8c6e42ed6b9 already in progress: creating tablet
W20260925 13:56:19.340924 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet c2d333f892f84a258f8a148a90ce3b06 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c2d333f892f84a258f8a148a90ce3b06 already in progress: creating tablet
I20260925 13:56:19.337860 19804 tablet_service.cc:1558] Processing DeleteTablet for tablet 26644768f516412f8c95d68a76bb99c5 with delete_type TABLET_DATA_DELETED (replaced by tablet b3172d99afee4a4c878e71b645037917) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:19.342550 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 4d3dbd9796564472b44a31ccaf56d898 (table default.Table_1aa8069da6504812a85380a42ca07583 [id=a6d1215ba3be4fca903af9b17566b4a8]) successfully deleted
I20260925 13:56:19.346138 19276 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 1513c0048bd244028d3dfa8d118bc14e
I20260925 13:56:19.245997 19670 tablet_service.cc:1511] Processing CreateTablet for tablet 5f8a96ae6eec4f15a221f9aad537c094 (DEFAULT_TABLE table=default.Table_3a06919b658e46f893318b4e1ec8b52f [id=e6f2a872536b4e55b071231020903581]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.349815 19670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f8a96ae6eec4f15a221f9aad537c094. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:19.246232 19631 leader_election.cc:343] T 5943795bc8b24bb48b80d6d06d17750b P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 5943795bc8b24bb48b80d6d06d17750b
W20260925 13:56:19.351792 19631 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 105624us
W20260925 13:56:19.352591 19631 leader_election.cc:343] T a88bbe1ebee7468d8f49dfd25637fe83 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: a88bbe1ebee7468d8f49dfd25637fe83
I20260925 13:56:19.353132 19631 leader_election.cc:304] T e6e59d6f4f6e4de1be1cf450620cfa59 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:19.312440 19676 tablet_service.cc:1558] Processing DeleteTablet for tablet 1da6eadb1648479889027fa911f4676b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:17 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:19.317750 19674 tablet_service.cc:1558] Processing DeleteTablet for tablet 5943795bc8b24bb48b80d6d06d17750b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:15 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:19.325620 19667 tablet_service.cc:1558] Processing DeleteTablet for tablet 0de8f2d2766c4ff197b552a1b5ed3d6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:18 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:19.339574 19673 tablet_service.cc:1558] Processing DeleteTablet for tablet 26644768f516412f8c95d68a76bb99c5 with delete_type TABLET_DATA_DELETED (replaced by tablet b3172d99afee4a4c878e71b645037917) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:19.248961 19672 tablet_service.cc:1558] Processing DeleteTablet for tablet a88bbe1ebee7468d8f49dfd25637fe83 with delete_type TABLET_DATA_DELETED (replaced by tablet a26f5a02311c435a8c2434fd030c80a6) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:19.324575 19669 tablet_service.cc:1511] Processing CreateTablet for tablet 31049bccd170436893512a7e9ef435cc (DEFAULT_TABLE table=default.Table_3751c59d6c7347db843dd4f021a6c620 [id=0213b487e92a47eb819dd3a868411f9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.354728 20011 tablet_replica.cc:333] T 1da6eadb1648479889027fa911f4676b P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:19.355567 19801 tablet_service.cc:1558] Processing DeleteTablet for tablet 1513c0048bd244028d3dfa8d118bc14e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:19 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:19.356012 19669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31049bccd170436893512a7e9ef435cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.356160 20011 raft_consensus.cc:2243] T 1da6eadb1648479889027fa911f4676b P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:19.356560 20011 raft_consensus.cc:2272] T 1da6eadb1648479889027fa911f4676b P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:19.359455 19522 tablet_service.cc:1511] Processing CreateTablet for tablet 000aad39953c4688b543c7d882a08ec8 (DEFAULT_TABLE table=default.Table_3a3e48a0641a43a1a783c6ba6374a708 [id=e64fe44a237f48859a288a87a565ee5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.354697 20242 raft_consensus.cc:2804] T e6e59d6f4f6e4de1be1cf450620cfa59 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:19.360445 20242 raft_consensus.cc:697] T e6e59d6f4f6e4de1be1cf450620cfa59 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Becoming Leader. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Running, Role: LEADER
I20260925 13:56:19.361157 20242 consensus_queue.cc:237] T e6e59d6f4f6e4de1be1cf450620cfa59 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:19.363126 19955 tablet_replica.cc:333] T 5943795bc8b24bb48b80d6d06d17750b P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:19.365638 20011 ts_tablet_manager.cc:1916] T 1da6eadb1648479889027fa911f4676b P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:19.365411 19938 raft_consensus.cc:359] T 28219970809145bc9d4e2a3289a9479a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:19.366055 19938 raft_consensus.cc:385] T 28219970809145bc9d4e2a3289a9479a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:19.366298 19938 raft_consensus.cc:740] T 28219970809145bc9d4e2a3289a9479a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:19.371207 19959 ts_tablet_manager.cc:1929] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:19.371636 19959 log.cc:1199] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/47521fb74f9c40ee999a7f704aa551c9
I20260925 13:56:19.372414 19959 ts_tablet_manager.cc:1950] T 47521fb74f9c40ee999a7f704aa551c9 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:19.243351 19942 ts_tablet_manager.cc:1403] T bbfc4f99a5964e2ba6146e390e8dba44 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.283s user 0.009s sys 0.000s
I20260925 13:56:19.373098 19959 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 94) took 2556 ms. Trace:
I20260925 13:56:19.373406 19959 rpcz_store.cc:276] 0925 13:56:16.817163 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:16.818283 (+ 1120us) service_pool.cc:224] Handling call
0925 13:56:18.936228 (+2117945us) ts_tablet_manager.cc:1151] Deleting tablet 47521fb74f9c40ee999a7f704aa551c9
0925 13:56:18.936281 (+ 53us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:19.186023 (+249742us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:19.371097 (+185074us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:19.373053 (+ 1956us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":120408,"tablet-delete.queue_time_us":2117408}
I20260925 13:56:19.374537 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 47521fb74f9c40ee999a7f704aa551c9 (table default.Table_3a3e48a0641a43a1a783c6ba6374a708 [id=e64fe44a237f48859a288a87a565ee5e]) successfully deleted
W20260925 13:56:19.375952 19490 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 82597.7us
I20260925 13:56:19.375389 19277 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_e982ff965b73447b8fbdb7d0bb22532c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:19.380775 19277 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e982ff965b73447b8fbdb7d0bb22532c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:19.383991 19522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 000aad39953c4688b543c7d882a08ec8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.392870 19832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a1f34dba13143b097a38bab3782c404" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:19.393525 19832 raft_consensus.cc:3060] T 2a1f34dba13143b097a38bab3782c404 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:19.369017 19519 tablet_service.cc:1511] Processing CreateTablet for tablet 089dc4dfeb5243788dc03061c8743a81 (DEFAULT_TABLE table=default.Table_29949770026642b09db53126aa0acf63 [id=fa30ce5cdc804d5f9d9f77e15bdb9ed9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.403575 19533 tablet_service.cc:1511] Processing CreateTablet for tablet 5f8a96ae6eec4f15a221f9aad537c094 (DEFAULT_TABLE table=default.Table_3a06919b658e46f893318b4e1ec8b52f [id=e6f2a872536b4e55b071231020903581]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.378010 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 8239d0cd3b0d40bdb721e807c0f403d3 (table default.Table_374792fc278744f5ad6b9c1818b7a370 [id=7bbf083be05c424fadc6fc2a1df86758]) successfully deleted
I20260925 13:56:19.404493 19533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f8a96ae6eec4f15a221f9aad537c094. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.405381 19519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 089dc4dfeb5243788dc03061c8743a81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.406026 19938 consensus_queue.cc:260] T 28219970809145bc9d4e2a3289a9479a P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:19.409891 19938 ts_tablet_manager.cc:1434] T 28219970809145bc9d4e2a3289a9479a P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.070s user 0.004s sys 0.001s
I20260925 13:56:19.413938 19520 tablet_service.cc:1511] Processing CreateTablet for tablet 82972fa4868d4c59b3b9bb1240300938 (DEFAULT_TABLE table=default.Table_43b8599ad301426687e4035cec94b286 [id=4232cecee9104120ac2521eb250bbd0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.414754 20297 raft_consensus.cc:493] T 28219970809145bc9d4e2a3289a9479a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:19.415459 19520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82972fa4868d4c59b3b9bb1240300938. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.415143 20297 raft_consensus.cc:515] T 28219970809145bc9d4e2a3289a9479a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:19.417110 20297 leader_election.cc:290] T 28219970809145bc9d4e2a3289a9479a P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:19.416833 19273 catalog_manager.cc:5838] T e6e59d6f4f6e4de1be1cf450620cfa59 P 7c1b52895a14482fb54a6758f5054f8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194). New cstate: current_term: 1 leader_uuid: "7c1b52895a14482fb54a6758f5054f8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:19.418118 19273 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet e6e59d6f4f6e4de1be1cf450620cfa59 (table default.Table_546097946d054781b0acae5186c5d43d [id=912de4c02216461f8a69b880207d8652]). Expected version 1 got 0
I20260925 13:56:19.402833 19942 raft_consensus.cc:359] T bbfc4f99a5964e2ba6146e390e8dba44 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:19.424145 19526 tablet_service.cc:1511] Processing CreateTablet for tablet a8a679a453e0430b866edc707fe7c234 (DEFAULT_TABLE table=default.Table_50a86db358374d4ea5164480fefdfee5 [id=7556e78d915f446fb0f67f55fcaa2bed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.427497 19526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8a679a453e0430b866edc707fe7c234. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.428995 19672 tablet_service.cc:1558] Processing DeleteTablet for tablet 1513c0048bd244028d3dfa8d118bc14e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:19 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:19.429579 19689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28219970809145bc9d4e2a3289a9479a" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:19.429919 19942 raft_consensus.cc:385] T bbfc4f99a5964e2ba6146e390e8dba44 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:19.435915 19959 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 95) took 2552 ms. Trace:
I20260925 13:56:19.436297 19959 rpcz_store.cc:276] 0925 13:56:16.884042 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:16.884189 (+ 147us) service_pool.cc:224] Handling call
0925 13:56:19.435590 (+2551401us) ts_tablet_manager.cc:1151] Deleting tablet 7292d04b411c415394e9d0df85aaf579
0925 13:56:19.435644 (+ 54us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:19.435858 (+ 214us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2550766}
I20260925 13:56:19.437340 19959 tablet_replica.cc:333] T f514fa52e7524cd3ba07f3f65de532d4 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:19.430577 19942 raft_consensus.cc:740] T bbfc4f99a5964e2ba6146e390e8dba44 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:19.438035 19959 raft_consensus.cc:2243] T f514fa52e7524cd3ba07f3f65de532d4 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:19.438469 19959 raft_consensus.cc:2272] T f514fa52e7524cd3ba07f3f65de532d4 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:19.440587 19518 tablet_service.cc:1511] Processing CreateTablet for tablet f3b46e4fac694084b06b7086bcc596b9 (DEFAULT_TABLE table=default.Table_6f8114f6052845bb85b139bfedbed143 [id=c31f813d79a244afa726df0238376aba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.443048 19942 consensus_queue.cc:260] T bbfc4f99a5964e2ba6146e390e8dba44 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:19.443974 19518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3b46e4fac694084b06b7086bcc596b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.446380 19531 tablet_service.cc:1511] Processing CreateTablet for tablet 31049bccd170436893512a7e9ef435cc (DEFAULT_TABLE table=default.Table_3751c59d6c7347db843dd4f021a6c620 [id=0213b487e92a47eb819dd3a868411f9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.447333 19531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31049bccd170436893512a7e9ef435cc. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:19.447960 19773 leader_election.cc:343] T 28219970809145bc9d4e2a3289a9479a P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:19.452706 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet 7292d04b411c415394e9d0df85aaf579 (table default.Table_547f995b454c497bb5f8d7241ab26876 [id=d2bc135460ab4f2fad4740a281bfc6e2]) successfully deleted
I20260925 13:56:19.456115 19534 tablet_service.cc:1558] Processing DeleteTablet for tablet 0de8f2d2766c4ff197b552a1b5ed3d6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:18 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:19.456319 19279 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_f1564f324ef645f9a62c24fb5d1ebd06"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:19.456660 19534 tablet_service.cc:1558] Processing DeleteTablet for tablet 26644768f516412f8c95d68a76bb99c5 with delete_type TABLET_DATA_DELETED (replaced by tablet b3172d99afee4a4c878e71b645037917) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:19.457228 19534 tablet_service.cc:1558] Processing DeleteTablet for tablet 1513c0048bd244028d3dfa8d118bc14e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:19 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:19.457870 19517 tablet_service.cc:1558] Processing DeleteTablet for tablet 2a1f34dba13143b097a38bab3782c404 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:14 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:19.438649 19523 tablet_service.cc:1511] Processing CreateTablet for tablet e2e44b2ca25c4bbcb91288424b3e18b9 (DEFAULT_TABLE table=default.Table_261023ab15a845df82deea31f0eac8c8 [id=f03b9baa231c40999b38848e9059553e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.458693 19942 ts_tablet_manager.cc:1434] T bbfc4f99a5964e2ba6146e390e8dba44 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.086s user 0.005s sys 0.000s
I20260925 13:56:19.459098 19523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2e44b2ca25c4bbcb91288424b3e18b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.459517 19942 tablet_bootstrap.cc:492] T c2d333f892f84a258f8a148a90ce3b06 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:19.462215 19938 tablet_bootstrap.cc:492] T 0de8f2d2766c4ff197b552a1b5ed3d6b P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
W20260925 13:56:19.466432 19527 consensus_meta.cc:287] T 1215d875209a40d38cc527e2795d6c75 P 8f578869bc084a5d8a9a69f143954eb3: Time spent flushing consensus metadata: real 0.510s user 0.007s sys 0.000s
I20260925 13:56:19.473829 19959 ts_tablet_manager.cc:1916] T f514fa52e7524cd3ba07f3f65de532d4 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 13:56:19.475421 19266 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.182.252:36181: connect: Connection refused (error 111) [suppressed 6 similar messages]
I20260925 13:56:19.441228 19525 tablet_service.cc:1511] Processing CreateTablet for tablet 908df914cfec490cb7058dbe7b501c97 (DEFAULT_TABLE table=default.Table_c90b2c8024944acb9ffe76b506fff5a2 [id=d06e00bf26bd4c47a9e27f7eb35835df]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 13:56:19.489181 19279 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f1564f324ef645f9a62c24fb5d1ebd06 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:19.445120 19529 tablet_service.cc:1511] Processing CreateTablet for tablet 321c740ba8eb4cbb8974305c5f8702af (DEFAULT_TABLE table=default.Table_d647554488064f3ea60ad9b397cabe30 [id=b889cefa13cb486f9fb18991d8b6a9ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.503311 19938 tablet_bootstrap.cc:654] T 0de8f2d2766c4ff197b552a1b5ed3d6b P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:19.499394 19955 raft_consensus.cc:2243] T 5943795bc8b24bb48b80d6d06d17750b P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:19.504925 19955 raft_consensus.cc:2272] T 5943795bc8b24bb48b80d6d06d17750b P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:19.506546 19955 ts_tablet_manager.cc:1916] T 5943795bc8b24bb48b80d6d06d17750b P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:19.504096 19525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 908df914cfec490cb7058dbe7b501c97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.507898 20339 consensus_queue.cc:1048] T c65a8cd3060c405c871270543b4623f7 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260925 13:56:19.518371 19529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 321c740ba8eb4cbb8974305c5f8702af. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:19.536784 19266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.252:36181, user_credentials={real_user=slave}} blocked reactor thread for 61436.4us
W20260925 13:56:19.538004 19266 catalog_manager.cc:5030] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): delete failed for tablet 7292d04b411c415394e9d0df85aaf579 because the tablet was not found. No further retry: Not found: Tablet not found: 7292d04b411c415394e9d0df85aaf579
I20260925 13:56:19.542747 19938 tablet_bootstrap.cc:492] T 0de8f2d2766c4ff197b552a1b5ed3d6b P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:19.543128 19938 ts_tablet_manager.cc:1403] T 0de8f2d2766c4ff197b552a1b5ed3d6b P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.081s user 0.007s sys 0.004s
I20260925 13:56:19.545605 19938 raft_consensus.cc:359] T 0de8f2d2766c4ff197b552a1b5ed3d6b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:19.546262 19938 raft_consensus.cc:385] T 0de8f2d2766c4ff197b552a1b5ed3d6b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:19.546520 19938 raft_consensus.cc:740] T 0de8f2d2766c4ff197b552a1b5ed3d6b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:19.551760 19959 ts_tablet_manager.cc:1929] T f514fa52e7524cd3ba07f3f65de532d4 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:19.552088 19959 log.cc:1199] T f514fa52e7524cd3ba07f3f65de532d4 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/f514fa52e7524cd3ba07f3f65de532d4
I20260925 13:56:19.552652 19959 ts_tablet_manager.cc:1950] T f514fa52e7524cd3ba07f3f65de532d4 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:19.554325 19959 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 96) took 2396 ms. Trace:
I20260925 13:56:19.554605 19959 rpcz_store.cc:276] 0925 13:56:17.157843 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:17.157985 (+ 142us) service_pool.cc:224] Handling call
0925 13:56:19.437195 (+2279210us) ts_tablet_manager.cc:1151] Deleting tablet f514fa52e7524cd3ba07f3f65de532d4
0925 13:56:19.437240 (+ 45us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:19.512290 (+ 75050us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:19.535249 (+ 22959us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:19.554271 (+ 19022us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2278610}
I20260925 13:56:19.551575 19938 consensus_queue.cc:260] T 0de8f2d2766c4ff197b552a1b5ed3d6b P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:19.560298 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet f514fa52e7524cd3ba07f3f65de532d4 (table default.Table_ba97a54a19744b058cffd5638ffb97db [id=7e7ee975d4bb457eb3cc4632e12f3cd5]) successfully deleted
W20260925 13:56:19.566037 19530 consensus_meta.cc:287] T f50ea68e3b3c4459b3f5e5b0df724724 P 8f578869bc084a5d8a9a69f143954eb3: Time spent flushing consensus metadata: real 0.668s user 0.009s sys 0.003s
I20260925 13:56:19.573889 19938 ts_tablet_manager.cc:1434] T 0de8f2d2766c4ff197b552a1b5ed3d6b P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.030s user 0.004s sys 0.001s
I20260925 13:56:19.574862 19277 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_07d4243785b14434b731eb80cf4f242c" } new_table_name: "default.Table_1da8b04f68ab4682a8ec32ad47854930" modify_external_catalogs: true
I20260925 13:56:19.585099 19938 tablet_bootstrap.cc:492] T 436b8d46e1d244deb8bb9dc128ee6279 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:19.587811 19942 tablet_bootstrap.cc:654] T c2d333f892f84a258f8a148a90ce3b06 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:19.589759 20242 raft_consensus.cc:493] T bbfc4f99a5964e2ba6146e390e8dba44 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:19.590147 20242 raft_consensus.cc:515] T bbfc4f99a5964e2ba6146e390e8dba44 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:19.591831 20242 leader_election.cc:290] T bbfc4f99a5964e2ba6146e390e8dba44 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:19.593480 19838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfc4f99a5964e2ba6146e390e8dba44" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:19.601261 19959 tablet_replica.cc:333] T 1da6eadb1648479889027fa911f4676b P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:19.602006 19959 raft_consensus.cc:2243] T 1da6eadb1648479889027fa911f4676b P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:19.617276 19942 tablet_bootstrap.cc:492] T c2d333f892f84a258f8a148a90ce3b06 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:19.617746 19942 ts_tablet_manager.cc:1403] T c2d333f892f84a258f8a148a90ce3b06 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.158s user 0.009s sys 0.000s
I20260925 13:56:19.618491 19276 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet bae8c5b6e1014731901bc3f336104784 (table default.Table_99593e98f3394811b4c91e234185cbef [id=0736e722eb8c43e9b6a151a7c2880fe3]). Expected version 1 got 0
I20260925 13:56:19.619182 19276 catalog_manager.cc:5838] T b12a6c6cff024d70b53cc2c9169dd48b P f790493ba7ad403993b678f2df2b55be reported cstate change: term changed from 0 to 1, leader changed from <none> to f790493ba7ad403993b678f2df2b55be (127.18.182.195). New cstate: current_term: 1 leader_uuid: "f790493ba7ad403993b678f2df2b55be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: HEALTHY } } }
I20260925 13:56:19.620637 19276 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 137c43fb74ec4cc48d30ed3edac81d98 (table default.Table_5321051e9f6e4cd28b9b77619c908dc9 [id=9dac142084c948f9a50cfc9acf24d461]). Expected version 2 got 0
I20260925 13:56:19.621178 19276 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet b36aebfe984f454ea910e84645c4ac7d (table default.Table_b2d4cd63d7124850a1070083a1c003c8 [id=e2b98c2d1dbf4efea5fa61f182fc2fd4]). Expected version 1 got 0
I20260925 13:56:19.621701 19276 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet f1c6e2137700448a8ebf859a9ca9085d (table default.Table_4a822937f96346c1bd348b842fe60dfc [id=5bf634ef72d243b1bfcafc4efe14f28a]). Expected version 1 got 0
I20260925 13:56:19.608318 20318 tablet.cc:1751] T b36aebfe984f454ea910e84645c4ac7d P f790493ba7ad403993b678f2df2b55be: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260925 13:56:19.634114 19277 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f16050752ad44720ba8703ef10bbb359 consensus state has no leader
I20260925 13:56:19.654873 20011 ts_tablet_manager.cc:1929] T 1da6eadb1648479889027fa911f4676b P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:19.657876 19279 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_977caf871df4415b9855949376feed24"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:19.659881 19279 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_977caf871df4415b9855949376feed24 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:19.673231 19272 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_5d692242213f4df6863089b93b4e9636" } modify_external_catalogs: true
I20260925 13:56:19.673729 19272 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_5d692242213f4df6863089b93b4e9636" } modify_external_catalogs: true
I20260925 13:56:19.674015 19273 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_65608937f2cd4986b2588a4831750601" } modify_external_catalogs: true
I20260925 13:56:19.674448 19273 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_65608937f2cd4986b2588a4831750601" } modify_external_catalogs: true
I20260925 13:56:19.674516 19275 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_547f995b454c497bb5f8d7241ab26876" } new_table_name: "default.Table_c389f408e051410dac7a521879c22729" modify_external_catalogs: true
I20260925 13:56:19.678771 19277 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_8aa1581769ff4b9bb2e158a3b0430a4f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:19.681090 20356 tablet.cc:1751] T b36aebfe984f454ea910e84645c4ac7d P 7c1b52895a14482fb54a6758f5054f8f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260925 13:56:19.683648 19277 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8aa1581769ff4b9bb2e158a3b0430a4f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:19.686702 19670 tablet_service.cc:1511] Processing CreateTablet for tablet e9dbcd85dc8647c68c120ba3fcfe1790 (DEFAULT_TABLE table=default.Table_93aa193a56f246aeaa089b2bfc0e9d8c [id=443536a0bd4d4d568d52f6ffe3d06ecb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.687794 19670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9dbcd85dc8647c68c120ba3fcfe1790. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.704758 20341 raft_consensus.cc:493] T f4429db544534f758883f2fae184ea6d P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f790493ba7ad403993b678f2df2b55be)
I20260925 13:56:19.708263 19942 raft_consensus.cc:359] T c2d333f892f84a258f8a148a90ce3b06 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:19.710778 19942 raft_consensus.cc:385] T c2d333f892f84a258f8a148a90ce3b06 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:19.712400 19942 raft_consensus.cc:740] T c2d333f892f84a258f8a148a90ce3b06 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:19.712953 20011 log.cc:1199] T 1da6eadb1648479889027fa911f4676b P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/1da6eadb1648479889027fa911f4676b
I20260925 13:56:19.714006 20011 ts_tablet_manager.cc:1950] T 1da6eadb1648479889027fa911f4676b P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:19.718585 19533 tablet_service.cc:1511] Processing CreateTablet for tablet e9dbcd85dc8647c68c120ba3fcfe1790 (DEFAULT_TABLE table=default.Table_93aa193a56f246aeaa089b2bfc0e9d8c [id=443536a0bd4d4d568d52f6ffe3d06ecb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.719341 19668 tablet_service.cc:1511] Processing CreateTablet for tablet eff33a56361946edb556ac611d2299eb (DEFAULT_TABLE table=default.Table_6f3fcafe361344c6920c158efa979438 [id=2c74fb31a35541c4a2dd0b2171ba41ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.721092 19668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eff33a56361946edb556ac611d2299eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.733384 19939 tablet_bootstrap.cc:492] T e6e59d6f4f6e4de1be1cf450620cfa59 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:19.733799 19939 ts_tablet_manager.cc:1403] T e6e59d6f4f6e4de1be1cf450620cfa59 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.842s user 0.002s sys 0.017s
I20260925 13:56:19.738970 19955 ts_tablet_manager.cc:1929] T 5943795bc8b24bb48b80d6d06d17750b P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260925 13:56:19.703234 19488 consensus_peers.cc:597] T 4d3dbd9796564472b44a31ccaf56d898 P 8f578869bc084a5d8a9a69f143954eb3 -> Peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Couldn't send request to peer 7c1b52895a14482fb54a6758f5054f8f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4d3dbd9796564472b44a31ccaf56d898. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:19.742954 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28219970809145bc9d4e2a3289a9479a" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:19.744711 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "137c43fb74ec4cc48d30ed3edac81d98" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:19.745157 19533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9dbcd85dc8647c68c120ba3fcfe1790. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.623322 19938 tablet_bootstrap.cc:654] T 436b8d46e1d244deb8bb9dc128ee6279 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:19.644130 19812 tablet_service.cc:1558] Processing DeleteTablet for tablet dc14c37aa393407a87af0d0eb8b88d82 with delete_type TABLET_DATA_DELETED (replaced by tablet 5f8a96ae6eec4f15a221f9aad537c094) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:19.674033 19819 tablet_service.cc:1511] Processing CreateTablet for tablet 4d01a791445149e7bee9493e1eec5c6b (DEFAULT_TABLE table=default.Table_883c781744824c2a9e6e86917bf2143e [id=4e1702861c25473e94c5ce639f3ab397]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.757876 19959 raft_consensus.cc:2272] T 1da6eadb1648479889027fa911f4676b P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:19.760144 19959 ts_tablet_manager.cc:1916] T 1da6eadb1648479889027fa911f4676b P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:19.760501 19532 tablet_service.cc:1511] Processing CreateTablet for tablet eff33a56361946edb556ac611d2299eb (DEFAULT_TABLE table=default.Table_6f3fcafe361344c6920c158efa979438 [id=2c74fb31a35541c4a2dd0b2171ba41ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.761520 19532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eff33a56361946edb556ac611d2299eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.673171 19813 tablet_service.cc:1511] Processing CreateTablet for tablet eff33a56361946edb556ac611d2299eb (DEFAULT_TABLE table=default.Table_6f3fcafe361344c6920c158efa979438 [id=2c74fb31a35541c4a2dd0b2171ba41ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.632625 19818 tablet_service.cc:1511] Processing CreateTablet for tablet e9dbcd85dc8647c68c120ba3fcfe1790 (DEFAULT_TABLE table=default.Table_93aa193a56f246aeaa089b2bfc0e9d8c [id=443536a0bd4d4d568d52f6ffe3d06ecb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.705252 20341 raft_consensus.cc:515] T f4429db544534f758883f2fae184ea6d P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:19.768385 19942 consensus_queue.cc:260] T c2d333f892f84a258f8a148a90ce3b06 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:19.780750 19819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d01a791445149e7bee9493e1eec5c6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.782632 20341 leader_election.cc:290] T f4429db544534f758883f2fae184ea6d P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
W20260925 13:56:19.756098 19488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.194:38641, user_credentials={real_user=slave}} blocked reactor thread for 61087.7us
I20260925 13:56:19.784257 19818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9dbcd85dc8647c68c120ba3fcfe1790. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.787875 19813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eff33a56361946edb556ac611d2299eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.789436 19959 ts_tablet_manager.cc:1929] T 1da6eadb1648479889027fa911f4676b P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:19.789739 19959 log.cc:1199] T 1da6eadb1648479889027fa911f4676b P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/1da6eadb1648479889027fa911f4676b
I20260925 13:56:19.790388 19959 ts_tablet_manager.cc:1950] T 1da6eadb1648479889027fa911f4676b P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:19.792451 19959 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 97) took 1966 ms. Trace:
I20260925 13:56:19.792783 19959 rpcz_store.cc:276] 0925 13:56:17.850111 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:17.852284 (+ 2173us) service_pool.cc:224] Handling call
0925 13:56:19.601123 (+1748839us) ts_tablet_manager.cc:1151] Deleting tablet 1da6eadb1648479889027fa911f4676b
0925 13:56:19.601163 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:19.782768 (+181605us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:19.789340 (+ 6572us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:19.792391 (+ 3051us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1748006}
I20260925 13:56:19.794282 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 1da6eadb1648479889027fa911f4676b (table default.Table_d3801e388b7b4b4d95188dc60386cce9 [id=479b481f813f4a60b2700734d4f51da3]) successfully deleted
I20260925 13:56:19.795471 19959 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 98) took 1900 ms. Trace:
I20260925 13:56:19.795804 19959 rpcz_store.cc:276] 0925 13:56:17.894608 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:17.894700 (+ 92us) service_pool.cc:224] Handling call
0925 13:56:19.795075 (+1900375us) ts_tablet_manager.cc:1151] Deleting tablet c161ae3edfed4f8488c770ecc6332607
0925 13:56:19.795121 (+ 46us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:19.795414 (+ 293us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1899736}
W20260925 13:56:19.797066 19266 catalog_manager.cc:5043] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): delete failed for tablet c161ae3edfed4f8488c770ecc6332607 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c161ae3edfed4f8488c770ecc6332607 already in progress: creating tablet
I20260925 13:56:19.797880 19959 tablet_replica.cc:333] T c2d333f892f84a258f8a148a90ce3b06 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:19.798615 19959 raft_consensus.cc:2243] T c2d333f892f84a258f8a148a90ce3b06 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:19.799069 19959 raft_consensus.cc:2272] T c2d333f892f84a258f8a148a90ce3b06 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:19.801152 19959 ts_tablet_manager.cc:1916] T c2d333f892f84a258f8a148a90ce3b06 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:19.814812 19959 ts_tablet_manager.cc:1929] T c2d333f892f84a258f8a148a90ce3b06 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:19.815333 19959 log.cc:1199] T c2d333f892f84a258f8a148a90ce3b06 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/c2d333f892f84a258f8a148a90ce3b06
I20260925 13:56:19.816015 19959 ts_tablet_manager.cc:1950] T c2d333f892f84a258f8a148a90ce3b06 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:19.816823 19533 tablet_service.cc:1511] Processing CreateTablet for tablet 4d01a791445149e7bee9493e1eec5c6b (DEFAULT_TABLE table=default.Table_883c781744824c2a9e6e86917bf2143e [id=4e1702861c25473e94c5ce639f3ab397]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.817767 19533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d01a791445149e7bee9493e1eec5c6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.817938 19959 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 112) took 1609 ms. Trace:
I20260925 13:56:19.817939 19274 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_98879cfd535843ff9cbc54221b3cf028"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:19.818224 19959 rpcz_store.cc:276] 0925 13:56:18.208253 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:18.295707 (+ 87454us) service_pool.cc:224] Handling call
0925 13:56:19.797758 (+1502051us) ts_tablet_manager.cc:1151] Deleting tablet c2d333f892f84a258f8a148a90ce3b06
0925 13:56:19.797793 (+ 35us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:19.808098 (+ 10305us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:19.814713 (+ 6615us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:19.817891 (+ 3178us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1498576}
W20260925 13:56:19.819475 19274 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_98879cfd535843ff9cbc54221b3cf028 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:19.819815 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet c2d333f892f84a258f8a148a90ce3b06 (table default.Table_c90b2c8024944acb9ffe76b506fff5a2 [id=d06e00bf26bd4c47a9e27f7eb35835df]) successfully deleted
I20260925 13:56:19.820701 19959 tablet_replica.cc:333] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:19.820343 19939 raft_consensus.cc:359] T e6e59d6f4f6e4de1be1cf450620cfa59 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:19.821079 19939 raft_consensus.cc:385] T e6e59d6f4f6e4de1be1cf450620cfa59 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:19.821408 19959 raft_consensus.cc:2243] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:19.821133 19278 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_271e58fa014d4b7f971f2b784b87fce6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:19.821861 19959 raft_consensus.cc:2272] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:19.821337 19939 raft_consensus.cc:740] T e6e59d6f4f6e4de1be1cf450620cfa59 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
W20260925 13:56:19.822604 19278 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_271e58fa014d4b7f971f2b784b87fce6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:19.823638 19959 ts_tablet_manager.cc:1916] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:19.825376 19939 consensus_queue.cc:260] T e6e59d6f4f6e4de1be1cf450620cfa59 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:19.838536 19959 ts_tablet_manager.cc:1929] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:19.838868 19959 log.cc:1199] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/7ea4fd6817654bff9c0fe8c6e42ed6b9
I20260925 13:56:19.839542 19959 ts_tablet_manager.cc:1950] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:19.841385 19959 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 113) took 1552 ms. Trace:
I20260925 13:56:19.841703 19959 rpcz_store.cc:276] 0925 13:56:18.288667 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:18.299305 (+ 10638us) service_pool.cc:224] Handling call
0925 13:56:19.820569 (+1521264us) ts_tablet_manager.cc:1151] Deleting tablet 7ea4fd6817654bff9c0fe8c6e42ed6b9
0925 13:56:19.820606 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:19.831030 (+ 10424us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:19.838432 (+ 7402us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:19.841327 (+ 2895us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1472313}
I20260925 13:56:19.843142 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 7ea4fd6817654bff9c0fe8c6e42ed6b9 (table default.Table_d647554488064f3ea60ad9b397cabe30 [id=b889cefa13cb486f9fb18991d8b6a9ba]) successfully deleted
I20260925 13:56:19.843983 19959 tablet_replica.cc:333] T 2a1f34dba13143b097a38bab3782c404 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:19.739333 19955 log.cc:1199] T 5943795bc8b24bb48b80d6d06d17750b P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/5943795bc8b24bb48b80d6d06d17750b
I20260925 13:56:19.853436 19955 ts_tablet_manager.cc:1950] T 5943795bc8b24bb48b80d6d06d17750b P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:19.854195 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 85) took 3702 ms. Trace:
I20260925 13:56:19.854588 19955 rpcz_store.cc:276] 0925 13:56:16.151740 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:16.167554 (+ 15814us) service_pool.cc:224] Handling call
0925 13:56:19.362989 (+3195435us) ts_tablet_manager.cc:1151] Deleting tablet 5943795bc8b24bb48b80d6d06d17750b
0925 13:56:19.363029 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:19.711129 (+348100us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:19.738861 (+ 27732us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:19.854142 (+115281us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":135486,"tablet-delete.queue_time_us":3194863}
I20260925 13:56:19.858186 19955 tablet_replica.cc:333] T f4429db544534f758883f2fae184ea6d P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:19.858894 19955 raft_consensus.cc:2243] T f4429db544534f758883f2fae184ea6d P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:19.859336 19955 raft_consensus.cc:2272] T f4429db544534f758883f2fae184ea6d P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:19.861124 19938 tablet_bootstrap.cc:492] T 436b8d46e1d244deb8bb9dc128ee6279 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:19.861737 19938 ts_tablet_manager.cc:1403] T 436b8d46e1d244deb8bb9dc128ee6279 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.281s user 0.015s sys 0.002s
I20260925 13:56:19.867417 19938 raft_consensus.cc:359] T 436b8d46e1d244deb8bb9dc128ee6279 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:19.868875 19938 raft_consensus.cc:385] T 436b8d46e1d244deb8bb9dc128ee6279 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:19.869411 19938 raft_consensus.cc:740] T 436b8d46e1d244deb8bb9dc128ee6279 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:19.870755 19938 consensus_queue.cc:260] T 436b8d46e1d244deb8bb9dc128ee6279 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:19.885099 19955 ts_tablet_manager.cc:1916] T f4429db544534f758883f2fae184ea6d P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:19.666923 19832 raft_consensus.cc:2468] T 2a1f34dba13143b097a38bab3782c404 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:19.891855 19959 raft_consensus.cc:2243] T 2a1f34dba13143b097a38bab3782c404 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:19.892270 19939 ts_tablet_manager.cc:1434] T e6e59d6f4f6e4de1be1cf450620cfa59 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.158s user 0.004s sys 0.003s
I20260925 13:56:19.893146 19939 tablet_bootstrap.cc:492] T 3e6c8b9bbf4549b3a8685f886973ed9c P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:19.893648 19959 raft_consensus.cc:2272] T 2a1f34dba13143b097a38bab3782c404 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:19.895241 19959 ts_tablet_manager.cc:1916] T 2a1f34dba13143b097a38bab3782c404 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:19.903430 19553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e59d6f4f6e4de1be1cf450620cfa59" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:19.904042 19553 raft_consensus.cc:2468] T e6e59d6f4f6e4de1be1cf450620cfa59 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:19.905992 19552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6e59d6f4f6e4de1be1cf450620cfa59" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:19.907395 19552 raft_consensus.cc:3060] T e6e59d6f4f6e4de1be1cf450620cfa59 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
W20260925 13:56:19.914211 19773 leader_election.cc:343] T 28219970809145bc9d4e2a3289a9479a P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:19.914616 19773 leader_election.cc:304] T 28219970809145bc9d4e2a3289a9479a P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
W20260925 13:56:19.915570 19773 leader_election.cc:343] T 137c43fb74ec4cc48d30ed3edac81d98 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:19.915990 19773 leader_election.cc:304] T 137c43fb74ec4cc48d30ed3edac81d98 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:19.916998 20297 raft_consensus.cc:2749] T 28219970809145bc9d4e2a3289a9479a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:19.917625 20297 raft_consensus.cc:2749] T 137c43fb74ec4cc48d30ed3edac81d98 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:19.929677 19942 ts_tablet_manager.cc:1434] T c2d333f892f84a258f8a148a90ce3b06 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.312s user 0.006s sys 0.000s
I20260925 13:56:19.929884 19669 tablet_service.cc:1511] Processing CreateTablet for tablet 4d01a791445149e7bee9493e1eec5c6b (DEFAULT_TABLE table=default.Table_883c781744824c2a9e6e86917bf2143e [id=4e1702861c25473e94c5ce639f3ab397]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:19.930474 19942 tablet_bootstrap.cc:492] T 1832cff5eb3e4da4b9766ae1591df81a P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:19.930847 19669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d01a791445149e7bee9493e1eec5c6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:19.947638 19959 ts_tablet_manager.cc:1929] T 2a1f34dba13143b097a38bab3782c404 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:19.948035 19959 log.cc:1199] T 2a1f34dba13143b097a38bab3782c404 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/2a1f34dba13143b097a38bab3782c404
I20260925 13:56:19.948676 19959 ts_tablet_manager.cc:1950] T 2a1f34dba13143b097a38bab3782c404 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:19.961607 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet 5943795bc8b24bb48b80d6d06d17750b (table default.Table_36b4dd2097aa4e2dbd58d98277e90281 [id=90bd44e8847549f2964a32f926ef1d99]) successfully deleted
I20260925 13:56:19.985986 19959 tablet_replica.cc:333] T 0de8f2d2766c4ff197b552a1b5ed3d6b P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:19.967767 19272 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet bbfc4f99a5964e2ba6146e390e8dba44
W20260925 13:56:19.994009 19490 consensus_peers.cc:597] T 1da6eadb1648479889027fa911f4676b P 8f578869bc084a5d8a9a69f143954eb3 -> Peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Couldn't send request to peer f790493ba7ad403993b678f2df2b55be. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:19.968442 19273 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet a26f5a02311c435a8c2434fd030c80a6
I20260925 13:56:19.998152 20011 tablet_replica.cc:333] T 5943795bc8b24bb48b80d6d06d17750b P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:19.999044 20011 raft_consensus.cc:2243] T 5943795bc8b24bb48b80d6d06d17750b P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:19.999531 20011 raft_consensus.cc:2272] T 5943795bc8b24bb48b80d6d06d17750b P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus is shut down!
W20260925 13:56:19.979682 19275 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet eb794cef052b43da831531455e2b5f15 consensus state has no leader
I20260925 13:56:20.003496 19672 tablet_service.cc:1558] Processing DeleteTablet for tablet bbfc4f99a5964e2ba6146e390e8dba44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:19 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:20.010119 19955 ts_tablet_manager.cc:1929] T f4429db544534f758883f2fae184ea6d P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:20.011068 19959 raft_consensus.cc:2243] T 0de8f2d2766c4ff197b552a1b5ed3d6b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:20.019616 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 1da6eadb1648479889027fa911f4676b (table default.Table_d3801e388b7b4b4d95188dc60386cce9 [id=479b481f813f4a60b2700734d4f51da3]) successfully deleted
I20260925 13:56:20.020612 19959 raft_consensus.cc:2272] T 0de8f2d2766c4ff197b552a1b5ed3d6b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:20.025777 19275 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 436b8d46e1d244deb8bb9dc128ee6279 (table default.Table_079723ff92144ba293733e54ab25d782 [id=4b0bfb9726854123864f8fe743a16a33]). Expected version 1 got 0
I20260925 13:56:20.028299 19672 tablet_service.cc:1558] Processing DeleteTablet for tablet a26f5a02311c435a8c2434fd030c80a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:19 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:20.039343 19959 ts_tablet_manager.cc:1916] T 0de8f2d2766c4ff197b552a1b5ed3d6b P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:20.020681 20339 raft_consensus.cc:493] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:20.041975 19552 raft_consensus.cc:2468] T e6e59d6f4f6e4de1be1cf450620cfa59 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
I20260925 13:56:20.045069 19690 raft_consensus.cc:3060] T c2d333f892f84a258f8a148a90ce3b06 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:20.044410 20339 raft_consensus.cc:515] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.064378 20011 ts_tablet_manager.cc:1916] T 5943795bc8b24bb48b80d6d06d17750b P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:20.072606 19942 tablet_bootstrap.cc:654] T 1832cff5eb3e4da4b9766ae1591df81a P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
W20260925 13:56:20.051769 19488 leader_election.cc:343] T f4429db544534f758883f2fae184ea6d P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Not found: Tablet not found: f4429db544534f758883f2fae184ea6d
I20260925 13:56:20.047518 19553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfc4f99a5964e2ba6146e390e8dba44" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:19.982324 19938 ts_tablet_manager.cc:1434] T 436b8d46e1d244deb8bb9dc128ee6279 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.120s user 0.010s sys 0.000s
I20260925 13:56:20.090998 20366 raft_consensus.cc:493] T c2d333f892f84a258f8a148a90ce3b06 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:20.091506 20366 raft_consensus.cc:515] T c2d333f892f84a258f8a148a90ce3b06 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.092978 19939 tablet_bootstrap.cc:654] T 3e6c8b9bbf4549b3a8685f886973ed9c P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:20.094638 20366 leader_election.cc:290] T c2d333f892f84a258f8a148a90ce3b06 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:20.095451 19694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dc5ea164d6749b0926c63d6bda9dcc5" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:20.097026 19695 raft_consensus.cc:3060] T bbfc4f99a5964e2ba6146e390e8dba44 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:20.098551 20339 leader_election.cc:290] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:20.100210 19938 tablet_bootstrap.cc:492] T 12984db09f34472ba365fdf63a33b720 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:20.102788 19942 tablet_bootstrap.cc:492] T 1832cff5eb3e4da4b9766ae1591df81a P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:20.103294 19942 ts_tablet_manager.cc:1403] T 1832cff5eb3e4da4b9766ae1591df81a P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.173s user 0.019s sys 0.007s
I20260925 13:56:20.106308 19942 raft_consensus.cc:359] T 1832cff5eb3e4da4b9766ae1591df81a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.110627 19959 ts_tablet_manager.cc:1929] T 0de8f2d2766c4ff197b552a1b5ed3d6b P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:20.111107 19959 log.cc:1199] T 0de8f2d2766c4ff197b552a1b5ed3d6b P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/0de8f2d2766c4ff197b552a1b5ed3d6b
W20260925 13:56:20.112267 19488 consensus_peers.cc:597] T 1da6eadb1648479889027fa911f4676b P 8f578869bc084a5d8a9a69f143954eb3 -> Peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Couldn't send request to peer 7c1b52895a14482fb54a6758f5054f8f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1da6eadb1648479889027fa911f4676b. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:20.106913 19942 raft_consensus.cc:385] T 1832cff5eb3e4da4b9766ae1591df81a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:20.112313 20375 consensus_queue.cc:1048] T c2d333f892f84a258f8a148a90ce3b06 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 13:56:20.112792 19942 raft_consensus.cc:740] T 1832cff5eb3e4da4b9766ae1591df81a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:20.113837 20375 consensus_queue.cc:1048] T f5a0bd0e205f4889be93739dfbd2a816 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:20.117211 19942 consensus_queue.cc:260] T 1832cff5eb3e4da4b9766ae1591df81a P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.031736 19273 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_93aa193a56f246aeaa089b2bfc0e9d8c" } new_table_name: "default.Table_5bf9b87b5083469fad04eb3a80a4b8d5" modify_external_catalogs: true
I20260925 13:56:20.128325 19942 ts_tablet_manager.cc:1434] T 1832cff5eb3e4da4b9766ae1591df81a P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.025s user 0.005s sys 0.000s
I20260925 13:56:20.130237 19939 tablet_bootstrap.cc:492] T 3e6c8b9bbf4549b3a8685f886973ed9c P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:20.130657 19939 ts_tablet_manager.cc:1403] T 3e6c8b9bbf4549b3a8685f886973ed9c P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.238s user 0.003s sys 0.007s
I20260925 13:56:20.136461 19942 tablet_bootstrap.cc:492] T b12a6c6cff024d70b53cc2c9169dd48b P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
W20260925 13:56:20.142220 19488 leader_election.cc:343] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:20.142805 19773 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.193:38269, user_credentials={real_user=slave}} blocked reactor thread for 227312us
I20260925 13:56:20.142648 19939 raft_consensus.cc:359] T 3e6c8b9bbf4549b3a8685f886973ed9c P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.144936 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 2a1f34dba13143b097a38bab3782c404 (table default.Table_3cf69a468c014c279a928211b6ff89a2 [id=fe0f78d1c6e44f589aede852f77f88fc]) successfully deleted
I20260925 13:56:20.151837 19959 ts_tablet_manager.cc:1950] T 0de8f2d2766c4ff197b552a1b5ed3d6b P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:20.153333 19819 tablet_service.cc:1558] Processing DeleteTablet for tablet bbfc4f99a5964e2ba6146e390e8dba44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:19 UTC) from {username='slave'} at 127.0.0.1:51442
W20260925 13:56:20.153687 19631 leader_election.cc:343] T c2d333f892f84a258f8a148a90ce3b06 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: c2d333f892f84a258f8a148a90ce3b06
I20260925 13:56:20.153944 19818 tablet_service.cc:1558] Processing DeleteTablet for tablet a26f5a02311c435a8c2434fd030c80a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:19 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:20.154327 20365 consensus_queue.cc:1048] T b12a6c6cff024d70b53cc2c9169dd48b P f790493ba7ad403993b678f2df2b55be [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 13:56:20.155560 19959 tablet_replica.cc:333] T 26644768f516412f8c95d68a76bb99c5 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:20.156034 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 0de8f2d2766c4ff197b552a1b5ed3d6b (table default.Table_052e4e4ee53e4e408b18cd0ce8063d31 [id=993a34f0c22c4555834fb5cf42da8579]) successfully deleted
I20260925 13:56:20.156996 19959 raft_consensus.cc:2243] T 26644768f516412f8c95d68a76bb99c5 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:20.163925 19942 tablet_bootstrap.cc:654] T b12a6c6cff024d70b53cc2c9169dd48b P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:20.045022 19955 log.cc:1199] T f4429db544534f758883f2fae184ea6d P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/f4429db544534f758883f2fae184ea6d
I20260925 13:56:20.177198 19955 ts_tablet_manager.cc:1950] T f4429db544534f758883f2fae184ea6d P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:20.178081 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 86) took 3982 ms. Trace:
I20260925 13:56:20.178413 19955 rpcz_store.cc:276] 0925 13:56:16.196117 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:16.196234 (+ 117us) service_pool.cc:224] Handling call
0925 13:56:19.858055 (+3661821us) ts_tablet_manager.cc:1151] Deleting tablet f4429db544534f758883f2fae184ea6d
0925 13:56:19.858090 (+ 35us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:19.936182 (+ 78092us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:20.010004 (+ 73822us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:20.178016 (+168012us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3661341}
I20260925 13:56:20.143256 19939 raft_consensus.cc:385] T 3e6c8b9bbf4549b3a8685f886973ed9c P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:20.179644 19939 raft_consensus.cc:740] T 3e6c8b9bbf4549b3a8685f886973ed9c P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:20.179768 19955 tablet_replica.cc:333] T 4d3dbd9796564472b44a31ccaf56d898 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:20.180316 19939 consensus_queue.cc:260] T 3e6c8b9bbf4549b3a8685f886973ed9c P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.182262 19939 ts_tablet_manager.cc:1434] T 3e6c8b9bbf4549b3a8685f886973ed9c P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.051s user 0.004s sys 0.001s
I20260925 13:56:20.183185 19939 tablet_bootstrap.cc:492] T 137c43fb74ec4cc48d30ed3edac81d98 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:20.191648 19959 pending_rounds.cc:70] T 26644768f516412f8c95d68a76bb99c5 P f790493ba7ad403993b678f2df2b55be: Trying to abort 1 pending ops.
I20260925 13:56:20.191862 19959 pending_rounds.cc:77] T 26644768f516412f8c95d68a76bb99c5 P f790493ba7ad403993b678f2df2b55be: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251386583080960 op_type: NO_OP noop_request { }
I20260925 13:56:20.192195 19959 raft_consensus.cc:2889] T 26644768f516412f8c95d68a76bb99c5 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:56:20.192526 19959 raft_consensus.cc:2272] T 26644768f516412f8c95d68a76bb99c5 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:20.194971 19938 tablet_bootstrap.cc:654] T 12984db09f34472ba365fdf63a33b720 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
W20260925 13:56:20.200676 19273 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e9dbcd85dc8647c68c120ba3fcfe1790 consensus state has no leader
I20260925 13:56:20.200953 19942 tablet_bootstrap.cc:492] T b12a6c6cff024d70b53cc2c9169dd48b P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:20.201314 19942 ts_tablet_manager.cc:1403] T b12a6c6cff024d70b53cc2c9169dd48b P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.065s user 0.009s sys 0.000s
I20260925 13:56:20.203789 19942 raft_consensus.cc:359] T b12a6c6cff024d70b53cc2c9169dd48b P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.204638 19959 ts_tablet_manager.cc:1916] T 26644768f516412f8c95d68a76bb99c5 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:20.206669 19942 raft_consensus.cc:385] T b12a6c6cff024d70b53cc2c9169dd48b P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:20.209854 19273 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_10868b99c6ec43b2a38a3a4a5bdcf960" } modify_external_catalogs: true
I20260925 13:56:20.210336 19273 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_10868b99c6ec43b2a38a3a4a5bdcf960" } modify_external_catalogs: true
I20260925 13:56:20.206955 19942 raft_consensus.cc:740] T b12a6c6cff024d70b53cc2c9169dd48b P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:20.212025 19274 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_546097946d054781b0acae5186c5d43d" } modify_external_catalogs: true
I20260925 13:56:20.212498 19274 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_546097946d054781b0acae5186c5d43d" } modify_external_catalogs: true
I20260925 13:56:20.212316 20382 tablet.cc:1751] T f1c6e2137700448a8ebf859a9ca9085d P 7c1b52895a14482fb54a6758f5054f8f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:20.213927 19942 consensus_queue.cc:260] T b12a6c6cff024d70b53cc2c9169dd48b P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.217228 19942 ts_tablet_manager.cc:1434] T b12a6c6cff024d70b53cc2c9169dd48b P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.016s user 0.002s sys 0.003s
I20260925 13:56:20.217988 19942 tablet_bootstrap.cc:492] T 2a1f34dba13143b097a38bab3782c404 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:20.243532 20377 tablet.cc:1751] T f1c6e2137700448a8ebf859a9ca9085d P 8f578869bc084a5d8a9a69f143954eb3: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260925 13:56:20.243462 19488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.194:38641, user_credentials={real_user=slave}} blocked reactor thread for 49478.2us
I20260925 13:56:20.239567 19668 tablet_service.cc:1558] Processing DeleteTablet for tablet c2d333f892f84a258f8a148a90ce3b06 with delete_type TABLET_DATA_DELETED (replaced by tablet 908df914cfec490cb7058dbe7b501c97) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:20.246924 20356 tablet.cc:1751] T bae8c5b6e1014731901bc3f336104784 P 7c1b52895a14482fb54a6758f5054f8f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:20.263433 19939 tablet_bootstrap.cc:654] T 137c43fb74ec4cc48d30ed3edac81d98 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
W20260925 13:56:20.272073 19490 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 277641us
I20260925 13:56:20.273161 20338 consensus_queue.cc:1048] T f5a0bd0e205f4889be93739dfbd2a816 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 13:56:20.274852 19490 leader_election.cc:304] T 2a1f34dba13143b097a38bab3782c404 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:20.275468 20338 raft_consensus.cc:2804] T 2a1f34dba13143b097a38bab3782c404 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:20.275852 20338 raft_consensus.cc:697] T 2a1f34dba13143b097a38bab3782c404 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Running, Role: LEADER
I20260925 13:56:20.276562 20338 consensus_queue.cc:237] T 2a1f34dba13143b097a38bab3782c404 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.277367 19938 tablet_bootstrap.cc:492] T 12984db09f34472ba365fdf63a33b720 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:20.285698 20202 consensus_queue.cc:1048] T c65a8cd3060c405c871270543b4623f7 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:20.299036 19278 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_e982ff965b73447b8fbdb7d0bb22532c" } new_table_name: "default.Table_25c2c368343741a3923a23f0a7cb2b62" modify_external_catalogs: true
I20260925 13:56:20.299685 19272 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_48dae3cbc7b245408eecc2a1413b794b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:20.301031 19272 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_48dae3cbc7b245408eecc2a1413b794b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:20.308012 19279 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_0b05542218754c9c8bbf04c76ba736b5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:20.312342 19279 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0b05542218754c9c8bbf04c76ba736b5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 13:56:20.315213 19490 leader_election.cc:343] T f4429db544534f758883f2fae184ea6d P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: f4429db544534f758883f2fae184ea6d
I20260925 13:56:20.315582 19490 leader_election.cc:304] T f4429db544534f758883f2fae184ea6d P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3; no voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:20.317416 20385 tablet.cc:1751] T bae8c5b6e1014731901bc3f336104784 P 8f578869bc084a5d8a9a69f143954eb3: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:20.327055 19938 ts_tablet_manager.cc:1403] T 12984db09f34472ba365fdf63a33b720 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.227s user 0.008s sys 0.003s
I20260925 13:56:20.329465 19938 raft_consensus.cc:359] T 12984db09f34472ba365fdf63a33b720 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.330017 19275 catalog_manager.cc:6391] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: default.Table_b2d4cd63d7124850a1070083a1c003c8 [id=e2b98c2d1dbf4efea5fa61f182fc2fd4] alter complete (version 1)
I20260925 13:56:20.330138 19938 raft_consensus.cc:385] T 12984db09f34472ba365fdf63a33b720 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:20.330369 19938 raft_consensus.cc:740] T 12984db09f34472ba365fdf63a33b720 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:20.342806 19668 tablet_service.cc:1558] Processing DeleteTablet for tablet 1832cff5eb3e4da4b9766ae1591df81a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:16 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:20.343710 19695 raft_consensus.cc:3060] T b12a6c6cff024d70b53cc2c9169dd48b P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
W20260925 13:56:20.346761 19267 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet e9dbcd85dc8647c68c120ba3fcfe1790 (table default.Table_5bf9b87b5083469fad04eb3a80a4b8d5 [id=443536a0bd4d4d568d52f6ffe3d06ecb], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet e9dbcd85dc8647c68c120ba3fcfe1790 consensus state has no leader
I20260925 13:56:20.346480 19277 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_f00423c3addb4d2a87e8d419f72a686c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:20.348148 19277 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f00423c3addb4d2a87e8d419f72a686c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 13:56:20.350764 19490 raft_consensus.cc:2686] T f4429db544534f758883f2fae184ea6d P 8f578869bc084a5d8a9a69f143954eb3: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260925 13:56:20.222854 19942 tablet_bootstrap.cc:654] T 2a1f34dba13143b097a38bab3782c404 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:20.253782 20011 ts_tablet_manager.cc:1929] T 5943795bc8b24bb48b80d6d06d17750b P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:20.351706 20011 log.cc:1199] T 5943795bc8b24bb48b80d6d06d17750b P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/5943795bc8b24bb48b80d6d06d17750b
I20260925 13:56:20.352336 20011 ts_tablet_manager.cc:1950] T 5943795bc8b24bb48b80d6d06d17750b P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
W20260925 13:56:20.352604 19490 consensus_peers.cc:597] T dc14c37aa393407a87af0d0eb8b88d82 P 8f578869bc084a5d8a9a69f143954eb3 -> Peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Couldn't send request to peer f790493ba7ad403993b678f2df2b55be. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dc14c37aa393407a87af0d0eb8b88d82. This is attempt 1: this message will repeat every 5th retry.
W20260925 13:56:20.353335 19629 leader_election.cc:343] T 5943795bc8b24bb48b80d6d06d17750b P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Not found: Tablet not found: 5943795bc8b24bb48b80d6d06d17750b
I20260925 13:56:20.353403 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 133) took 1035 ms. Trace:
I20260925 13:56:20.353775 19629 leader_election.cc:304] T 5943795bc8b24bb48b80d6d06d17750b P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
W20260925 13:56:20.354362 19629 raft_consensus.cc:2686] T 5943795bc8b24bb48b80d6d06d17750b P 7c1b52895a14482fb54a6758f5054f8f: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260925 13:56:20.356509 19274 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet e6e59d6f4f6e4de1be1cf450620cfa59
I20260925 13:56:20.359819 19818 tablet_service.cc:1558] Processing DeleteTablet for tablet e6e59d6f4f6e4de1be1cf450620cfa59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:20 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:20.362102 19668 tablet_service.cc:1558] Processing DeleteTablet for tablet e6e59d6f4f6e4de1be1cf450620cfa59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:20 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:20.331090 19938 consensus_queue.cc:260] T 12984db09f34472ba365fdf63a33b720 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.372475 19818 tablet_service.cc:1511] Processing CreateTablet for tablet b3172d99afee4a4c878e71b645037917 (DEFAULT_TABLE table=default.Table_bd52a86b8ff14bdbb06fcf88a60b1e82 [id=203024d67e444f839391c8670d7c7d3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:20.373622 19818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3172d99afee4a4c878e71b645037917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:20.373376 19668 tablet_service.cc:1511] Processing CreateTablet for tablet c5a9df833271480aa49feb7d9f925459 (DEFAULT_TABLE table=default.Table_e982ff965b73447b8fbdb7d0bb22532c [id=7d4523c5927040ba846c35a4b02172a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:20.375996 19275 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 436b8d46e1d244deb8bb9dc128ee6279 (table default.Table_079723ff92144ba293733e54ab25d782 [id=4b0bfb9726854123864f8fe743a16a33]). Expected version 1 got 0
I20260925 13:56:20.379366 19819 tablet_service.cc:1511] Processing CreateTablet for tablet c5a9df833271480aa49feb7d9f925459 (DEFAULT_TABLE table=default.Table_e982ff965b73447b8fbdb7d0bb22532c [id=7d4523c5927040ba846c35a4b02172a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:20.379781 19670 tablet_service.cc:1511] Processing CreateTablet for tablet b3172d99afee4a4c878e71b645037917 (DEFAULT_TABLE table=default.Table_bd52a86b8ff14bdbb06fcf88a60b1e82 [id=203024d67e444f839391c8670d7c7d3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:20.380275 19819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5a9df833271480aa49feb7d9f925459. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:20.383853 19955 raft_consensus.cc:2243] T 4d3dbd9796564472b44a31ccaf56d898 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:20.384817 19955 raft_consensus.cc:2272] T 4d3dbd9796564472b44a31ccaf56d898 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:20.389055 19669 tablet_service.cc:1511] Processing CreateTablet for tablet 78d8c01c88d24deeae21d647e7f205ba (DEFAULT_TABLE table=default.Table_f1564f324ef645f9a62c24fb5d1ebd06 [id=4ca863b59dfd4410822e15705934b503]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:20.392206 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet f4429db544534f758883f2fae184ea6d (table default.Table_731e8683c6714e22a725eae1b86ff4ec [id=e441f4e1a35e400da9cd488ac1d4a754]) successfully deleted
I20260925 13:56:20.392951 19938 ts_tablet_manager.cc:1434] T 12984db09f34472ba365fdf63a33b720 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.066s user 0.004s sys 0.000s
I20260925 13:56:20.394399 19938 tablet_bootstrap.cc:492] T e3cf9676160c4af08f1f62ae952b4c57 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:20.353789 20011 rpcz_store.cc:276] 0925 13:56:19.317528 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:19.317641 (+ 113us) service_pool.cc:224] Handling call
0925 13:56:19.953229 (+635588us) ts_tablet_manager.cc:1151] Deleting tablet 5943795bc8b24bb48b80d6d06d17750b
0925 13:56:19.997988 (+ 44759us) spinlock_profiling.cc:243] Waited 43.5 ms on lock 0x7b5800000f98. stack: 00007f1ff9889323 00007f1ff9889220 00007f1ff942d574 0000000000516964 00000000005168f9 00007f1ff973f8a1 00007f1ff9883b51 00007f1ff9884182 00007f20027f72c9 00007f2001a08918 00007f2001acf2c9 00007f2001acf8c3 00007f2001b00960 00007f2001ae6e6d 00007f2001ae6e19 00007f2001ae6da9
0925 13:56:19.998031 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:20.162959 (+164928us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:20.253660 (+ 90701us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:20.353352 (+ 99692us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":100010368,"tablet-delete.queue_time_us":598259}
I20260925 13:56:20.399803 19668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5a9df833271480aa49feb7d9f925459. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:20.399890 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 5943795bc8b24bb48b80d6d06d17750b (table default.Table_36b4dd2097aa4e2dbd58d98277e90281 [id=90bd44e8847549f2964a32f926ef1d99]) successfully deleted
I20260925 13:56:20.409214 19271 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_43b8599ad301426687e4035cec94b286" } new_table_name: "default.Table_d3d2076fbb6241a68f04e83462a34182" modify_external_catalogs: true
I20260925 13:56:20.414975 19669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78d8c01c88d24deeae21d647e7f205ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:20.426120 19280 master_service.cc:960] ReplaceTablet: received request to replace tablet 137c43fb74ec4cc48d30ed3edac81d98 from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:20.432838 19670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3172d99afee4a4c878e71b645037917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:20.411580 19518 tablet_service.cc:1558] Processing DeleteTablet for tablet a26f5a02311c435a8c2434fd030c80a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:19 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:20.412336 19520 tablet_service.cc:1558] Processing DeleteTablet for tablet e6e59d6f4f6e4de1be1cf450620cfa59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:20 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:20.413646 19522 tablet_service.cc:1511] Processing CreateTablet for tablet b3172d99afee4a4c878e71b645037917 (DEFAULT_TABLE table=default.Table_bd52a86b8ff14bdbb06fcf88a60b1e82 [id=203024d67e444f839391c8670d7c7d3b]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 13:56:20.447387 19629 leader_election.cc:343] T a88bbe1ebee7468d8f49dfd25637fe83 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Not found: Tablet not found: a88bbe1ebee7468d8f49dfd25637fe83
I20260925 13:56:20.447795 19629 leader_election.cc:304] T a88bbe1ebee7468d8f49dfd25637fe83 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:20.448252 19522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3172d99afee4a4c878e71b645037917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:20.448757 20366 raft_consensus.cc:2749] T a88bbe1ebee7468d8f49dfd25637fe83 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:20.451205 19629 leader_election.cc:304] T bbfc4f99a5964e2ba6146e390e8dba44 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:20.452203 20366 raft_consensus.cc:2749] T bbfc4f99a5964e2ba6146e390e8dba44 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:20.458725 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d333f892f84a258f8a148a90ce3b06" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:20.461522 19959 ts_tablet_manager.cc:1929] T 26644768f516412f8c95d68a76bb99c5 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:20.461591 19838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dc5ea164d6749b0926c63d6bda9dcc5" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:20.461971 19959 log.cc:1199] T 26644768f516412f8c95d68a76bb99c5 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/26644768f516412f8c95d68a76bb99c5
I20260925 13:56:20.462332 19838 raft_consensus.cc:2468] T 8dc5ea164d6749b0926c63d6bda9dcc5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:20.437042 19939 tablet_bootstrap.cc:492] T 137c43fb74ec4cc48d30ed3edac81d98 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:20.464576 19939 ts_tablet_manager.cc:1403] T 137c43fb74ec4cc48d30ed3edac81d98 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.282s user 0.009s sys 0.000s
I20260925 13:56:20.473881 19939 raft_consensus.cc:359] T 137c43fb74ec4cc48d30ed3edac81d98 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.474751 19939 raft_consensus.cc:385] T 137c43fb74ec4cc48d30ed3edac81d98 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:20.477998 19955 ts_tablet_manager.cc:1916] T 4d3dbd9796564472b44a31ccaf56d898 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:20.479707 19959 ts_tablet_manager.cc:1950] T 26644768f516412f8c95d68a76bb99c5 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:20.481137 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 135) took 1155 ms. Trace:
I20260925 13:56:20.482996 19818 tablet_service.cc:1511] Processing CreateTablet for tablet 78d8c01c88d24deeae21d647e7f205ba (DEFAULT_TABLE table=default.Table_f1564f324ef645f9a62c24fb5d1ebd06 [id=4ca863b59dfd4410822e15705934b503]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:20.484138 19818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78d8c01c88d24deeae21d647e7f205ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:20.506999 20011 rpcz_store.cc:276] 0925 13:56:19.325373 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:19.325493 (+ 120us) service_pool.cc:224] Handling call
0925 13:56:20.397312 (+1071819us) ts_tablet_manager.cc:1151] Deleting tablet 0de8f2d2766c4ff197b552a1b5ed3d6b
0925 13:56:20.480827 (+ 83515us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:20.481083 (+ 256us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1042237}
I20260925 13:56:20.508196 19959 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 137) took 1187 ms. Trace:
I20260925 13:56:20.508790 19959 rpcz_store.cc:276] 0925 13:56:19.320958 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:19.322170 (+ 1212us) service_pool.cc:224] Handling call
0925 13:56:20.155422 (+833252us) ts_tablet_manager.cc:1151] Deleting tablet 26644768f516412f8c95d68a76bb99c5
0925 13:56:20.155464 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:20.326593 (+171129us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:20.461391 (+134798us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:20.482325 (+ 20934us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":812444}
I20260925 13:56:20.510895 19959 tablet_replica.cc:333] T 1513c0048bd244028d3dfa8d118bc14e P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:20.510933 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 26644768f516412f8c95d68a76bb99c5 (table default.Table_bd52a86b8ff14bdbb06fcf88a60b1e82 [id=203024d67e444f839391c8670d7c7d3b]) successfully deleted
I20260925 13:56:20.511695 19959 raft_consensus.cc:2243] T 1513c0048bd244028d3dfa8d118bc14e P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:20.512657 20366 raft_consensus.cc:493] T 1832cff5eb3e4da4b9766ae1591df81a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:20.513094 20366 raft_consensus.cc:515] T 1832cff5eb3e4da4b9766ae1591df81a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:20.513391 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 0de8f2d2766c4ff197b552a1b5ed3d6b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0de8f2d2766c4ff197b552a1b5ed3d6b already in progress: creating tablet
I20260925 13:56:20.513558 19942 tablet_bootstrap.cc:492] T 2a1f34dba13143b097a38bab3782c404 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:20.514074 19942 ts_tablet_manager.cc:1403] T 2a1f34dba13143b097a38bab3782c404 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.296s user 0.020s sys 0.025s
I20260925 13:56:20.514340 20011 tablet_replica.cc:333] T 26644768f516412f8c95d68a76bb99c5 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:20.514966 20366 leader_election.cc:290] T 1832cff5eb3e4da4b9766ae1591df81a P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:20.515067 20011 raft_consensus.cc:2243] T 26644768f516412f8c95d68a76bb99c5 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:20.512233 19959 raft_consensus.cc:2272] T 1513c0048bd244028d3dfa8d118bc14e P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:20.516594 19942 raft_consensus.cc:359] T 2a1f34dba13143b097a38bab3782c404 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.517247 19942 raft_consensus.cc:385] T 2a1f34dba13143b097a38bab3782c404 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260925 13:56:20.517701 19631 leader_election.cc:343] T 1832cff5eb3e4da4b9766ae1591df81a P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 1832cff5eb3e4da4b9766ae1591df81a
I20260925 13:56:20.517545 19942 raft_consensus.cc:740] T 2a1f34dba13143b097a38bab3782c404 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:20.518880 19959 ts_tablet_manager.cc:1916] T 1513c0048bd244028d3dfa8d118bc14e P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:20.518683 19942 consensus_queue.cc:260] T 2a1f34dba13143b097a38bab3782c404 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.524417 19942 ts_tablet_manager.cc:1434] T 2a1f34dba13143b097a38bab3782c404 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20260925 13:56:20.526304 20366 consensus_queue.cc:1048] T e6e59d6f4f6e4de1be1cf450620cfa59 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:20.527190 19942 tablet_bootstrap.cc:492] T 3e6c8b9bbf4549b3a8685f886973ed9c P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
W20260925 13:56:20.532099 19265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.254:40825, user_credentials={real_user=slave}} blocked reactor thread for 99979us
I20260925 13:56:20.551771 19942 tablet_bootstrap.cc:654] T 3e6c8b9bbf4549b3a8685f886973ed9c P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:20.443776 19532 tablet_service.cc:1558] Processing DeleteTablet for tablet bbfc4f99a5964e2ba6146e390e8dba44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:19 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:20.482311 19524 tablet_service.cc:1511] Processing CreateTablet for tablet c5a9df833271480aa49feb7d9f925459 (DEFAULT_TABLE table=default.Table_e982ff965b73447b8fbdb7d0bb22532c [id=7d4523c5927040ba846c35a4b02172a7]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 13:56:20.487706 19490 consensus_peers.cc:597] T c2d333f892f84a258f8a148a90ce3b06 P 8f578869bc084a5d8a9a69f143954eb3 -> Peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Couldn't send request to peer f790493ba7ad403993b678f2df2b55be. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c2d333f892f84a258f8a148a90ce3b06. This is attempt 1: this message will repeat every 5th retry.
W20260925 13:56:20.552721 19490 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 65128.9us
W20260925 13:56:20.553573 19490 consensus_peers.cc:597] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3 -> Peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Couldn't send request to peer f790493ba7ad403993b678f2df2b55be. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:20.554059 19490 leader_election.cc:304] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:20.554873 19524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5a9df833271480aa49feb7d9f925459. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:20.555799 20375 raft_consensus.cc:2804] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:20.556133 20375 raft_consensus.cc:493] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:20.556444 20375 raft_consensus.cc:3060] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:20.557545 19629 leader_election.cc:304] T c2d333f892f84a258f8a148a90ce3b06 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:20.491641 20385 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:40018 (request call id 138) took 1045 ms. Trace:
I20260925 13:56:20.475075 19939 raft_consensus.cc:740] T 137c43fb74ec4cc48d30ed3edac81d98 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:20.562191 20366 raft_consensus.cc:2749] T c2d333f892f84a258f8a148a90ce3b06 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:56:20.577620 19942 tablet_bootstrap.cc:492] T 3e6c8b9bbf4549b3a8685f886973ed9c P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:20.578002 19942 ts_tablet_manager.cc:1403] T 3e6c8b9bbf4549b3a8685f886973ed9c P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.051s user 0.003s sys 0.006s
I20260925 13:56:20.580780 19942 raft_consensus.cc:359] T 3e6c8b9bbf4549b3a8685f886973ed9c P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.581403 19942 raft_consensus.cc:385] T 3e6c8b9bbf4549b3a8685f886973ed9c P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:20.581657 19942 raft_consensus.cc:740] T 3e6c8b9bbf4549b3a8685f886973ed9c P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:20.591696 19942 consensus_queue.cc:260] T 3e6c8b9bbf4549b3a8685f886973ed9c P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.593538 20366 raft_consensus.cc:493] T 3e6c8b9bbf4549b3a8685f886973ed9c P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:20.593946 20366 raft_consensus.cc:515] T 3e6c8b9bbf4549b3a8685f886973ed9c P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.595729 20366 leader_election.cc:290] T 3e6c8b9bbf4549b3a8685f886973ed9c P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:20.600474 19274 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet bae8c5b6e1014731901bc3f336104784 (table default.Table_99593e98f3394811b4c91e234185cbef [id=0736e722eb8c43e9b6a151a7c2880fe3]). Expected version 1 got 0
I20260925 13:56:20.613399 19520 tablet_service.cc:1511] Processing CreateTablet for tablet 78d8c01c88d24deeae21d647e7f205ba (DEFAULT_TABLE table=default.Table_f1564f324ef645f9a62c24fb5d1ebd06 [id=4ca863b59dfd4410822e15705934b503]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:20.614611 19520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78d8c01c88d24deeae21d647e7f205ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:20.615531 19938 tablet_bootstrap.cc:700] T e3cf9676160c4af08f1f62ae952b4c57 P f790493ba7ad403993b678f2df2b55be: Time spent opening tablet: real 0.217s user 0.004s sys 0.000s
I20260925 13:56:20.616277 19938 tablet_bootstrap.cc:654] T e3cf9676160c4af08f1f62ae952b4c57 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:20.617439 19553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1832cff5eb3e4da4b9766ae1591df81a" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:20.618016 19553 raft_consensus.cc:2468] T 1832cff5eb3e4da4b9766ae1591df81a P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:20.620255 19942 ts_tablet_manager.cc:1434] T 3e6c8b9bbf4549b3a8685f886973ed9c P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.042s user 0.007s sys 0.001s
I20260925 13:56:20.621948 19942 tablet_bootstrap.cc:492] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:20.606092 20201 raft_consensus.cc:493] T 3e6c8b9bbf4549b3a8685f886973ed9c P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:20.626248 20011 raft_consensus.cc:2272] T 26644768f516412f8c95d68a76bb99c5 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:20.560179 20385 rpcz_store.cc:276] 0925 13:56:19.446367 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:19.448479 (+ 2112us) service_pool.cc:224] Handling call
0925 13:56:20.491571 (+1043092us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:19.550170 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0925 13:56:19.550619 (+ 449us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0925 13:56:19.550651 (+ 32us) alter_schema_op.cc:66] Acquired schema lock
0925 13:56:19.562399 (+ 11748us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0925 13:56:19.644165 (+ 81766us) alter_schema_op.cc:128] START. Timestamp: 1790344579576488
0925 13:56:19.645094 (+ 929us) op_driver.cc:348] REPLICATION: starting
0925 13:56:19.696903 (+ 51809us) spinlock_profiling.cc:243] Waited 1.35 ms on lock 0x7b540009d6a0. stack: 00007f1358052323 00007f1358052220 00007f1357bf6574 0000000000516964 00000000005168f9 00007f135f14faa1 00007f135f163a50 00007f135f1ed302 00007f135f1e2727 00007f135f1e2564 00007f135f672741 00007f135f671f18 00007f135f676bb1 00007f135f676b69 00007f135f676af9 00007f135f676ac1
0925 13:56:19.697486 (+ 583us) log.cc:844] Serialized 173 byte log entry
0925 13:56:20.195827 (+498341us) op_driver.cc:464] REPLICATION: finished
0925 13:56:20.317202 (+121375us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0925 13:56:20.318619 (+ 1417us) log.cc:844] Serialized 5 byte log entry
0925 13:56:20.430118 (+111499us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:20.430793 (+ 675us) log.cc:844] Serialized 14 byte log entry
0925 13:56:20.491440 (+ 60647us) alter_schema_op.cc:73] Released schema lock
0925 13:56:20.491475 (+ 35us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":121203,"prepare.queue_time_us":66149,"prepare.run_cpu_time_us":24679,"prepare.run_wall_time_us":358319,"raft.queue_time_us":2938,"raft.run_cpu_time_us":1040,"raft.run_wall_time_us":174837,"replication_time_us":550611,"spinlock_wait_cycles":4368000,"thread_start_us":280580,"threads_started":3,"wal-append.queue_time_us":78325}]]}
I20260925 13:56:20.637042 20408 tablet.cc:1751] T f1c6e2137700448a8ebf859a9ca9085d P f790493ba7ad403993b678f2df2b55be: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:20.633389 19939 consensus_queue.cc:260] T 137c43fb74ec4cc48d30ed3edac81d98 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.641785 20011 ts_tablet_manager.cc:1916] T 26644768f516412f8c95d68a76bb99c5 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:20.644739 19838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e6c8b9bbf4549b3a8685f886973ed9c" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:20.645282 19838 raft_consensus.cc:2468] T 3e6c8b9bbf4549b3a8685f886973ed9c P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:20.647670 19942 tablet_bootstrap.cc:654] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:20.655118 19631 leader_election.cc:304] T 3e6c8b9bbf4549b3a8685f886973ed9c P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:20.665073 20407 consensus_queue.cc:1048] T e6e59d6f4f6e4de1be1cf450620cfa59 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:20.665889 20366 raft_consensus.cc:2804] T 3e6c8b9bbf4549b3a8685f886973ed9c P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:20.666251 20366 raft_consensus.cc:493] T 3e6c8b9bbf4549b3a8685f886973ed9c P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:20.666563 20366 raft_consensus.cc:3060] T 3e6c8b9bbf4549b3a8685f886973ed9c P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:20.666775 19629 leader_election.cc:304] T 1832cff5eb3e4da4b9766ae1591df81a P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3; no voters: f790493ba7ad403993b678f2df2b55be
W20260925 13:56:20.672365 19278 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c5a9df833271480aa49feb7d9f925459 consensus state has no leader
I20260925 13:56:20.673146 20409 raft_consensus.cc:2804] T 1832cff5eb3e4da4b9766ae1591df81a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:20.673424 19278 master_service.cc:960] ReplaceTablet: received request to replace tablet 5a9b3dfaa7474611a1824d63eb906b43 from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:20.673480 20409 raft_consensus.cc:493] T 1832cff5eb3e4da4b9766ae1591df81a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:20.673767 20409 raft_consensus.cc:3060] T 1832cff5eb3e4da4b9766ae1591df81a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:20.671679 20201 raft_consensus.cc:515] T 3e6c8b9bbf4549b3a8685f886973ed9c P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.687006 20201 leader_election.cc:290] T 3e6c8b9bbf4549b3a8685f886973ed9c P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:20.687482 19553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e6c8b9bbf4549b3a8685f886973ed9c" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:20.688134 19553 raft_consensus.cc:2468] T 3e6c8b9bbf4549b3a8685f886973ed9c P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:20.689059 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e6c8b9bbf4549b3a8685f886973ed9c" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:20.700162 19272 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_16c91b8d55be449899b5e3a1884d81b4" } modify_external_catalogs: true
I20260925 13:56:20.706265 19277 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_e7478883ce9046579ee733f571ea95e9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:20.708196 19277 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e7478883ce9046579ee733f571ea95e9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:20.710709 19273 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 12984db09f34472ba365fdf63a33b720
I20260925 13:56:20.700767 19272 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_16c91b8d55be449899b5e3a1884d81b4" } modify_external_catalogs: true
I20260925 13:56:20.713689 19832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e6c8b9bbf4549b3a8685f886973ed9c" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:20.714222 19832 raft_consensus.cc:2468] T 3e6c8b9bbf4549b3a8685f886973ed9c P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:20.717851 19490 leader_election.cc:304] T 3e6c8b9bbf4549b3a8685f886973ed9c P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:20.711875 20375 raft_consensus.cc:515] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.720261 19818 tablet_service.cc:1558] Processing DeleteTablet for tablet 12984db09f34472ba365fdf63a33b720 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:20 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:20.721289 20375 leader_election.cc:290] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:20.722216 20408 tablet.cc:1751] T bae8c5b6e1014731901bc3f336104784 P f790493ba7ad403993b678f2df2b55be: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:20.725797 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dc5ea164d6749b0926c63d6bda9dcc5" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
W20260925 13:56:20.727068 19488 leader_election.cc:343] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:20.728672 19939 ts_tablet_manager.cc:1434] T 137c43fb74ec4cc48d30ed3edac81d98 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.264s user 0.003s sys 0.002s
I20260925 13:56:20.729547 19939 tablet_bootstrap.cc:492] T f514fa52e7524cd3ba07f3f65de532d4 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:20.736651 19669 tablet_service.cc:1558] Processing DeleteTablet for tablet 12984db09f34472ba365fdf63a33b720 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:20 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:20.738183 19832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dc5ea164d6749b0926c63d6bda9dcc5" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:20.738714 19832 raft_consensus.cc:3060] T 8dc5ea164d6749b0926c63d6bda9dcc5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:20.744246 20338 raft_consensus.cc:2804] T 3e6c8b9bbf4549b3a8685f886973ed9c P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:20.744632 20338 raft_consensus.cc:493] T 3e6c8b9bbf4549b3a8685f886973ed9c P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:20.744946 20338 raft_consensus.cc:3060] T 3e6c8b9bbf4549b3a8685f886973ed9c P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:20.747618 19832 raft_consensus.cc:2468] T 8dc5ea164d6749b0926c63d6bda9dcc5 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:20.751487 19938 tablet_bootstrap.cc:492] T e3cf9676160c4af08f1f62ae952b4c57 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:20.751876 19938 ts_tablet_manager.cc:1403] T e3cf9676160c4af08f1f62ae952b4c57 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.358s user 0.040s sys 0.045s
I20260925 13:56:20.754069 19938 raft_consensus.cc:359] T e3cf9676160c4af08f1f62ae952b4c57 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.754642 19938 raft_consensus.cc:385] T e3cf9676160c4af08f1f62ae952b4c57 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:20.758232 19938 raft_consensus.cc:740] T e3cf9676160c4af08f1f62ae952b4c57 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:20.758992 19938 consensus_queue.cc:260] T e3cf9676160c4af08f1f62ae952b4c57 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.762801 19938 ts_tablet_manager.cc:1434] T e3cf9676160c4af08f1f62ae952b4c57 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20260925 13:56:20.764675 19938 tablet_bootstrap.cc:492] T 2ef3efe644364e93a089ddeed328389d P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:20.768018 19942 tablet_bootstrap.cc:492] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:20.768471 19942 ts_tablet_manager.cc:1403] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.147s user 0.011s sys 0.007s
I20260925 13:56:20.773662 19938 tablet_bootstrap.cc:654] T 2ef3efe644364e93a089ddeed328389d P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
W20260925 13:56:20.775691 19631 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 120667us
I20260925 13:56:20.776147 19942 raft_consensus.cc:359] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.776834 19942 raft_consensus.cc:385] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:20.777112 19942 raft_consensus.cc:740] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:20.778437 20338 raft_consensus.cc:515] T 3e6c8b9bbf4549b3a8685f886973ed9c P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.778911 19942 consensus_queue.cc:260] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.785136 19942 ts_tablet_manager.cc:1434] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.016s user 0.005s sys 0.000s
I20260925 13:56:20.788375 20338 leader_election.cc:290] T 3e6c8b9bbf4549b3a8685f886973ed9c P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:20.788787 19938 tablet_bootstrap.cc:492] T 2ef3efe644364e93a089ddeed328389d P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:20.789295 19938 ts_tablet_manager.cc:1403] T 2ef3efe644364e93a089ddeed328389d P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.025s user 0.011s sys 0.004s
I20260925 13:56:20.792781 19938 raft_consensus.cc:359] T 2ef3efe644364e93a089ddeed328389d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.787220 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e6c8b9bbf4549b3a8685f886973ed9c" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:20.793346 19938 raft_consensus.cc:385] T 2ef3efe644364e93a089ddeed328389d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:20.793591 19938 raft_consensus.cc:740] T 2ef3efe644364e93a089ddeed328389d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:20.794276 19938 consensus_queue.cc:260] T 2ef3efe644364e93a089ddeed328389d P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.796142 19522 tablet_service.cc:1558] Processing DeleteTablet for tablet 12984db09f34472ba365fdf63a33b720 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:20 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:20.798573 19942 tablet_bootstrap.cc:492] T 137c43fb74ec4cc48d30ed3edac81d98 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:20.800136 19490 leader_election.cc:304] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
W20260925 13:56:20.800880 19490 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 50117.9us
I20260925 13:56:20.805651 19938 ts_tablet_manager.cc:1434] T 2ef3efe644364e93a089ddeed328389d P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.016s user 0.005s sys 0.000s
I20260925 13:56:20.808815 20413 raft_consensus.cc:493] T 2a1f34dba13143b097a38bab3782c404 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:20.810992 20368 raft_consensus.cc:2804] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:20.811471 20413 raft_consensus.cc:515] T 2a1f34dba13143b097a38bab3782c404 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.811807 20368 raft_consensus.cc:697] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Running, Role: LEADER
I20260925 13:56:20.812588 20368 consensus_queue.cc:237] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.813565 20413 leader_election.cc:290] T 2a1f34dba13143b097a38bab3782c404 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
W20260925 13:56:20.815403 19631 leader_election.cc:343] T 2a1f34dba13143b097a38bab3782c404 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 2a1f34dba13143b097a38bab3782c404
I20260925 13:56:20.818972 19942 tablet_bootstrap.cc:654] T 137c43fb74ec4cc48d30ed3edac81d98 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
W20260925 13:56:20.822193 19271 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 82972fa4868d4c59b3b9bb1240300938 consensus state has no leader
I20260925 13:56:20.824156 19938 tablet_bootstrap.cc:492] T 70a86ea84d1d49d1b0950b4ce3a3d770 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:20.825732 20385 tablet.cc:1751] T b36aebfe984f454ea910e84645c4ac7d P 8f578869bc084a5d8a9a69f143954eb3: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:20.829224 19938 tablet_bootstrap.cc:654] T 70a86ea84d1d49d1b0950b4ce3a3d770 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:20.846637 19938 tablet_bootstrap.cc:492] T 70a86ea84d1d49d1b0950b4ce3a3d770 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:20.847143 19938 ts_tablet_manager.cc:1403] T 70a86ea84d1d49d1b0950b4ce3a3d770 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.023s user 0.010s sys 0.005s
I20260925 13:56:20.848958 20365 raft_consensus.cc:493] T 436b8d46e1d244deb8bb9dc128ee6279 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:20.849484 20365 raft_consensus.cc:515] T 436b8d46e1d244deb8bb9dc128ee6279 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.850106 19938 raft_consensus.cc:359] T 70a86ea84d1d49d1b0950b4ce3a3d770 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.850855 19938 raft_consensus.cc:385] T 70a86ea84d1d49d1b0950b4ce3a3d770 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:20.851686 20365 leader_election.cc:290] T 436b8d46e1d244deb8bb9dc128ee6279 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:20.851179 19938 raft_consensus.cc:740] T 70a86ea84d1d49d1b0950b4ce3a3d770 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:20.852742 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "436b8d46e1d244deb8bb9dc128ee6279" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:20.853825 19773 leader_election.cc:343] T 436b8d46e1d244deb8bb9dc128ee6279 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:20.852927 19938 consensus_queue.cc:260] T 70a86ea84d1d49d1b0950b4ce3a3d770 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.858156 19838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e6c8b9bbf4549b3a8685f886973ed9c" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:20.858675 19838 raft_consensus.cc:3060] T 3e6c8b9bbf4549b3a8685f886973ed9c P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:20.863526 20377 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:40018 (request call id 134) took 1444 ms. Trace:
I20260925 13:56:20.863932 20377 rpcz_store.cc:276] 0925 13:56:19.419515 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:19.419612 (+ 97us) service_pool.cc:224] Handling call
0925 13:56:20.863459 (+1443847us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:19.672103 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0925 13:56:19.672481 (+ 378us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0925 13:56:19.672511 (+ 30us) alter_schema_op.cc:66] Acquired schema lock
0925 13:56:19.672593 (+ 82us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0925 13:56:19.672702 (+ 109us) alter_schema_op.cc:128] START. Timestamp: 1790344579672666
0925 13:56:19.672721 (+ 19us) op_driver.cc:348] REPLICATION: starting
0925 13:56:19.673267 (+ 546us) log.cc:844] Serialized 173 byte log entry
0925 13:56:20.108603 (+435336us) op_driver.cc:464] REPLICATION: finished
0925 13:56:20.243280 (+134677us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0925 13:56:20.245403 (+ 2123us) log.cc:844] Serialized 5 byte log entry
0925 13:56:20.790564 (+545161us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:20.791181 (+ 617us) log.cc:844] Serialized 14 byte log entry
0925 13:56:20.791266 (+ 85us) alter_schema_op.cc:73] Released schema lock
0925 13:56:20.791293 (+ 27us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":134464,"prepare.queue_time_us":171810,"prepare.run_cpu_time_us":33007,"prepare.run_wall_time_us":347742,"replication_time_us":435771,"thread_start_us":564540,"threads_started":3,"wal-append.queue_time_us":386098}]]}
I20260925 13:56:20.866456 19938 ts_tablet_manager.cc:1434] T 70a86ea84d1d49d1b0950b4ce3a3d770 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.019s user 0.002s sys 0.003s
I20260925 13:56:20.873885 19280 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 137c43fb74ec4cc48d30ed3edac81d98
I20260925 13:56:20.875396 19280 catalog_manager.cc:6934] ReplaceTablet: tablet 137c43fb74ec4cc48d30ed3edac81d98 deleted and replaced by tablet 540132db164c403da72b9377078c4bf0
I20260925 13:56:20.881814 19818 tablet_service.cc:1558] Processing DeleteTablet for tablet 137c43fb74ec4cc48d30ed3edac81d98 with delete_type TABLET_DATA_DELETED (replaced by tablet 540132db164c403da72b9377078c4bf0) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:20.892246 19959 ts_tablet_manager.cc:1929] T 1513c0048bd244028d3dfa8d118bc14e P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:56:20.892680 19959 log.cc:1199] T 1513c0048bd244028d3dfa8d118bc14e P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/1513c0048bd244028d3dfa8d118bc14e
I20260925 13:56:20.893111 20365 raft_consensus.cc:493] T 70a86ea84d1d49d1b0950b4ce3a3d770 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:20.893471 19938 tablet_bootstrap.cc:492] T f50ea68e3b3c4459b3f5e5b0df724724 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:20.893929 19959 ts_tablet_manager.cc:1950] T 1513c0048bd244028d3dfa8d118bc14e P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:20.893746 20365 raft_consensus.cc:515] T 70a86ea84d1d49d1b0950b4ce3a3d770 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.895792 19959 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 138) took 1547 ms. Trace:
I20260925 13:56:20.896083 19959 rpcz_store.cc:276] 0925 13:56:19.348694 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:19.355434 (+ 6740us) service_pool.cc:224] Handling call
0925 13:56:20.510752 (+1155318us) ts_tablet_manager.cc:1151] Deleting tablet 1513c0048bd244028d3dfa8d118bc14e
0925 13:56:20.510798 (+ 46us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:20.640830 (+130032us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:20.892090 (+251260us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:20.895738 (+ 3648us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1154815}
I20260925 13:56:20.896337 19696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a86ea84d1d49d1b0950b4ce3a3d770" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:20.897205 19959 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 140) took 1253 ms. Trace:
I20260925 13:56:20.897351 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 1513c0048bd244028d3dfa8d118bc14e (table default.Table_74b6076b22674a31bc58fef71cc9bfb1 [id=d81cf07e5f9a49d4bbf5aceda3da5a25]) successfully deleted
I20260925 13:56:20.897475 19959 rpcz_store.cc:276] 0925 13:56:19.643857 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:19.644000 (+ 143us) service_pool.cc:224] Handling call
0925 13:56:20.896974 (+1252974us) ts_tablet_manager.cc:1151] Deleting tablet dc14c37aa393407a87af0d0eb8b88d82
0925 13:56:20.897011 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:20.897163 (+ 152us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1139810}
I20260925 13:56:20.898375 19959 tablet_replica.cc:333] T bbfc4f99a5964e2ba6146e390e8dba44 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
W20260925 13:56:20.898708 19773 leader_election.cc:343] T 70a86ea84d1d49d1b0950b4ce3a3d770 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:20.898849 19266 catalog_manager.cc:5030] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): delete failed for tablet dc14c37aa393407a87af0d0eb8b88d82 because the tablet was not found. No further retry: Not found: Tablet not found: dc14c37aa393407a87af0d0eb8b88d82
I20260925 13:56:20.899020 19959 raft_consensus.cc:2243] T bbfc4f99a5964e2ba6146e390e8dba44 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:20.899856 20365 leader_election.cc:290] T 70a86ea84d1d49d1b0950b4ce3a3d770 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:20.903927 19938 tablet_bootstrap.cc:654] T f50ea68e3b3c4459b3f5e5b0df724724 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:20.904574 19279 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_dffaefeede134bde97c6bcdff8bfc010"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:20.906046 19279 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dffaefeede134bde97c6bcdff8bfc010 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:20.908600 20409 raft_consensus.cc:515] T 1832cff5eb3e4da4b9766ae1591df81a P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.909226 19271 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_fc58dcc408944cbabdc5601c1f71623e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:20.910617 19271 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fc58dcc408944cbabdc5601c1f71623e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:20.913940 19280 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_83f9b6c4585f4edbb857f16696339ff3" } new_table_name: "default.Table_441a94250b5e4930a0b266c22acbda8a" modify_external_catalogs: true
I20260925 13:56:20.918365 19938 tablet_bootstrap.cc:492] T f50ea68e3b3c4459b3f5e5b0df724724 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:20.918836 19938 ts_tablet_manager.cc:1403] T f50ea68e3b3c4459b3f5e5b0df724724 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.026s user 0.011s sys 0.007s
I20260925 13:56:20.919915 19273 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_dfe9d9863d364dfc8df4c405e87dd635" } modify_external_catalogs: true
I20260925 13:56:20.920439 19273 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_dfe9d9863d364dfc8df4c405e87dd635" } modify_external_catalogs: true
I20260925 13:56:20.867396 19553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a1f34dba13143b097a38bab3782c404" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:20.923204 19938 raft_consensus.cc:359] T f50ea68e3b3c4459b3f5e5b0df724724 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:20.923839 19938 raft_consensus.cc:385] T f50ea68e3b3c4459b3f5e5b0df724724 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:20.924095 19938 raft_consensus.cc:740] T f50ea68e3b3c4459b3f5e5b0df724724 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:20.799810 19939 tablet_bootstrap.cc:654] T f514fa52e7524cd3ba07f3f65de532d4 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:20.900215 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "436b8d46e1d244deb8bb9dc128ee6279" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:20.901806 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a86ea84d1d49d1b0950b4ce3a3d770" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:20.925114 19938 consensus_queue.cc:260] T f50ea68e3b3c4459b3f5e5b0df724724 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:20.929024 19938 ts_tablet_manager.cc:1434] T f50ea68e3b3c4459b3f5e5b0df724724 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.010s user 0.008s sys 0.000s
I20260925 13:56:20.930444 20366 raft_consensus.cc:515] T 3e6c8b9bbf4549b3a8685f886973ed9c P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.931300 19520 tablet_service.cc:1558] Processing DeleteTablet for tablet 137c43fb74ec4cc48d30ed3edac81d98 with delete_type TABLET_DATA_DELETED (replaced by tablet 540132db164c403da72b9377078c4bf0) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:20.940585 19552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1832cff5eb3e4da4b9766ae1591df81a" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:20.938973 19629 leader_election.cc:304] T 2a1f34dba13143b097a38bab3782c404 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:20.941128 19552 raft_consensus.cc:3060] T 1832cff5eb3e4da4b9766ae1591df81a P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:20.941329 20413 raft_consensus.cc:2749] T 2a1f34dba13143b097a38bab3782c404 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:56:20.942407 19773 leader_election.cc:343] T 70a86ea84d1d49d1b0950b4ce3a3d770 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:20.942566 19695 raft_consensus.cc:2393] T 3e6c8b9bbf4549b3a8685f886973ed9c P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f578869bc084a5d8a9a69f143954eb3 in current term 1: Already voted for candidate 7c1b52895a14482fb54a6758f5054f8f in this term.
I20260925 13:56:20.942773 19773 leader_election.cc:304] T 70a86ea84d1d49d1b0950b4ce3a3d770 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
W20260925 13:56:20.943933 19773 leader_election.cc:343] T 436b8d46e1d244deb8bb9dc128ee6279 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:20.944164 20365 raft_consensus.cc:2749] T 70a86ea84d1d49d1b0950b4ce3a3d770 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:20.944406 19773 leader_election.cc:304] T 436b8d46e1d244deb8bb9dc128ee6279 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:20.945218 20365 raft_consensus.cc:2749] T 436b8d46e1d244deb8bb9dc128ee6279 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:20.933921 19938 tablet_bootstrap.cc:492] T 5788849b7336496488e5401b4ac91191 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:20.956175 19670 tablet_service.cc:1558] Processing DeleteTablet for tablet 137c43fb74ec4cc48d30ed3edac81d98 with delete_type TABLET_DATA_DELETED (replaced by tablet 540132db164c403da72b9377078c4bf0) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:20.956243 19838 raft_consensus.cc:2468] T 3e6c8b9bbf4549b3a8685f886973ed9c P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:20.959323 19942 tablet_bootstrap.cc:492] T 137c43fb74ec4cc48d30ed3edac81d98 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:20.959765 19942 ts_tablet_manager.cc:1403] T 137c43fb74ec4cc48d30ed3edac81d98 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.161s user 0.047s sys 0.072s
I20260925 13:56:20.960206 19938 tablet_bootstrap.cc:654] T 5788849b7336496488e5401b4ac91191 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:20.962118 19942 raft_consensus.cc:359] T 137c43fb74ec4cc48d30ed3edac81d98 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.962695 19942 raft_consensus.cc:385] T 137c43fb74ec4cc48d30ed3edac81d98 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:20.962958 19942 raft_consensus.cc:740] T 137c43fb74ec4cc48d30ed3edac81d98 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:20.963652 19942 consensus_queue.cc:260] T 137c43fb74ec4cc48d30ed3edac81d98 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.964687 19938 tablet_bootstrap.cc:492] T 5788849b7336496488e5401b4ac91191 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:20.965039 19938 ts_tablet_manager.cc:1403] T 5788849b7336496488e5401b4ac91191 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.031s user 0.002s sys 0.006s
I20260925 13:56:20.965298 20366 leader_election.cc:290] T 3e6c8b9bbf4549b3a8685f886973ed9c P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:20.967135 19938 raft_consensus.cc:359] T 5788849b7336496488e5401b4ac91191 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.967744 19938 raft_consensus.cc:385] T 5788849b7336496488e5401b4ac91191 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:20.968016 19938 raft_consensus.cc:740] T 5788849b7336496488e5401b4ac91191 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:20.968741 19938 consensus_queue.cc:260] T 5788849b7336496488e5401b4ac91191 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.969831 19838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e6c8b9bbf4549b3a8685f886973ed9c" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:20.970510 19838 raft_consensus.cc:2393] T 3e6c8b9bbf4549b3a8685f886973ed9c P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7c1b52895a14482fb54a6758f5054f8f in current term 1: Already voted for candidate 8f578869bc084a5d8a9a69f143954eb3 in this term.
I20260925 13:56:20.971441 19942 ts_tablet_manager.cc:1434] T 137c43fb74ec4cc48d30ed3edac81d98 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20260925 13:56:20.971601 19490 leader_election.cc:304] T 3e6c8b9bbf4549b3a8685f886973ed9c P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:20.971698 19276 catalog_manager.cc:6391] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: default.Table_4a822937f96346c1bd348b842fe60dfc [id=5bf634ef72d243b1bfcafc4efe14f28a] alter complete (version 1)
I20260925 13:56:20.972258 20338 raft_consensus.cc:2804] T 3e6c8b9bbf4549b3a8685f886973ed9c P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:20.972671 20338 raft_consensus.cc:697] T 3e6c8b9bbf4549b3a8685f886973ed9c P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Running, Role: LEADER
I20260925 13:56:20.973404 20338 consensus_queue.cc:237] T 3e6c8b9bbf4549b3a8685f886973ed9c P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.974509 19942 tablet_bootstrap.cc:492] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:20.978046 19939 tablet_bootstrap.cc:492] T f514fa52e7524cd3ba07f3f65de532d4 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:20.978415 19939 ts_tablet_manager.cc:1403] T f514fa52e7524cd3ba07f3f65de532d4 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.249s user 0.008s sys 0.000s
I20260925 13:56:20.983692 19942 tablet_bootstrap.cc:654] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:20.967669 19955 ts_tablet_manager.cc:1929] T 4d3dbd9796564472b44a31ccaf56d898 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:20.988250 19955 log.cc:1199] T 4d3dbd9796564472b44a31ccaf56d898 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/4d3dbd9796564472b44a31ccaf56d898
I20260925 13:56:20.988991 19955 ts_tablet_manager.cc:1950] T 4d3dbd9796564472b44a31ccaf56d898 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:20.994220 19942 tablet_bootstrap.cc:492] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:20.994582 19942 ts_tablet_manager.cc:1403] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.020s user 0.012s sys 0.002s
I20260925 13:56:20.996770 19942 raft_consensus.cc:359] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.997355 19942 raft_consensus.cc:385] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:20.997611 19942 raft_consensus.cc:740] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:20.992280 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e6c8b9bbf4549b3a8685f886973ed9c" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:20.998348 19942 consensus_queue.cc:260] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.997776 19939 raft_consensus.cc:359] T f514fa52e7524cd3ba07f3f65de532d4 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:20.999132 19939 raft_consensus.cc:385] T f514fa52e7524cd3ba07f3f65de532d4 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:20.999467 19939 raft_consensus.cc:740] T f514fa52e7524cd3ba07f3f65de532d4 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:21.000083 19939 consensus_queue.cc:260] T f514fa52e7524cd3ba07f3f65de532d4 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.002314 19939 ts_tablet_manager.cc:1434] T f514fa52e7524cd3ba07f3f65de532d4 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.024s user 0.006s sys 0.000s
I20260925 13:56:21.003134 19938 ts_tablet_manager.cc:1434] T 5788849b7336496488e5401b4ac91191 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.038s user 0.005s sys 0.001s
I20260925 13:56:21.005039 19942 ts_tablet_manager.cc:1434] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20260925 13:56:21.006366 19942 tablet_bootstrap.cc:492] T f514fa52e7524cd3ba07f3f65de532d4 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:21.011819 19942 tablet_bootstrap.cc:654] T f514fa52e7524cd3ba07f3f65de532d4 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:21.013176 19938 tablet_bootstrap.cc:492] T c161ae3edfed4f8488c770ecc6332607 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:21.018051 20409 leader_election.cc:290] T 1832cff5eb3e4da4b9766ae1591df81a P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:21.019986 19278 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 5a9b3dfaa7474611a1824d63eb906b43
W20260925 13:56:21.020530 19631 leader_election.cc:343] T 1832cff5eb3e4da4b9766ae1591df81a P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 1832cff5eb3e4da4b9766ae1591df81a
I20260925 13:56:21.021247 19278 catalog_manager.cc:6934] ReplaceTablet: tablet 5a9b3dfaa7474611a1824d63eb906b43 deleted and replaced by tablet a218f033a132482a82bd1f60126fa0f9
I20260925 13:56:21.022517 19818 tablet_service.cc:1558] Processing DeleteTablet for tablet 5a9b3dfaa7474611a1824d63eb906b43 with delete_type TABLET_DATA_DELETED (replaced by tablet a218f033a132482a82bd1f60126fa0f9) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:21.023568 19670 tablet_service.cc:1558] Processing DeleteTablet for tablet 5a9b3dfaa7474611a1824d63eb906b43 with delete_type TABLET_DATA_DELETED (replaced by tablet a218f033a132482a82bd1f60126fa0f9) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:21.025928 19278 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_ef5b8cfb027343188a849018298dbfa0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:21.027413 19278 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ef5b8cfb027343188a849018298dbfa0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:21.027802 19629 leader_election.cc:304] T 3e6c8b9bbf4549b3a8685f886973ed9c P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:21.028405 19938 tablet_bootstrap.cc:654] T c161ae3edfed4f8488c770ecc6332607 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:21.029469 20409 raft_consensus.cc:2749] T 3e6c8b9bbf4549b3a8685f886973ed9c P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260925 13:56:21.025298 20011 ts_tablet_manager.cc:1929] T 26644768f516412f8c95d68a76bb99c5 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:21.031399 20011 log.cc:1199] T 26644768f516412f8c95d68a76bb99c5 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/26644768f516412f8c95d68a76bb99c5
I20260925 13:56:21.032127 20011 ts_tablet_manager.cc:1950] T 26644768f516412f8c95d68a76bb99c5 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:21.033284 20365 raft_consensus.cc:493] T 2ef3efe644364e93a089ddeed328389d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:21.033980 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 136) took 1697 ms. Trace:
I20260925 13:56:21.033769 20365 raft_consensus.cc:515] T 2ef3efe644364e93a089ddeed328389d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.034260 20011 rpcz_store.cc:276] 0925 13:56:19.336211 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:19.339430 (+ 3219us) service_pool.cc:224] Handling call
0925 13:56:20.514198 (+1174768us) ts_tablet_manager.cc:1151] Deleting tablet 26644768f516412f8c95d68a76bb99c5
0925 13:56:20.514241 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:20.798478 (+284237us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:20.951335 (+152857us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:21.033929 (+ 82594us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1158852}
I20260925 13:56:21.035071 19942 tablet_bootstrap.cc:492] T f514fa52e7524cd3ba07f3f65de532d4 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:21.035539 19942 ts_tablet_manager.cc:1403] T f514fa52e7524cd3ba07f3f65de532d4 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.029s user 0.006s sys 0.006s
I20260925 13:56:21.035456 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 26644768f516412f8c95d68a76bb99c5 (table default.Table_bd52a86b8ff14bdbb06fcf88a60b1e82 [id=203024d67e444f839391c8670d7c7d3b]) successfully deleted
I20260925 13:56:21.036357 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ef3efe644364e93a089ddeed328389d" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:21.037693 19773 leader_election.cc:343] T 2ef3efe644364e93a089ddeed328389d P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:21.038506 19938 tablet_bootstrap.cc:492] T c161ae3edfed4f8488c770ecc6332607 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:21.038100 19942 raft_consensus.cc:359] T f514fa52e7524cd3ba07f3f65de532d4 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.038443 20365 leader_election.cc:290] T 2ef3efe644364e93a089ddeed328389d P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:21.038781 19942 raft_consensus.cc:385] T f514fa52e7524cd3ba07f3f65de532d4 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:21.038920 19938 ts_tablet_manager.cc:1403] T c161ae3edfed4f8488c770ecc6332607 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.026s user 0.007s sys 0.005s
I20260925 13:56:21.039017 19942 raft_consensus.cc:740] T f514fa52e7524cd3ba07f3f65de532d4 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:21.039647 19942 consensus_queue.cc:260] T f514fa52e7524cd3ba07f3f65de532d4 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.041168 19938 raft_consensus.cc:359] T c161ae3edfed4f8488c770ecc6332607 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:21.041574 19942 ts_tablet_manager.cc:1434] T f514fa52e7524cd3ba07f3f65de532d4 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260925 13:56:21.041770 19938 raft_consensus.cc:385] T c161ae3edfed4f8488c770ecc6332607 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:21.042201 19938 raft_consensus.cc:740] T c161ae3edfed4f8488c770ecc6332607 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:21.042846 19938 consensus_queue.cc:260] T c161ae3edfed4f8488c770ecc6332607 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:21.046785 19942 tablet_bootstrap.cc:492] T 12984db09f34472ba365fdf63a33b720 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:21.051537 19959 raft_consensus.cc:2272] T bbfc4f99a5964e2ba6146e390e8dba44 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:21.052199 19942 tablet_bootstrap.cc:654] T 12984db09f34472ba365fdf63a33b720 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:21.053584 19959 ts_tablet_manager.cc:1916] T bbfc4f99a5964e2ba6146e390e8dba44 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:21.058007 19938 ts_tablet_manager.cc:1434] T c161ae3edfed4f8488c770ecc6332607 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.019s user 0.002s sys 0.003s
I20260925 13:56:21.059523 19272 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 5788849b7336496488e5401b4ac91191
I20260925 13:56:21.061331 19272 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_48dae3cbc7b245408eecc2a1413b794b" } new_table_name: "default.Table_240f5ce2f31541eda914442e2cab887a" modify_external_catalogs: true
I20260925 13:56:21.063915 19818 tablet_service.cc:1558] Processing DeleteTablet for tablet 5788849b7336496488e5401b4ac91191 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:20 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:21.065158 19670 tablet_service.cc:1558] Processing DeleteTablet for tablet 5788849b7336496488e5401b4ac91191 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:20 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:21.069213 19938 tablet_bootstrap.cc:492] T 0ad6442b2de249119ba0ef85dfc06da9 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:21.072276 19939 tablet_bootstrap.cc:492] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:21.074760 19942 tablet_bootstrap.cc:492] T 12984db09f34472ba365fdf63a33b720 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:21.075218 19942 ts_tablet_manager.cc:1403] T 12984db09f34472ba365fdf63a33b720 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.029s user 0.008s sys 0.008s
I20260925 13:56:21.007738 19552 raft_consensus.cc:2468] T 1832cff5eb3e4da4b9766ae1591df81a P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
I20260925 13:56:21.017323 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 88) took 4744 ms. Trace:
I20260925 13:56:21.024719 19520 tablet_service.cc:1558] Processing DeleteTablet for tablet 5a9b3dfaa7474611a1824d63eb906b43 with delete_type TABLET_DATA_DELETED (replaced by tablet a218f033a132482a82bd1f60126fa0f9) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:21.075851 19938 tablet_bootstrap.cc:654] T 0ad6442b2de249119ba0ef85dfc06da9 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:21.043097 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ef3efe644364e93a089ddeed328389d" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:21.064357 19524 tablet_service.cc:1558] Processing DeleteTablet for tablet 5788849b7336496488e5401b4ac91191 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:20 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:21.076541 19955 rpcz_store.cc:276] 0925 13:56:16.273415 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:16.273530 (+ 115us) service_pool.cc:224] Handling call
0925 13:56:20.179542 (+3906012us) ts_tablet_manager.cc:1151] Deleting tablet 4d3dbd9796564472b44a31ccaf56d898
0925 13:56:20.179580 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:20.694391 (+514811us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:20.967546 (+273155us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:21.017264 (+ 49718us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":202742,"tablet-delete.queue_time_us":3905516}
I20260925 13:56:21.078105 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet 4d3dbd9796564472b44a31ccaf56d898 (table default.Table_1aa8069da6504812a85380a42ca07583 [id=a6d1215ba3be4fca903af9b17566b4a8]) successfully deleted
I20260925 13:56:21.078976 19955 tablet_replica.cc:333] T 1832cff5eb3e4da4b9766ae1591df81a P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:21.078545 19942 raft_consensus.cc:359] T 12984db09f34472ba365fdf63a33b720 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.079367 19942 raft_consensus.cc:385] T 12984db09f34472ba365fdf63a33b720 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:21.079684 19942 raft_consensus.cc:740] T 12984db09f34472ba365fdf63a33b720 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:21.079972 19955 raft_consensus.cc:2243] T 1832cff5eb3e4da4b9766ae1591df81a P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:21.080487 19955 raft_consensus.cc:2272] T 1832cff5eb3e4da4b9766ae1591df81a P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:21.080644 19942 consensus_queue.cc:260] T 12984db09f34472ba365fdf63a33b720 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.081724 20366 raft_consensus.cc:493] T f514fa52e7524cd3ba07f3f65de532d4 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:21.082216 20366 raft_consensus.cc:515] T f514fa52e7524cd3ba07f3f65de532d4 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.084592 20366 leader_election.cc:290] T f514fa52e7524cd3ba07f3f65de532d4 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:21.086760 19938 tablet_bootstrap.cc:492] T 0ad6442b2de249119ba0ef85dfc06da9 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:21.087101 19938 ts_tablet_manager.cc:1403] T 0ad6442b2de249119ba0ef85dfc06da9 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.008s
W20260925 13:56:21.088843 19631 leader_election.cc:343] T f514fa52e7524cd3ba07f3f65de532d4 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: f514fa52e7524cd3ba07f3f65de532d4
I20260925 13:56:21.089251 19938 raft_consensus.cc:359] T 0ad6442b2de249119ba0ef85dfc06da9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.089802 19938 raft_consensus.cc:385] T 0ad6442b2de249119ba0ef85dfc06da9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:21.090020 19938 raft_consensus.cc:740] T 0ad6442b2de249119ba0ef85dfc06da9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:21.090426 19942 ts_tablet_manager.cc:1434] T 12984db09f34472ba365fdf63a33b720 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.015s user 0.005s sys 0.000s
I20260925 13:56:21.091241 19942 tablet_bootstrap.cc:492] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:21.090855 19938 consensus_queue.cc:260] T 0ad6442b2de249119ba0ef85dfc06da9 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.098122 19942 tablet_bootstrap.cc:654] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
W20260925 13:56:21.101032 19488 consensus_peers.cc:597] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3 -> Peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Couldn't send request to peer 7c1b52895a14482fb54a6758f5054f8f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 26644768f516412f8c95d68a76bb99c5. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:21.103330 19629 leader_election.cc:304] T 1832cff5eb3e4da4b9766ae1591df81a P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3; no voters: f790493ba7ad403993b678f2df2b55be
I20260925 13:56:21.104067 20366 raft_consensus.cc:2804] T 1832cff5eb3e4da4b9766ae1591df81a P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:21.104293 19938 ts_tablet_manager.cc:1434] T 0ad6442b2de249119ba0ef85dfc06da9 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.017s user 0.005s sys 0.001s
I20260925 13:56:21.104492 20366 raft_consensus.cc:697] T 1832cff5eb3e4da4b9766ae1591df81a P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Becoming Leader. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Running, Role: LEADER
W20260925 13:56:21.104813 19773 leader_election.cc:343] T 2ef3efe644364e93a089ddeed328389d P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:21.105165 19773 leader_election.cc:304] T 2ef3efe644364e93a089ddeed328389d P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:21.105239 20366 consensus_queue.cc:237] T 1832cff5eb3e4da4b9766ae1591df81a P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.107779 20365 raft_consensus.cc:2749] T 2ef3efe644364e93a089ddeed328389d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:21.112738 20365 raft_consensus.cc:493] T 5788849b7336496488e5401b4ac91191 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:21.113114 20365 raft_consensus.cc:515] T 5788849b7336496488e5401b4ac91191 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.115391 20365 leader_election.cc:290] T 5788849b7336496488e5401b4ac91191 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:21.115715 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5788849b7336496488e5401b4ac91191" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:21.116945 19773 leader_election.cc:343] T 5788849b7336496488e5401b4ac91191 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:21.119477 19938 tablet_bootstrap.cc:492] T eb794cef052b43da831531455e2b5f15 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:21.119822 20011 tablet_replica.cc:333] T a88bbe1ebee7468d8f49dfd25637fe83 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:21.120543 20011 raft_consensus.cc:2243] T a88bbe1ebee7468d8f49dfd25637fe83 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:21.120961 20011 raft_consensus.cc:2272] T a88bbe1ebee7468d8f49dfd25637fe83 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:21.122517 20011 ts_tablet_manager.cc:1916] T a88bbe1ebee7468d8f49dfd25637fe83 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:21.126094 19938 tablet_bootstrap.cc:654] T eb794cef052b43da831531455e2b5f15 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:21.131847 20409 raft_consensus.cc:493] T 12984db09f34472ba365fdf63a33b720 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:21.132468 20409 raft_consensus.cc:515] T 12984db09f34472ba365fdf63a33b720 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.136368 19955 ts_tablet_manager.cc:1916] T 1832cff5eb3e4da4b9766ae1591df81a P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:21.136344 19838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12984db09f34472ba365fdf63a33b720" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:21.136885 19838 raft_consensus.cc:2468] T 12984db09f34472ba365fdf63a33b720 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:21.137229 19938 tablet_bootstrap.cc:492] T eb794cef052b43da831531455e2b5f15 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:21.137578 19938 ts_tablet_manager.cc:1403] T eb794cef052b43da831531455e2b5f15 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.018s user 0.012s sys 0.002s
I20260925 13:56:21.137578 20407 raft_consensus.cc:493] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:21.137982 20407 raft_consensus.cc:515] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.138959 19631 leader_election.cc:304] T 12984db09f34472ba365fdf63a33b720 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:21.139631 20413 raft_consensus.cc:2804] T 12984db09f34472ba365fdf63a33b720 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:21.139941 20413 raft_consensus.cc:493] T 12984db09f34472ba365fdf63a33b720 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:21.140210 20413 raft_consensus.cc:3060] T 12984db09f34472ba365fdf63a33b720 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:21.140280 19938 raft_consensus.cc:359] T eb794cef052b43da831531455e2b5f15 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.140831 19938 raft_consensus.cc:385] T eb794cef052b43da831531455e2b5f15 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:21.141057 19938 raft_consensus.cc:740] T eb794cef052b43da831531455e2b5f15 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:21.141832 19938 consensus_queue.cc:260] T eb794cef052b43da831531455e2b5f15 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.146776 20411 raft_consensus.cc:493] T 137c43fb74ec4cc48d30ed3edac81d98 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:21.147212 20411 raft_consensus.cc:515] T 137c43fb74ec4cc48d30ed3edac81d98 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.147704 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f514fa52e7524cd3ba07f3f65de532d4" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:21.148294 19554 raft_consensus.cc:2468] T f514fa52e7524cd3ba07f3f65de532d4 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:21.149118 19552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12984db09f34472ba365fdf63a33b720" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:21.149434 20411 leader_election.cc:290] T 137c43fb74ec4cc48d30ed3edac81d98 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:21.151201 19629 leader_election.cc:304] T f514fa52e7524cd3ba07f3f65de532d4 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3; no voters: f790493ba7ad403993b678f2df2b55be
I20260925 13:56:21.151492 19838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "137c43fb74ec4cc48d30ed3edac81d98" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:21.152019 19838 raft_consensus.cc:2468] T 137c43fb74ec4cc48d30ed3edac81d98 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:21.152808 19938 ts_tablet_manager.cc:1434] T eb794cef052b43da831531455e2b5f15 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.015s user 0.005s sys 0.000s
I20260925 13:56:21.153190 19631 leader_election.cc:304] T 137c43fb74ec4cc48d30ed3edac81d98 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:21.153308 20411 raft_consensus.cc:2804] T f514fa52e7524cd3ba07f3f65de532d4 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:21.153684 20411 raft_consensus.cc:493] T f514fa52e7524cd3ba07f3f65de532d4 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:21.153756 19552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5788849b7336496488e5401b4ac91191" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:21.154003 19938 tablet_bootstrap.cc:492] T 1215d875209a40d38cc527e2795d6c75 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:21.154045 20411 raft_consensus.cc:3060] T f514fa52e7524cd3ba07f3f65de532d4 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:21.157977 20413 raft_consensus.cc:515] T 12984db09f34472ba365fdf63a33b720 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.159519 19938 tablet_bootstrap.cc:654] T 1215d875209a40d38cc527e2795d6c75 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:21.160614 20407 leader_election.cc:290] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:21.161095 20407 raft_consensus.cc:2804] T 137c43fb74ec4cc48d30ed3edac81d98 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:21.161422 20407 raft_consensus.cc:493] T 137c43fb74ec4cc48d30ed3edac81d98 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:21.161067 20409 leader_election.cc:290] T 12984db09f34472ba365fdf63a33b720 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:21.161816 20407 raft_consensus.cc:3060] T 137c43fb74ec4cc48d30ed3edac81d98 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:21.164235 20413 leader_election.cc:290] T 12984db09f34472ba365fdf63a33b720 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:21.168552 20407 raft_consensus.cc:515] T 137c43fb74ec4cc48d30ed3edac81d98 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.170812 20407 leader_election.cc:290] T 137c43fb74ec4cc48d30ed3edac81d98 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:21.170836 19838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12984db09f34472ba365fdf63a33b720" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:21.171078 20338 raft_consensus.cc:493] T 137c43fb74ec4cc48d30ed3edac81d98 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:21.171380 19838 raft_consensus.cc:3060] T 12984db09f34472ba365fdf63a33b720 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:21.171552 20338 raft_consensus.cc:515] T 137c43fb74ec4cc48d30ed3edac81d98 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.173254 20338 leader_election.cc:290] T 137c43fb74ec4cc48d30ed3edac81d98 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:21.175239 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "137c43fb74ec4cc48d30ed3edac81d98" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:21.175932 19692 raft_consensus.cc:2393] T 137c43fb74ec4cc48d30ed3edac81d98 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f578869bc084a5d8a9a69f143954eb3 in current term 1: Already voted for candidate 7c1b52895a14482fb54a6758f5054f8f in this term.
W20260925 13:56:21.176272 19631 leader_election.cc:343] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 7ea4fd6817654bff9c0fe8c6e42ed6b9
I20260925 13:56:21.176832 19832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "137c43fb74ec4cc48d30ed3edac81d98" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:21.177320 19832 raft_consensus.cc:3060] T 137c43fb74ec4cc48d30ed3edac81d98 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:21.178822 19838 raft_consensus.cc:2468] T 12984db09f34472ba365fdf63a33b720 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
I20260925 13:56:21.161417 20411 raft_consensus.cc:515] T f514fa52e7524cd3ba07f3f65de532d4 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.179786 19631 leader_election.cc:304] T 12984db09f34472ba365fdf63a33b720 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:21.180801 20435 raft_consensus.cc:2804] T 12984db09f34472ba365fdf63a33b720 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:21.181278 20435 raft_consensus.cc:697] T 12984db09f34472ba365fdf63a33b720 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Becoming Leader. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Running, Role: LEADER
I20260925 13:56:21.181633 20411 leader_election.cc:290] T f514fa52e7524cd3ba07f3f65de532d4 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:21.185222 19832 raft_consensus.cc:2468] T 137c43fb74ec4cc48d30ed3edac81d98 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
I20260925 13:56:21.185240 20365 raft_consensus.cc:493] T 0ad6442b2de249119ba0ef85dfc06da9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:21.186100 20365 raft_consensus.cc:515] T 0ad6442b2de249119ba0ef85dfc06da9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.188604 20365 leader_election.cc:290] T 0ad6442b2de249119ba0ef85dfc06da9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
W20260925 13:56:21.191457 19631 leader_election.cc:343] T f514fa52e7524cd3ba07f3f65de532d4 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: f514fa52e7524cd3ba07f3f65de532d4
I20260925 13:56:21.192279 19631 leader_election.cc:304] T 137c43fb74ec4cc48d30ed3edac81d98 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:21.193188 20413 raft_consensus.cc:2804] T 137c43fb74ec4cc48d30ed3edac81d98 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:21.193718 20413 raft_consensus.cc:697] T 137c43fb74ec4cc48d30ed3edac81d98 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Becoming Leader. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Running, Role: LEADER
I20260925 13:56:21.195801 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad6442b2de249119ba0ef85dfc06da9" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:21.197624 19773 leader_election.cc:343] T 0ad6442b2de249119ba0ef85dfc06da9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:21.194695 20413 consensus_queue.cc:237] T 137c43fb74ec4cc48d30ed3edac81d98 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.182035 20435 consensus_queue.cc:237] T 12984db09f34472ba365fdf63a33b720 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.211041 19280 master_service.cc:960] ReplaceTablet: received request to replace tablet c1f15b9de572479aae6c2e644da3392a from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:21.212483 19942 tablet_bootstrap.cc:492] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:21.213177 19942 ts_tablet_manager.cc:1403] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.122s user 0.014s sys 0.004s
W20260925 13:56:21.221668 19629 leader_election.cc:343] T 12984db09f34472ba365fdf63a33b720 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:21.223865 19942 raft_consensus.cc:359] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.224610 19942 raft_consensus.cc:385] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:21.224882 19942 raft_consensus.cc:740] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:21.225528 19942 consensus_queue.cc:260] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.226897 19938 tablet_bootstrap.cc:492] T 1215d875209a40d38cc527e2795d6c75 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:21.227315 19938 ts_tablet_manager.cc:1403] T 1215d875209a40d38cc527e2795d6c75 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.074s user 0.029s sys 0.021s
I20260925 13:56:21.229614 19938 raft_consensus.cc:359] T 1215d875209a40d38cc527e2795d6c75 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:21.230221 19938 raft_consensus.cc:385] T 1215d875209a40d38cc527e2795d6c75 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:21.230695 19938 raft_consensus.cc:740] T 1215d875209a40d38cc527e2795d6c75 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:21.231685 20365 raft_consensus.cc:493] T c161ae3edfed4f8488c770ecc6332607 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:21.231619 19938 consensus_queue.cc:260] T 1215d875209a40d38cc527e2795d6c75 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
W20260925 13:56:21.232138 19773 leader_election.cc:343] T 5788849b7336496488e5401b4ac91191 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:21.232545 19773 leader_election.cc:304] T 5788849b7336496488e5401b4ac91191 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:21.232149 20365 raft_consensus.cc:515] T c161ae3edfed4f8488c770ecc6332607 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:21.235036 20365 leader_election.cc:290] T c161ae3edfed4f8488c770ecc6332607 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:21.235739 20365 raft_consensus.cc:2749] T 5788849b7336496488e5401b4ac91191 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:21.236483 19942 ts_tablet_manager.cc:1434] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.023s user 0.007s sys 0.000s
I20260925 13:56:21.237315 19942 tablet_bootstrap.cc:492] T 2ef3efe644364e93a089ddeed328389d P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:21.249670 19838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "137c43fb74ec4cc48d30ed3edac81d98" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:21.250612 19838 raft_consensus.cc:2393] T 137c43fb74ec4cc48d30ed3edac81d98 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f578869bc084a5d8a9a69f143954eb3 in current term 1: Already voted for candidate 7c1b52895a14482fb54a6758f5054f8f in this term.
I20260925 13:56:21.251636 19942 tablet_bootstrap.cc:654] T 2ef3efe644364e93a089ddeed328389d P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:21.253767 19939 tablet_bootstrap.cc:654] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:21.257498 19938 ts_tablet_manager.cc:1434] T 1215d875209a40d38cc527e2795d6c75 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.030s user 0.007s sys 0.004s
I20260925 13:56:21.262187 19490 leader_election.cc:304] T 137c43fb74ec4cc48d30ed3edac81d98 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3; no voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:21.263986 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c161ae3edfed4f8488c770ecc6332607" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:21.266007 19773 leader_election.cc:343] T c161ae3edfed4f8488c770ecc6332607 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:21.266925 19271 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_83acf26db75e4c02b312cef4df19901f" } modify_external_catalogs: true
I20260925 13:56:21.267467 19271 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_83acf26db75e4c02b312cef4df19901f" } modify_external_catalogs: true
I20260925 13:56:21.269048 19938 tablet_bootstrap.cc:492] T a8a679a453e0430b866edc707fe7c234 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:21.273522 19942 tablet_bootstrap.cc:492] T 2ef3efe644364e93a089ddeed328389d P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:21.273917 19942 ts_tablet_manager.cc:1403] T 2ef3efe644364e93a089ddeed328389d P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.037s user 0.001s sys 0.009s
I20260925 13:56:21.276546 19942 raft_consensus.cc:359] T 2ef3efe644364e93a089ddeed328389d P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.277127 19942 raft_consensus.cc:385] T 2ef3efe644364e93a089ddeed328389d P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:21.277383 19942 raft_consensus.cc:740] T 2ef3efe644364e93a089ddeed328389d P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:21.277952 19942 consensus_queue.cc:260] T 2ef3efe644364e93a089ddeed328389d P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.281904 19942 ts_tablet_manager.cc:1434] T 2ef3efe644364e93a089ddeed328389d P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.008s user 0.006s sys 0.001s
I20260925 13:56:21.282703 19942 tablet_bootstrap.cc:492] T 5788849b7336496488e5401b4ac91191 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:21.289476 19942 tablet_bootstrap.cc:654] T 5788849b7336496488e5401b4ac91191 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:21.293701 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "137c43fb74ec4cc48d30ed3edac81d98" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:21.294693 19552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad6442b2de249119ba0ef85dfc06da9" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:21.293632 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea4fd6817654bff9c0fe8c6e42ed6b9" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:21.305819 19553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12984db09f34472ba365fdf63a33b720" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
W20260925 13:56:21.316679 19773 leader_election.cc:343] T 0ad6442b2de249119ba0ef85dfc06da9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:21.316946 19629 leader_election.cc:343] T 12984db09f34472ba365fdf63a33b720 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:21.317126 19773 leader_election.cc:304] T 0ad6442b2de249119ba0ef85dfc06da9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:21.319123 19938 tablet_bootstrap.cc:654] T a8a679a453e0430b866edc707fe7c234 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:21.320286 19551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "137c43fb74ec4cc48d30ed3edac81d98" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:21.320871 19552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c161ae3edfed4f8488c770ecc6332607" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:21.321066 20441 raft_consensus.cc:2749] T 0ad6442b2de249119ba0ef85dfc06da9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:21.295557 20368 raft_consensus.cc:3060] T 137c43fb74ec4cc48d30ed3edac81d98 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:21.326224 19692 raft_consensus.cc:3060] T 2a1f34dba13143b097a38bab3782c404 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:21.328411 19279 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_5c6e964736a149d5b977b442a928a1ad"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:21.330211 19279 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5c6e964736a149d5b977b442a928a1ad in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:21.332268 19938 tablet_bootstrap.cc:492] T a8a679a453e0430b866edc707fe7c234 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:21.332640 19938 ts_tablet_manager.cc:1403] T a8a679a453e0430b866edc707fe7c234 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.064s user 0.007s sys 0.009s
I20260925 13:56:21.332397 19277 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_a2effd0e8c1d4199b1f4192ab3d781e7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:21.333750 19277 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a2effd0e8c1d4199b1f4192ab3d781e7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:21.334928 19938 raft_consensus.cc:359] T a8a679a453e0430b866edc707fe7c234 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.335698 19938 raft_consensus.cc:385] T a8a679a453e0430b866edc707fe7c234 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:21.337900 20369 consensus_queue.cc:1048] T 2a1f34dba13143b097a38bab3782c404 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:21.339600 19553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f514fa52e7524cd3ba07f3f65de532d4" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:21.340137 19553 raft_consensus.cc:3060] T f514fa52e7524cd3ba07f3f65de532d4 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:21.335952 19938 raft_consensus.cc:740] T a8a679a453e0430b866edc707fe7c234 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:21.342715 19938 consensus_queue.cc:260] T a8a679a453e0430b866edc707fe7c234 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:21.354914 19773 leader_election.cc:343] T c161ae3edfed4f8488c770ecc6332607 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:21.355312 19773 leader_election.cc:304] T c161ae3edfed4f8488c770ecc6332607 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:21.356451 20440 raft_consensus.cc:2749] T c161ae3edfed4f8488c770ecc6332607 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:21.360108 20440 raft_consensus.cc:493] T f50ea68e3b3c4459b3f5e5b0df724724 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:21.360612 20440 raft_consensus.cc:515] T f50ea68e3b3c4459b3f5e5b0df724724 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:21.363005 20440 leader_election.cc:290] T f50ea68e3b3c4459b3f5e5b0df724724 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:21.376417 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50ea68e3b3c4459b3f5e5b0df724724" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:21.377846 19773 leader_election.cc:343] T f50ea68e3b3c4459b3f5e5b0df724724 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:21.344587 19938 ts_tablet_manager.cc:1434] T a8a679a453e0430b866edc707fe7c234 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20260925 13:56:21.385898 19938 tablet_bootstrap.cc:492] T 5a9b3dfaa7474611a1824d63eb906b43 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
W20260925 13:56:21.392668 19272 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e4b29a3d0edc481cba6c61cfb78f8f40 has no consensus state
I20260925 13:56:21.393296 19938 tablet_bootstrap.cc:654] T 5a9b3dfaa7474611a1824d63eb906b43 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:21.393888 19280 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet c1f15b9de572479aae6c2e644da3392a
I20260925 13:56:21.395505 19280 catalog_manager.cc:6934] ReplaceTablet: tablet c1f15b9de572479aae6c2e644da3392a deleted and replaced by tablet 45ab2817d7e84185b7ec2bfddcf4997f
I20260925 13:56:21.395915 19818 tablet_service.cc:1558] Processing DeleteTablet for tablet c1f15b9de572479aae6c2e644da3392a with delete_type TABLET_DATA_DELETED (replaced by tablet 45ab2817d7e84185b7ec2bfddcf4997f) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:21.397650 19670 tablet_service.cc:1558] Processing DeleteTablet for tablet c1f15b9de572479aae6c2e644da3392a with delete_type TABLET_DATA_DELETED (replaced by tablet 45ab2817d7e84185b7ec2bfddcf4997f) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:21.400036 20413 raft_consensus.cc:493] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:21.400501 20413 raft_consensus.cc:515] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.402338 20413 leader_election.cc:290] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:21.404470 19832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dc5ea164d6749b0926c63d6bda9dcc5" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:21.405165 19832 raft_consensus.cc:2393] T 8dc5ea164d6749b0926c63d6bda9dcc5 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7c1b52895a14482fb54a6758f5054f8f in current term 1: Already voted for candidate 8f578869bc084a5d8a9a69f143954eb3 in this term.
W20260925 13:56:21.408828 19629 leader_election.cc:343] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:21.409286 19629 leader_election.cc:304] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:21.410392 20413 raft_consensus.cc:2749] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:21.410183 19280 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_9ae8fc862af64663b8f5f175a4085a03"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:21.412051 19490 consensus_peers.cc:597] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3 -> Peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Couldn't send request to peer f790493ba7ad403993b678f2df2b55be. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1513c0048bd244028d3dfa8d118bc14e. This is attempt 1: this message will repeat every 5th retry.
W20260925 13:56:21.412285 19280 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9ae8fc862af64663b8f5f175a4085a03 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:21.413961 19272 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_02ff478a0f98431cacbe0cb5cfba6e8c" } new_table_name: "default.Table_05aaf80be87b4d8489b392a7b7a382ea" modify_external_catalogs: true
I20260925 13:56:21.425702 20454 tablet.cc:1751] T 6b52ed579b19479686247d3ecba9489a P f790493ba7ad403993b678f2df2b55be: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 13:56:21.436213 19959 ts_tablet_manager.cc:1929] T bbfc4f99a5964e2ba6146e390e8dba44 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:21.436559 19959 log.cc:1199] T bbfc4f99a5964e2ba6146e390e8dba44 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/bbfc4f99a5964e2ba6146e390e8dba44
I20260925 13:56:21.437263 19959 ts_tablet_manager.cc:1950] T bbfc4f99a5964e2ba6146e390e8dba44 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:21.441746 20453 tablet.cc:1751] T 6b52ed579b19479686247d3ecba9489a P 7c1b52895a14482fb54a6758f5054f8f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260925 13:56:21.449656 19265 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet e4b29a3d0edc481cba6c61cfb78f8f40 (table default.Table_240f5ce2f31541eda914442e2cab887a [id=45fc848c759b4365a5c69016b6aa3309], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet e4b29a3d0edc481cba6c61cfb78f8f40 has no consensus state
I20260925 13:56:21.449589 20368 raft_consensus.cc:2749] T 137c43fb74ec4cc48d30ed3edac81d98 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:21.450533 19554 raft_consensus.cc:2468] T 137c43fb74ec4cc48d30ed3edac81d98 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
I20260925 13:56:21.452440 19942 tablet_bootstrap.cc:492] T 5788849b7336496488e5401b4ac91191 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:21.453624 19942 ts_tablet_manager.cc:1403] T 5788849b7336496488e5401b4ac91191 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.171s user 0.021s sys 0.028s
I20260925 13:56:21.458360 19942 raft_consensus.cc:359] T 5788849b7336496488e5401b4ac91191 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.459358 19524 tablet_service.cc:1558] Processing DeleteTablet for tablet c1f15b9de572479aae6c2e644da3392a with delete_type TABLET_DATA_DELETED (replaced by tablet 45ab2817d7e84185b7ec2bfddcf4997f) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:21.472020 20011 ts_tablet_manager.cc:1929] T a88bbe1ebee7468d8f49dfd25637fe83 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:21.472388 20011 log.cc:1199] T a88bbe1ebee7468d8f49dfd25637fe83 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/a88bbe1ebee7468d8f49dfd25637fe83
I20260925 13:56:21.472985 20011 ts_tablet_manager.cc:1950] T a88bbe1ebee7468d8f49dfd25637fe83 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:21.459152 19942 raft_consensus.cc:385] T 5788849b7336496488e5401b4ac91191 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:21.473598 19942 raft_consensus.cc:740] T 5788849b7336496488e5401b4ac91191 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:21.474273 19942 consensus_queue.cc:260] T 5788849b7336496488e5401b4ac91191 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.474704 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 131) took 2226 ms. Trace:
I20260925 13:56:21.439152 19959 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 143) took 1286 ms. Trace:
I20260925 13:56:21.475524 19959 rpcz_store.cc:276] 0925 13:56:20.153093 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:20.153198 (+ 105us) service_pool.cc:224] Handling call
0925 13:56:20.898244 (+745046us) ts_tablet_manager.cc:1151] Deleting tablet bbfc4f99a5964e2ba6146e390e8dba44
0925 13:56:20.898280 (+ 36us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:21.271879 (+373599us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:21.436076 (+164197us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:21.439099 (+ 3023us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":744495}
I20260925 13:56:21.476310 19942 ts_tablet_manager.cc:1434] T 5788849b7336496488e5401b4ac91191 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.022s user 0.007s sys 0.000s
I20260925 13:56:21.477211 19959 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 144) took 1323 ms. Trace:
I20260925 13:56:21.477511 19959 rpcz_store.cc:276] 0925 13:56:20.153358 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:20.153830 (+ 472us) service_pool.cc:224] Handling call
0925 13:56:21.476858 (+1323028us) ts_tablet_manager.cc:1151] Deleting tablet a26f5a02311c435a8c2434fd030c80a6
0925 13:56:21.476902 (+ 44us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:21.477155 (+ 253us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1322535}
I20260925 13:56:21.480968 19553 raft_consensus.cc:2468] T f514fa52e7524cd3ba07f3f65de532d4 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
I20260925 13:56:21.486954 20413 raft_consensus.cc:493] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:21.487545 20413 raft_consensus.cc:515] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.489986 20413 leader_election.cc:290] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:21.490490 19670 tablet_service.cc:1511] Processing CreateTablet for tablet 7e544c93488541da9ebb588a465b27b7 (DEFAULT_TABLE table=default.Table_8aa1581769ff4b9bb2e158a3b0430a4f [id=f71e790b94b34e3d88f7539e396bc94d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:21.491652 19670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e544c93488541da9ebb588a465b27b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:21.495229 19524 tablet_service.cc:1511] Processing CreateTablet for tablet 7e544c93488541da9ebb588a465b27b7 (DEFAULT_TABLE table=default.Table_8aa1581769ff4b9bb2e158a3b0430a4f [id=f71e790b94b34e3d88f7539e396bc94d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:21.496344 19524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e544c93488541da9ebb588a465b27b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:21.500854 19942 tablet_bootstrap.cc:492] T 28219970809145bc9d4e2a3289a9479a P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
W20260925 13:56:21.504359 19631 leader_election.cc:343] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 0de8f2d2766c4ff197b552a1b5ed3d6b
I20260925 13:56:21.492202 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet bbfc4f99a5964e2ba6146e390e8dba44 (table default.Table_5d692242213f4df6863089b93b4e9636 [id=6d79107dc2d941058c5ecbe9a4410843]) successfully deleted
I20260925 13:56:21.506385 20441 raft_consensus.cc:493] T e3cf9676160c4af08f1f62ae952b4c57 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:21.506865 20441 raft_consensus.cc:515] T e3cf9676160c4af08f1f62ae952b4c57 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:21.506533 19266 catalog_manager.cc:5043] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): delete failed for tablet a26f5a02311c435a8c2434fd030c80a6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a26f5a02311c435a8c2434fd030c80a6 already in progress: creating tablet
I20260925 13:56:21.509119 20441 leader_election.cc:290] T e3cf9676160c4af08f1f62ae952b4c57 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:21.514611 19818 tablet_service.cc:1511] Processing CreateTablet for tablet 7e544c93488541da9ebb588a465b27b7 (DEFAULT_TABLE table=default.Table_8aa1581769ff4b9bb2e158a3b0430a4f [id=f71e790b94b34e3d88f7539e396bc94d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:21.515583 19818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e544c93488541da9ebb588a465b27b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:21.519544 19955 ts_tablet_manager.cc:1929] T 1832cff5eb3e4da4b9766ae1591df81a P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:21.519913 19955 log.cc:1199] T 1832cff5eb3e4da4b9766ae1591df81a P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/1832cff5eb3e4da4b9766ae1591df81a
I20260925 13:56:21.520618 19955 ts_tablet_manager.cc:1950] T 1832cff5eb3e4da4b9766ae1591df81a P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:21.527777 19959 tablet_replica.cc:333] T e6e59d6f4f6e4de1be1cf450620cfa59 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:21.528542 19959 raft_consensus.cc:2243] T e6e59d6f4f6e4de1be1cf450620cfa59 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:21.529019 19959 raft_consensus.cc:2272] T e6e59d6f4f6e4de1be1cf450620cfa59 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:21.540915 19939 tablet_bootstrap.cc:492] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:21.541342 19939 ts_tablet_manager.cc:1403] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.469s user 0.000s sys 0.008s
W20260925 13:56:21.542131 19490 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 98865.7us
I20260925 13:56:21.545258 19942 tablet_bootstrap.cc:654] T 28219970809145bc9d4e2a3289a9479a P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:21.548084 19696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3cf9676160c4af08f1f62ae952b4c57" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:21.550527 19773 leader_election.cc:343] T e3cf9676160c4af08f1f62ae952b4c57 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:21.552963 19939 raft_consensus.cc:359] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.553592 19939 raft_consensus.cc:385] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:21.553834 19939 raft_consensus.cc:740] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:21.561560 19939 consensus_queue.cc:260] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.565521 19959 ts_tablet_manager.cc:1916] T e6e59d6f4f6e4de1be1cf450620cfa59 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:21.566913 19942 tablet_bootstrap.cc:492] T 28219970809145bc9d4e2a3289a9479a P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:21.567296 19942 ts_tablet_manager.cc:1403] T 28219970809145bc9d4e2a3289a9479a P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.067s user 0.010s sys 0.001s
I20260925 13:56:21.568886 19520 tablet_service.cc:1511] Processing CreateTablet for tablet b23948807b2c4df9bf9a4da5f767e3b5 (DEFAULT_TABLE table=default.Table_98879cfd535843ff9cbc54221b3cf028 [id=0165d602da03423f82104b16da9a20b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:21.569485 19942 raft_consensus.cc:359] T 28219970809145bc9d4e2a3289a9479a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.569926 19520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b23948807b2c4df9bf9a4da5f767e3b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:21.570112 19942 raft_consensus.cc:385] T 28219970809145bc9d4e2a3289a9479a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:21.570382 19942 raft_consensus.cc:740] T 28219970809145bc9d4e2a3289a9479a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:21.571007 19942 consensus_queue.cc:260] T 28219970809145bc9d4e2a3289a9479a P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.570307 19670 tablet_service.cc:1511] Processing CreateTablet for tablet b23948807b2c4df9bf9a4da5f767e3b5 (DEFAULT_TABLE table=default.Table_98879cfd535843ff9cbc54221b3cf028 [id=0165d602da03423f82104b16da9a20b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:21.572772 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 90) took 5003 ms. Trace:
I20260925 13:56:21.573073 19955 rpcz_store.cc:276] 0925 13:56:16.569040 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:16.569229 (+ 189us) service_pool.cc:224] Handling call
0925 13:56:21.078828 (+4509599us) ts_tablet_manager.cc:1151] Deleting tablet 1832cff5eb3e4da4b9766ae1591df81a
0925 13:56:21.078869 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:21.347382 (+268513us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:21.519417 (+172035us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:21.572714 (+ 53297us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50307,"tablet-delete.queue_time_us":4509074}
I20260925 13:56:21.574250 19955 tablet_replica.cc:333] T dc14c37aa393407a87af0d0eb8b88d82 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:21.574939 19955 raft_consensus.cc:2243] T dc14c37aa393407a87af0d0eb8b88d82 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:21.575803 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet 1832cff5eb3e4da4b9766ae1591df81a (table default.Table_9896a33d94e44d76ada6ec8d00c87ad7 [id=a5549f9e40c644afa1fafcc16d649f68]) successfully deleted
I20260925 13:56:21.577849 19670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b23948807b2c4df9bf9a4da5f767e3b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:21.573241 19942 ts_tablet_manager.cc:1434] T 28219970809145bc9d4e2a3289a9479a P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260925 13:56:21.580746 19819 tablet_service.cc:1511] Processing CreateTablet for tablet b23948807b2c4df9bf9a4da5f767e3b5 (DEFAULT_TABLE table=default.Table_98879cfd535843ff9cbc54221b3cf028 [id=0165d602da03423f82104b16da9a20b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:21.581673 19819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b23948807b2c4df9bf9a4da5f767e3b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:21.582974 19942 tablet_bootstrap.cc:492] T e3cf9676160c4af08f1f62ae952b4c57 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:21.595286 19942 tablet_bootstrap.cc:654] T e3cf9676160c4af08f1f62ae952b4c57 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:21.614850 19939 ts_tablet_manager.cc:1434] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.073s user 0.005s sys 0.002s
I20260925 13:56:21.615077 19942 tablet_bootstrap.cc:492] T e3cf9676160c4af08f1f62ae952b4c57 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:21.615765 19939 tablet_bootstrap.cc:492] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:21.615962 19942 ts_tablet_manager.cc:1403] T e3cf9676160c4af08f1f62ae952b4c57 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.033s user 0.009s sys 0.002s
I20260925 13:56:21.618184 19942 raft_consensus.cc:359] T e3cf9676160c4af08f1f62ae952b4c57 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.618759 19942 raft_consensus.cc:385] T e3cf9676160c4af08f1f62ae952b4c57 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:21.618992 19942 raft_consensus.cc:740] T e3cf9676160c4af08f1f62ae952b4c57 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:21.621210 19670 tablet_service.cc:1511] Processing CreateTablet for tablet 82c1002d2bcb4b11839e2f4a93b899ad (DEFAULT_TABLE table=default.Table_271e58fa014d4b7f971f2b784b87fce6 [id=d889d7a56c614b36bdc3a866556e58d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:21.622079 19670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82c1002d2bcb4b11839e2f4a93b899ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:21.624447 19813 tablet_service.cc:1511] Processing CreateTablet for tablet 82c1002d2bcb4b11839e2f4a93b899ad (DEFAULT_TABLE table=default.Table_271e58fa014d4b7f971f2b784b87fce6 [id=d889d7a56c614b36bdc3a866556e58d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:21.625448 19813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82c1002d2bcb4b11839e2f4a93b899ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:21.475010 20011 rpcz_store.cc:276] 0925 13:56:19.248732 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:19.248843 (+ 111us) service_pool.cc:224] Handling call
0925 13:56:21.119663 (+1870820us) ts_tablet_manager.cc:1151] Deleting tablet a88bbe1ebee7468d8f49dfd25637fe83
0925 13:56:21.119704 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:21.302430 (+182726us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:21.471894 (+169464us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:21.474650 (+ 2756us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1764084}
I20260925 13:56:21.632539 19277 master_service.cc:960] ReplaceTablet: received request to replace tablet 78d8c01c88d24deeae21d647e7f205ba from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:21.635795 20011 tablet_replica.cc:333] T 1513c0048bd244028d3dfa8d118bc14e P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:21.619707 19942 consensus_queue.cc:260] T e3cf9676160c4af08f1f62ae952b4c57 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.636605 20011 raft_consensus.cc:2243] T 1513c0048bd244028d3dfa8d118bc14e P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:21.638072 19942 ts_tablet_manager.cc:1434] T e3cf9676160c4af08f1f62ae952b4c57 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.022s user 0.001s sys 0.003s
I20260925 13:56:21.639776 20011 raft_consensus.cc:2272] T 1513c0048bd244028d3dfa8d118bc14e P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:21.641465 19669 tablet_service.cc:1511] Processing CreateTablet for tablet 1ce5d15e67644247b95960f444acc621 (DEFAULT_TABLE table=default.Table_977caf871df4415b9855949376feed24 [id=d132b2c46a9a40108112e0e16755ada7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:21.642829 19955 raft_consensus.cc:2272] T dc14c37aa393407a87af0d0eb8b88d82 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:21.644800 19818 tablet_service.cc:1511] Processing CreateTablet for tablet 1ce5d15e67644247b95960f444acc621 (DEFAULT_TABLE table=default.Table_977caf871df4415b9855949376feed24 [id=d132b2c46a9a40108112e0e16755ada7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:21.644657 19279 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_7046f5be3ed44b3e94bd4d14e34c25ee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:21.645704 19818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ce5d15e67644247b95960f444acc621. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:21.645864 19669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ce5d15e67644247b95960f444acc621. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:21.646065 19279 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7046f5be3ed44b3e94bd4d14e34c25ee in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:21.652554 19955 ts_tablet_manager.cc:1916] T dc14c37aa393407a87af0d0eb8b88d82 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:21.656945 19938 tablet_bootstrap.cc:492] T 5a9b3dfaa7474611a1824d63eb906b43 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:21.657325 19938 ts_tablet_manager.cc:1403] T 5a9b3dfaa7474611a1824d63eb906b43 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.272s user 0.066s sys 0.077s
I20260925 13:56:21.659758 19942 tablet_bootstrap.cc:492] T c161ae3edfed4f8488c770ecc6332607 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:21.659580 19938 raft_consensus.cc:359] T 5a9b3dfaa7474611a1824d63eb906b43 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.660189 19938 raft_consensus.cc:385] T 5a9b3dfaa7474611a1824d63eb906b43 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:21.660444 19938 raft_consensus.cc:740] T 5a9b3dfaa7474611a1824d63eb906b43 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:21.661062 19938 consensus_queue.cc:260] T 5a9b3dfaa7474611a1824d63eb906b43 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.661904 20011 ts_tablet_manager.cc:1916] T 1513c0048bd244028d3dfa8d118bc14e P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:21.664358 19938 ts_tablet_manager.cc:1434] T 5a9b3dfaa7474611a1824d63eb906b43 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.007s user 0.006s sys 0.001s
I20260925 13:56:21.684335 19938 tablet_bootstrap.cc:492] T 3aaf29a2310a41d0971eded7ef51fdc9 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:21.691967 20011 ts_tablet_manager.cc:1929] T 1513c0048bd244028d3dfa8d118bc14e P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:56:21.692301 20011 log.cc:1199] T 1513c0048bd244028d3dfa8d118bc14e P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/1513c0048bd244028d3dfa8d118bc14e
I20260925 13:56:21.692972 20011 ts_tablet_manager.cc:1950] T 1513c0048bd244028d3dfa8d118bc14e P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:21.695125 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 138) took 2266 ms. Trace:
I20260925 13:56:21.695421 20011 rpcz_store.cc:276] 0925 13:56:19.428423 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:19.428866 (+ 443us) service_pool.cc:224] Handling call
0925 13:56:21.635648 (+2206782us) ts_tablet_manager.cc:1151] Deleting tablet 1513c0048bd244028d3dfa8d118bc14e
0925 13:56:21.635691 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:21.679774 (+ 44083us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:21.691858 (+ 12084us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:21.695072 (+ 3214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2496,"tablet-delete.queue_time_us":2205904}
I20260925 13:56:21.696625 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 1513c0048bd244028d3dfa8d118bc14e (table default.Table_74b6076b22674a31bc58fef71cc9bfb1 [id=d81cf07e5f9a49d4bbf5aceda3da5a25]) successfully deleted
I20260925 13:56:21.697430 20011 tablet_replica.cc:333] T bbfc4f99a5964e2ba6146e390e8dba44 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:21.698082 20011 raft_consensus.cc:2243] T bbfc4f99a5964e2ba6146e390e8dba44 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:21.698586 20011 raft_consensus.cc:2272] T bbfc4f99a5964e2ba6146e390e8dba44 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:21.701808 19938 tablet_bootstrap.cc:654] T 3aaf29a2310a41d0971eded7ef51fdc9 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:21.704747 19942 tablet_bootstrap.cc:654] T c161ae3edfed4f8488c770ecc6332607 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:21.708624 20011 ts_tablet_manager.cc:1916] T bbfc4f99a5964e2ba6146e390e8dba44 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:21.719754 19959 ts_tablet_manager.cc:1929] T e6e59d6f4f6e4de1be1cf450620cfa59 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:21.720182 19959 log.cc:1199] T e6e59d6f4f6e4de1be1cf450620cfa59 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/e6e59d6f4f6e4de1be1cf450620cfa59
I20260925 13:56:21.720947 19959 ts_tablet_manager.cc:1950] T e6e59d6f4f6e4de1be1cf450620cfa59 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:21.722107 19280 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_fc58dcc408944cbabdc5601c1f71623e" } new_table_name: "default.Table_92f5e6ebf342495b8333c8da779d6998" modify_external_catalogs: true
I20260925 13:56:21.722999 19959 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 145) took 1363 ms. Trace:
I20260925 13:56:21.729128 19942 tablet_bootstrap.cc:492] T c161ae3edfed4f8488c770ecc6332607 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:21.729707 19942 ts_tablet_manager.cc:1403] T c161ae3edfed4f8488c770ecc6332607 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.070s user 0.008s sys 0.008s
I20260925 13:56:21.731962 19942 raft_consensus.cc:359] T c161ae3edfed4f8488c770ecc6332607 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:21.732568 19942 raft_consensus.cc:385] T c161ae3edfed4f8488c770ecc6332607 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:21.732815 19942 raft_consensus.cc:740] T c161ae3edfed4f8488c770ecc6332607 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:21.735342 19938 tablet_bootstrap.cc:492] T 3aaf29a2310a41d0971eded7ef51fdc9 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:21.735751 19938 ts_tablet_manager.cc:1403] T 3aaf29a2310a41d0971eded7ef51fdc9 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.052s user 0.002s sys 0.007s
I20260925 13:56:21.735730 20407 raft_consensus.cc:493] T 28219970809145bc9d4e2a3289a9479a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:21.736129 20407 raft_consensus.cc:515] T 28219970809145bc9d4e2a3289a9479a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.737948 20407 leader_election.cc:290] T 28219970809145bc9d4e2a3289a9479a P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:21.723310 19959 rpcz_store.cc:276] 0925 13:56:20.359525 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:20.359652 (+ 127us) service_pool.cc:224] Handling call
0925 13:56:21.527603 (+1167951us) ts_tablet_manager.cc:1151] Deleting tablet e6e59d6f4f6e4de1be1cf450620cfa59
0925 13:56:21.527650 (+ 47us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:21.608313 (+ 80663us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:21.695315 (+ 87002us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:21.722934 (+ 27619us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1167350}
W20260925 13:56:21.738072 19490 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 195107us
I20260925 13:56:21.744307 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet e6e59d6f4f6e4de1be1cf450620cfa59 (table default.Table_546097946d054781b0acae5186c5d43d [id=912de4c02216461f8a69b880207d8652]) successfully deleted
I20260925 13:56:21.745019 19942 consensus_queue.cc:260] T c161ae3edfed4f8488c770ecc6332607 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:21.745447 19832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28219970809145bc9d4e2a3289a9479a" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:21.746264 19832 raft_consensus.cc:2468] T 28219970809145bc9d4e2a3289a9479a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:21.747690 19959 tablet_replica.cc:333] T 12984db09f34472ba365fdf63a33b720 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:21.747691 19629 leader_election.cc:304] T f514fa52e7524cd3ba07f3f65de532d4 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3; no voters: f790493ba7ad403993b678f2df2b55be
I20260925 13:56:21.748452 19959 raft_consensus.cc:2243] T 12984db09f34472ba365fdf63a33b720 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:21.748483 20407 raft_consensus.cc:2804] T f514fa52e7524cd3ba07f3f65de532d4 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:21.747699 19631 leader_election.cc:304] T 28219970809145bc9d4e2a3289a9479a P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:21.748868 19959 raft_consensus.cc:2272] T 12984db09f34472ba365fdf63a33b720 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:21.748929 20407 raft_consensus.cc:697] T f514fa52e7524cd3ba07f3f65de532d4 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Becoming Leader. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Running, Role: LEADER
I20260925 13:56:21.749675 19271 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 2711e7a6973546b1822d93f91095dae2
I20260925 13:56:21.749881 20435 raft_consensus.cc:2804] T 28219970809145bc9d4e2a3289a9479a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:21.750186 20435 raft_consensus.cc:493] T 28219970809145bc9d4e2a3289a9479a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:21.750504 20435 raft_consensus.cc:3060] T 28219970809145bc9d4e2a3289a9479a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:21.755615 19669 tablet_service.cc:1558] Processing DeleteTablet for tablet 2711e7a6973546b1822d93f91095dae2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:21 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:21.757478 19818 tablet_service.cc:1558] Processing DeleteTablet for tablet 2711e7a6973546b1822d93f91095dae2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:21 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:21.759894 20407 consensus_queue.cc:237] T f514fa52e7524cd3ba07f3f65de532d4 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.764158 20435 raft_consensus.cc:515] T 28219970809145bc9d4e2a3289a9479a P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:21.763284 19490 consensus_peers.cc:597] T 2a1f34dba13143b097a38bab3782c404 P 8f578869bc084a5d8a9a69f143954eb3 -> Peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Couldn't send request to peer f790493ba7ad403993b678f2df2b55be. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2a1f34dba13143b097a38bab3782c404. This is attempt 1: this message will repeat every 5th retry.
W20260925 13:56:21.766285 19490 consensus_peers.cc:597] T bbfc4f99a5964e2ba6146e390e8dba44 P 8f578869bc084a5d8a9a69f143954eb3 -> Peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Couldn't send request to peer f790493ba7ad403993b678f2df2b55be. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbfc4f99a5964e2ba6146e390e8dba44. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:21.767925 19833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28219970809145bc9d4e2a3289a9479a" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:21.768435 19833 raft_consensus.cc:3060] T 28219970809145bc9d4e2a3289a9479a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:21.768357 19938 raft_consensus.cc:359] T 3aaf29a2310a41d0971eded7ef51fdc9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.768772 19942 ts_tablet_manager.cc:1434] T c161ae3edfed4f8488c770ecc6332607 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.039s user 0.000s sys 0.004s
I20260925 13:56:21.768955 19938 raft_consensus.cc:385] T 3aaf29a2310a41d0971eded7ef51fdc9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:21.769207 19938 raft_consensus.cc:740] T 3aaf29a2310a41d0971eded7ef51fdc9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:21.769950 20435 leader_election.cc:290] T 28219970809145bc9d4e2a3289a9479a P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:21.769775 19938 consensus_queue.cc:260] T 3aaf29a2310a41d0971eded7ef51fdc9 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.772643 19959 ts_tablet_manager.cc:1916] T 12984db09f34472ba365fdf63a33b720 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:21.775049 19833 raft_consensus.cc:2468] T 28219970809145bc9d4e2a3289a9479a P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
I20260925 13:56:21.776427 19631 leader_election.cc:304] T 28219970809145bc9d4e2a3289a9479a P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:21.777159 20435 raft_consensus.cc:2804] T 28219970809145bc9d4e2a3289a9479a P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:21.780073 20435 raft_consensus.cc:697] T 28219970809145bc9d4e2a3289a9479a P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Becoming Leader. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Running, Role: LEADER
I20260925 13:56:21.780813 20435 consensus_queue.cc:237] T 28219970809145bc9d4e2a3289a9479a P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.782609 19942 tablet_bootstrap.cc:492] T 0ad6442b2de249119ba0ef85dfc06da9 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:21.787786 19518 tablet_service.cc:1558] Processing DeleteTablet for tablet 2711e7a6973546b1822d93f91095dae2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:21 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:21.804487 19522 tablet_service.cc:1511] Processing CreateTablet for tablet 82c1002d2bcb4b11839e2f4a93b899ad (DEFAULT_TABLE table=default.Table_271e58fa014d4b7f971f2b784b87fce6 [id=d889d7a56c614b36bdc3a866556e58d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:21.805833 19522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82c1002d2bcb4b11839e2f4a93b899ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:21.805917 20011 ts_tablet_manager.cc:1929] T bbfc4f99a5964e2ba6146e390e8dba44 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:21.806262 20011 log.cc:1199] T bbfc4f99a5964e2ba6146e390e8dba44 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/bbfc4f99a5964e2ba6146e390e8dba44
I20260925 13:56:21.806948 19938 ts_tablet_manager.cc:1434] T 3aaf29a2310a41d0971eded7ef51fdc9 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.071s user 0.005s sys 0.000s
I20260925 13:56:21.807837 19938 tablet_bootstrap.cc:492] T ec7fe5c834be4a51b627b8b1c3ebe888 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:21.807953 19942 tablet_bootstrap.cc:654] T 0ad6442b2de249119ba0ef85dfc06da9 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:21.806905 20011 ts_tablet_manager.cc:1950] T bbfc4f99a5964e2ba6146e390e8dba44 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:21.810755 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 143) took 1807 ms. Trace:
I20260925 13:56:21.811030 20011 rpcz_store.cc:276] 0925 13:56:20.002897 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:20.003022 (+ 125us) service_pool.cc:224] Handling call
0925 13:56:21.697295 (+1694273us) ts_tablet_manager.cc:1151] Deleting tablet bbfc4f99a5964e2ba6146e390e8dba44
0925 13:56:21.697335 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:21.734927 (+ 37592us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:21.805807 (+ 70880us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:21.810701 (+ 4894us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1693373}
I20260925 13:56:21.812158 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet bbfc4f99a5964e2ba6146e390e8dba44 (table default.Table_5d692242213f4df6863089b93b4e9636 [id=6d79107dc2d941058c5ecbe9a4410843]) successfully deleted
I20260925 13:56:21.816396 19277 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 78d8c01c88d24deeae21d647e7f205ba
I20260925 13:56:21.817880 19277 catalog_manager.cc:6934] ReplaceTablet: tablet 78d8c01c88d24deeae21d647e7f205ba deleted and replaced by tablet ab1ba6ef73a046b59ffac150f7c73953
I20260925 13:56:21.817889 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 144) took 1789 ms. Trace:
I20260925 13:56:21.819197 19669 tablet_service.cc:1558] Processing DeleteTablet for tablet 78d8c01c88d24deeae21d647e7f205ba with delete_type TABLET_DATA_DELETED (replaced by tablet ab1ba6ef73a046b59ffac150f7c73953) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:21.822775 19942 tablet_bootstrap.cc:492] T 0ad6442b2de249119ba0ef85dfc06da9 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:21.823138 19942 ts_tablet_manager.cc:1403] T 0ad6442b2de249119ba0ef85dfc06da9 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.047s user 0.008s sys 0.002s
I20260925 13:56:21.824452 19518 tablet_service.cc:1558] Processing DeleteTablet for tablet 78d8c01c88d24deeae21d647e7f205ba with delete_type TABLET_DATA_DELETED (replaced by tablet ab1ba6ef73a046b59ffac150f7c73953) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:21.825150 19818 tablet_service.cc:1558] Processing DeleteTablet for tablet 78d8c01c88d24deeae21d647e7f205ba with delete_type TABLET_DATA_DELETED (replaced by tablet ab1ba6ef73a046b59ffac150f7c73953) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:21.818152 20011 rpcz_store.cc:276] 0925 13:56:20.028022 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:20.028154 (+ 132us) service_pool.cc:224] Handling call
0925 13:56:21.817570 (+1789416us) ts_tablet_manager.cc:1151] Deleting tablet a26f5a02311c435a8c2434fd030c80a6
0925 13:56:21.817610 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:21.817836 (+ 226us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1788867}
I20260925 13:56:21.826336 19547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28219970809145bc9d4e2a3289a9479a" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:21.826572 19532 tablet_service.cc:1511] Processing CreateTablet for tablet 1ce5d15e67644247b95960f444acc621 (DEFAULT_TABLE table=default.Table_977caf871df4415b9855949376feed24 [id=d132b2c46a9a40108112e0e16755ada7]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 13:56:21.826828 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet a26f5a02311c435a8c2434fd030c80a6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a26f5a02311c435a8c2434fd030c80a6 already in progress: creating tablet
I20260925 13:56:21.827848 20011 tablet_replica.cc:333] T c2d333f892f84a258f8a148a90ce3b06 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:21.828004 19548 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28219970809145bc9d4e2a3289a9479a" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:21.828593 20011 raft_consensus.cc:2243] T c2d333f892f84a258f8a148a90ce3b06 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:21.829041 20011 raft_consensus.cc:2272] T c2d333f892f84a258f8a148a90ce3b06 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:21.830711 20011 ts_tablet_manager.cc:1916] T c2d333f892f84a258f8a148a90ce3b06 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:21.832077 19551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0de8f2d2766c4ff197b552a1b5ed3d6b" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:21.833397 19552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dc5ea164d6749b0926c63d6bda9dcc5" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:21.825515 19942 raft_consensus.cc:359] T 0ad6442b2de249119ba0ef85dfc06da9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.834730 19942 raft_consensus.cc:385] T 0ad6442b2de249119ba0ef85dfc06da9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:21.834991 19942 raft_consensus.cc:740] T 0ad6442b2de249119ba0ef85dfc06da9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:21.835144 19532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ce5d15e67644247b95960f444acc621. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:21.836930 19277 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_42018ccf33f9446eaa36e1a3a38dc1ef"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:21.837447 19942 consensus_queue.cc:260] T 0ad6442b2de249119ba0ef85dfc06da9 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:21.838971 19277 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_42018ccf33f9446eaa36e1a3a38dc1ef in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:21.842344 19553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50ea68e3b3c4459b3f5e5b0df724724" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:21.843695 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3cf9676160c4af08f1f62ae952b4c57" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:21.845131 19942 ts_tablet_manager.cc:1434] T 0ad6442b2de249119ba0ef85dfc06da9 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.022s user 0.005s sys 0.000s
I20260925 13:56:21.849359 20441 raft_consensus.cc:493] T 1215d875209a40d38cc527e2795d6c75 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:21.849937 20441 raft_consensus.cc:515] T 1215d875209a40d38cc527e2795d6c75 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:21.852130 20441 leader_election.cc:290] T 1215d875209a40d38cc527e2795d6c75 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
W20260925 13:56:21.852267 19629 leader_election.cc:343] T 28219970809145bc9d4e2a3289a9479a P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:21.855674 19696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215d875209a40d38cc527e2795d6c75" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:21.859407 19942 tablet_bootstrap.cc:492] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
W20260925 13:56:21.859779 19773 leader_election.cc:343] T 1215d875209a40d38cc527e2795d6c75 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:21.861872 19629 leader_election.cc:343] T 28219970809145bc9d4e2a3289a9479a P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:21.874042 19631 consensus_peers.cc:597] T e6e59d6f4f6e4de1be1cf450620cfa59 P 7c1b52895a14482fb54a6758f5054f8f -> Peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Couldn't send request to peer f790493ba7ad403993b678f2df2b55be. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e6e59d6f4f6e4de1be1cf450620cfa59. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:21.881523 20011 ts_tablet_manager.cc:1929] T c2d333f892f84a258f8a148a90ce3b06 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:21.883630 19939 tablet_bootstrap.cc:654] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
W20260925 13:56:21.891197 19773 leader_election.cc:343] T f50ea68e3b3c4459b3f5e5b0df724724 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:21.891583 19773 leader_election.cc:304] T f50ea68e3b3c4459b3f5e5b0df724724 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:21.892089 19279 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_45345226ecf3438eb41e5b811058cfee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:21.892480 20441 raft_consensus.cc:2749] T f50ea68e3b3c4459b3f5e5b0df724724 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:56:21.893453 19773 leader_election.cc:343] T e3cf9676160c4af08f1f62ae952b4c57 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:21.893600 19279 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_45345226ecf3438eb41e5b811058cfee in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:21.893795 19773 leader_election.cc:304] T e3cf9676160c4af08f1f62ae952b4c57 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:21.894599 20441 raft_consensus.cc:2749] T e3cf9676160c4af08f1f62ae952b4c57 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:21.882423 19942 tablet_bootstrap.cc:654] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:21.884940 19695 raft_consensus.cc:3060] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
W20260925 13:56:21.890305 19629 leader_election.cc:343] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:21.899516 19629 leader_election.cc:304] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:21.900211 20407 raft_consensus.cc:2749] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:21.901396 19629 leader_election.cc:304] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:21.901727 20011 log.cc:1199] T c2d333f892f84a258f8a148a90ce3b06 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/c2d333f892f84a258f8a148a90ce3b06
I20260925 13:56:21.903287 20011 ts_tablet_manager.cc:1950] T c2d333f892f84a258f8a148a90ce3b06 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:21.905416 20407 raft_consensus.cc:2749] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:21.908119 19939 tablet_bootstrap.cc:492] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:21.908527 19939 ts_tablet_manager.cc:1403] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.293s user 0.008s sys 0.005s
I20260925 13:56:21.909713 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 145) took 1670 ms. Trace:
I20260925 13:56:21.911520 20011 rpcz_store.cc:276] 0925 13:56:20.239246 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:20.239420 (+ 174us) service_pool.cc:224] Handling call
0925 13:56:21.827712 (+1588292us) ts_tablet_manager.cc:1151] Deleting tablet c2d333f892f84a258f8a148a90ce3b06
0925 13:56:21.827752 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:21.850625 (+ 22873us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:21.881405 (+ 30780us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:21.909662 (+ 28257us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1164,"tablet-delete.queue_time_us":1569503}
I20260925 13:56:21.912101 19955 ts_tablet_manager.cc:1929] T dc14c37aa393407a87af0d0eb8b88d82 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:21.912554 19955 log.cc:1199] T dc14c37aa393407a87af0d0eb8b88d82 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/dc14c37aa393407a87af0d0eb8b88d82
I20260925 13:56:21.913445 19955 ts_tablet_manager.cc:1950] T dc14c37aa393407a87af0d0eb8b88d82 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:21.913769 20455 consensus_queue.cc:1048] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 13:56:21.914302 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 91) took 5272 ms. Trace:
I20260925 13:56:21.914414 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet c2d333f892f84a258f8a148a90ce3b06 (table default.Table_c90b2c8024944acb9ffe76b506fff5a2 [id=d06e00bf26bd4c47a9e27f7eb35835df]) successfully deleted
I20260925 13:56:21.915714 20011 tablet_replica.cc:333] T 1832cff5eb3e4da4b9766ae1591df81a P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:21.916716 19939 raft_consensus.cc:359] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.917430 19939 raft_consensus.cc:385] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:21.917711 19939 raft_consensus.cc:740] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:21.918346 19939 consensus_queue.cc:260] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.918490 20011 raft_consensus.cc:2243] T 1832cff5eb3e4da4b9766ae1591df81a P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Raft consensus shutting down.
W20260925 13:56:21.919135 19280 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a0109e4115174d8eb8174a1a2236150f has no consensus state
I20260925 13:56:21.920923 19939 ts_tablet_manager.cc:1434] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.012s user 0.005s sys 0.001s
I20260925 13:56:21.921749 19939 tablet_bootstrap.cc:492] T 12984db09f34472ba365fdf63a33b720 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:21.922652 20011 pending_rounds.cc:70] T 1832cff5eb3e4da4b9766ae1591df81a P 7c1b52895a14482fb54a6758f5054f8f: Trying to abort 1 pending ops.
I20260925 13:56:21.923910 20011 pending_rounds.cc:77] T 1832cff5eb3e4da4b9766ae1591df81a P 7c1b52895a14482fb54a6758f5054f8f: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251404262776832 op_type: NO_OP noop_request { }
I20260925 13:56:21.924965 20011 raft_consensus.cc:2889] T 1832cff5eb3e4da4b9766ae1591df81a P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:56:21.925902 20011 raft_consensus.cc:2272] T 1832cff5eb3e4da4b9766ae1591df81a P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:21.927549 19280 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_727855fadd224779a3638b48e6b0266f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:21.930265 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215d875209a40d38cc527e2795d6c75" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:21.937286 19938 tablet_bootstrap.cc:654] T ec7fe5c834be4a51b627b8b1c3ebe888 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:21.944476 20011 ts_tablet_manager.cc:1916] T 1832cff5eb3e4da4b9766ae1591df81a P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:21.960098 19277 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_883c781744824c2a9e6e86917bf2143e" } modify_external_catalogs: true
I20260925 13:56:21.960752 19277 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_883c781744824c2a9e6e86917bf2143e" } modify_external_catalogs: true
I20260925 13:56:21.914640 19955 rpcz_store.cc:276] 0925 13:56:16.642234 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:16.646864 (+ 4630us) service_pool.cc:224] Handling call
0925 13:56:21.574096 (+4927232us) ts_tablet_manager.cc:1151] Deleting tablet dc14c37aa393407a87af0d0eb8b88d82
0925 13:56:21.574136 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:21.897952 (+323816us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:21.911898 (+ 13946us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:21.914238 (+ 2340us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4926523}
I20260925 13:56:21.964036 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet dc14c37aa393407a87af0d0eb8b88d82 (table default.Table_3a06919b658e46f893318b4e1ec8b52f [id=e6f2a872536b4e55b071231020903581]) successfully deleted
I20260925 13:56:21.964901 19955 tablet_replica.cc:333] T 47521fb74f9c40ee999a7f704aa551c9 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:21.965835 19955 raft_consensus.cc:2243] T 47521fb74f9c40ee999a7f704aa551c9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:21.966409 19955 raft_consensus.cc:2272] T 47521fb74f9c40ee999a7f704aa551c9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 13:56:21.968751 19280 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_727855fadd224779a3638b48e6b0266f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:21.972860 20441 raft_consensus.cc:493] T eb794cef052b43da831531455e2b5f15 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:21.973306 20441 raft_consensus.cc:515] T eb794cef052b43da831531455e2b5f15 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.975159 19942 tablet_bootstrap.cc:492] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:21.975080 20441 leader_election.cc:290] T eb794cef052b43da831531455e2b5f15 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:21.976182 20407 raft_consensus.cc:493] T 0ad6442b2de249119ba0ef85dfc06da9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:21.983821 20407 raft_consensus.cc:515] T 0ad6442b2de249119ba0ef85dfc06da9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.977315 19696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb794cef052b43da831531455e2b5f15" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:21.985781 20407 leader_election.cc:290] T 0ad6442b2de249119ba0ef85dfc06da9 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:21.986877 19942 ts_tablet_manager.cc:1403] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.128s user 0.012s sys 0.012s
W20260925 13:56:21.987978 19773 leader_election.cc:343] T eb794cef052b43da831531455e2b5f15 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:21.988682 19838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad6442b2de249119ba0ef85dfc06da9" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:21.989212 19838 raft_consensus.cc:2468] T 0ad6442b2de249119ba0ef85dfc06da9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:21.990146 19631 leader_election.cc:304] T 0ad6442b2de249119ba0ef85dfc06da9 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:21.990823 20407 raft_consensus.cc:2804] T 0ad6442b2de249119ba0ef85dfc06da9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:21.991138 20407 raft_consensus.cc:493] T 0ad6442b2de249119ba0ef85dfc06da9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:21.991467 20407 raft_consensus.cc:3060] T 0ad6442b2de249119ba0ef85dfc06da9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:21.992599 19955 ts_tablet_manager.cc:1916] T 47521fb74f9c40ee999a7f704aa551c9 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:21.993939 19942 raft_consensus.cc:359] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.995936 20413 raft_consensus.cc:493] T 5788849b7336496488e5401b4ac91191 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:22.000260 20413 raft_consensus.cc:515] T 5788849b7336496488e5401b4ac91191 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:22.001303 19938 tablet_bootstrap.cc:492] T ec7fe5c834be4a51b627b8b1c3ebe888 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:22.001700 19938 ts_tablet_manager.cc:1403] T ec7fe5c834be4a51b627b8b1c3ebe888 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.194s user 0.009s sys 0.000s
I20260925 13:56:22.002130 20413 leader_election.cc:290] T 5788849b7336496488e5401b4ac91191 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:22.003901 19942 raft_consensus.cc:385] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:22.005528 19942 raft_consensus.cc:740] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:22.007488 19959 ts_tablet_manager.cc:1929] T 12984db09f34472ba365fdf63a33b720 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:22.007834 19959 log.cc:1199] T 12984db09f34472ba365fdf63a33b720 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/12984db09f34472ba365fdf63a33b720
I20260925 13:56:22.007226 19832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5788849b7336496488e5401b4ac91191" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:22.008244 19832 raft_consensus.cc:2468] T 5788849b7336496488e5401b4ac91191 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:22.008445 19959 ts_tablet_manager.cc:1950] T 12984db09f34472ba365fdf63a33b720 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:22.009341 19631 leader_election.cc:304] T 5788849b7336496488e5401b4ac91191 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:22.010118 20413 raft_consensus.cc:2804] T 5788849b7336496488e5401b4ac91191 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:22.010233 19959 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 149) took 1297 ms. Trace:
I20260925 13:56:22.010411 20413 raft_consensus.cc:493] T 5788849b7336496488e5401b4ac91191 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:22.010723 20413 raft_consensus.cc:3060] T 5788849b7336496488e5401b4ac91191 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:22.010514 19959 rpcz_store.cc:276] 0925 13:56:20.712906 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:20.720122 (+ 7216us) service_pool.cc:224] Handling call
0925 13:56:21.747528 (+1027406us) ts_tablet_manager.cc:1151] Deleting tablet 12984db09f34472ba365fdf63a33b720
0925 13:56:21.747569 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:21.931120 (+183551us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:21.982911 (+ 51791us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:22.010184 (+ 27273us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1026870}
I20260925 13:56:22.010926 20407 raft_consensus.cc:515] T 0ad6442b2de249119ba0ef85dfc06da9 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:22.012086 19938 raft_consensus.cc:359] T ec7fe5c834be4a51b627b8b1c3ebe888 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:22.012712 19938 raft_consensus.cc:385] T ec7fe5c834be4a51b627b8b1c3ebe888 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:22.012980 19938 raft_consensus.cc:740] T ec7fe5c834be4a51b627b8b1c3ebe888 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:22.013561 19938 consensus_queue.cc:260] T ec7fe5c834be4a51b627b8b1c3ebe888 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:22.014200 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 12984db09f34472ba365fdf63a33b720 (table default.Table_10868b99c6ec43b2a38a3a4a5bdcf960 [id=46b6ad1a631c4c08a05ef5a1b5f62ab9]) successfully deleted
I20260925 13:56:22.018107 20413 raft_consensus.cc:515] T 5788849b7336496488e5401b4ac91191 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:22.021199 20413 leader_election.cc:290] T 5788849b7336496488e5401b4ac91191 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:22.007975 19942 consensus_queue.cc:260] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:21.987284 19939 tablet_bootstrap.cc:654] T 12984db09f34472ba365fdf63a33b720 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:22.041210 19942 ts_tablet_manager.cc:1434] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.052s user 0.005s sys 0.000s
I20260925 13:56:22.042264 19280 master_service.cc:960] ReplaceTablet: received request to replace tablet 26644768f516412f8c95d68a76bb99c5 from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:22.044644 19938 ts_tablet_manager.cc:1434] T ec7fe5c834be4a51b627b8b1c3ebe888 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.043s user 0.006s sys 0.000s
I20260925 13:56:22.045490 19942 tablet_bootstrap.cc:492] T a26f5a02311c435a8c2434fd030c80a6 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:22.045521 19938 tablet_bootstrap.cc:492] T a26f5a02311c435a8c2434fd030c80a6 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:22.051764 19959 tablet_replica.cc:333] T 137c43fb74ec4cc48d30ed3edac81d98 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
W20260925 13:56:22.052094 19773 leader_election.cc:343] T 1215d875209a40d38cc527e2795d6c75 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:22.052448 19773 leader_election.cc:304] T 1215d875209a40d38cc527e2795d6c75 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:22.053658 20441 raft_consensus.cc:2749] T 1215d875209a40d38cc527e2795d6c75 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:22.057091 19832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5788849b7336496488e5401b4ac91191" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:22.057812 19832 raft_consensus.cc:3060] T 5788849b7336496488e5401b4ac91191 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:22.052552 19959 raft_consensus.cc:2243] T 137c43fb74ec4cc48d30ed3edac81d98 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:22.061343 19959 raft_consensus.cc:2272] T 137c43fb74ec4cc48d30ed3edac81d98 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:22.063063 19959 ts_tablet_manager.cc:1916] T 137c43fb74ec4cc48d30ed3edac81d98 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:22.063872 20407 leader_election.cc:290] T 0ad6442b2de249119ba0ef85dfc06da9 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:22.064715 19942 tablet_bootstrap.cc:654] T a26f5a02311c435a8c2434fd030c80a6 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:22.067075 20011 ts_tablet_manager.cc:1929] T 1832cff5eb3e4da4b9766ae1591df81a P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:22.067440 20011 log.cc:1199] T 1832cff5eb3e4da4b9766ae1591df81a P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/1832cff5eb3e4da4b9766ae1591df81a
I20260925 13:56:22.068354 20011 ts_tablet_manager.cc:1950] T 1832cff5eb3e4da4b9766ae1591df81a P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:22.068780 19939 tablet_bootstrap.cc:492] T 12984db09f34472ba365fdf63a33b720 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:22.069391 19939 ts_tablet_manager.cc:1403] T 12984db09f34472ba365fdf63a33b720 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.148s user 0.007s sys 0.003s
I20260925 13:56:22.069852 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad6442b2de249119ba0ef85dfc06da9" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:22.071364 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5788849b7336496488e5401b4ac91191" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:22.072902 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad6442b2de249119ba0ef85dfc06da9" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:22.073350 19939 raft_consensus.cc:359] T 12984db09f34472ba365fdf63a33b720 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:22.074134 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5788849b7336496488e5401b4ac91191" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:22.082084 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 146) took 1742 ms. Trace:
I20260925 13:56:22.090682 19280 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_cbf20322808c49ef86dff2c576d5654c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:22.092700 19280 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cbf20322808c49ef86dff2c576d5654c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:22.101790 20481 tablet.cc:1751] T 6b52ed579b19479686247d3ecba9489a P 8f578869bc084a5d8a9a69f143954eb3: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 13:56:22.110528 20455 consensus_queue.cc:1048] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 13:56:22.112530 20455 consensus_queue.cc:1048] T 3e6c8b9bbf4549b3a8685f886973ed9c P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 13:56:22.114837 19266 consensus_peers.cc:597] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 -> Peer fc3953a2ba984a658630c423ec9b9728 (127.18.182.252:36181): Couldn't send request to peer fc3953a2ba984a658630c423ec9b9728. Status: Network error: Client connection negotiation failed: client connection to 127.18.182.252:36181: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260925 13:56:22.115504 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb794cef052b43da831531455e2b5f15" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
W20260925 13:56:22.122100 19773 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.193:38269, user_credentials={real_user=slave}} blocked reactor thread for 70052.4us
I20260925 13:56:22.122741 19280 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_256401fd998c430db7f494657ef8e75d" } new_table_name: "default.Table_d65fe24e58a949bca116a094fbb025c5" modify_external_catalogs: true
I20260925 13:56:22.123772 19838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ad6442b2de249119ba0ef85dfc06da9" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:22.124322 19838 raft_consensus.cc:3060] T 0ad6442b2de249119ba0ef85dfc06da9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:22.074390 19939 raft_consensus.cc:385] T 12984db09f34472ba365fdf63a33b720 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:22.127530 19939 raft_consensus.cc:740] T 12984db09f34472ba365fdf63a33b720 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:22.128286 19939 consensus_queue.cc:260] T 12984db09f34472ba365fdf63a33b720 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:22.129951 19939 ts_tablet_manager.cc:1434] T 12984db09f34472ba365fdf63a33b720 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.060s user 0.004s sys 0.001s
I20260925 13:56:22.130762 19939 tablet_bootstrap.cc:492] T 2ef3efe644364e93a089ddeed328389d P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:22.124394 19938 tablet_bootstrap.cc:654] T a26f5a02311c435a8c2434fd030c80a6 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:22.149405 19838 raft_consensus.cc:2468] T 0ad6442b2de249119ba0ef85dfc06da9 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
I20260925 13:56:22.150637 19631 leader_election.cc:304] T 0ad6442b2de249119ba0ef85dfc06da9 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:22.151486 20407 raft_consensus.cc:2804] T 0ad6442b2de249119ba0ef85dfc06da9 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election won for term 1
W20260925 13:56:22.152166 19631 consensus_peers.cc:597] T 12984db09f34472ba365fdf63a33b720 P 7c1b52895a14482fb54a6758f5054f8f -> Peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Couldn't send request to peer f790493ba7ad403993b678f2df2b55be. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 12984db09f34472ba365fdf63a33b720. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:22.082494 20011 rpcz_store.cc:276] 0925 13:56:20.339560 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:20.342674 (+ 3114us) service_pool.cc:224] Handling call
0925 13:56:21.915568 (+1572894us) ts_tablet_manager.cc:1151] Deleting tablet 1832cff5eb3e4da4b9766ae1591df81a
0925 13:56:21.915608 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:22.004261 (+ 88653us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:22.066963 (+ 62702us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:22.081997 (+ 15034us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1572378}
I20260925 13:56:22.151990 20407 raft_consensus.cc:697] T 0ad6442b2de249119ba0ef85dfc06da9 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Becoming Leader. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Running, Role: LEADER
I20260925 13:56:22.154515 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 1832cff5eb3e4da4b9766ae1591df81a (table default.Table_9896a33d94e44d76ada6ec8d00c87ad7 [id=a5549f9e40c644afa1fafcc16d649f68]) successfully deleted
I20260925 13:56:22.155560 20407 consensus_queue.cc:237] T 0ad6442b2de249119ba0ef85dfc06da9 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:22.154491 20011 tablet_replica.cc:333] T e6e59d6f4f6e4de1be1cf450620cfa59 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
W20260925 13:56:22.161262 19629 leader_election.cc:343] T 0ad6442b2de249119ba0ef85dfc06da9 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:22.162384 19629 leader_election.cc:343] T 5788849b7336496488e5401b4ac91191 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:22.162441 20011 raft_consensus.cc:2243] T e6e59d6f4f6e4de1be1cf450620cfa59 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Raft consensus shutting down.
W20260925 13:56:22.163527 19629 leader_election.cc:343] T 0ad6442b2de249119ba0ef85dfc06da9 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:22.165704 19629 leader_election.cc:343] T 5788849b7336496488e5401b4ac91191 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:22.166697 20455 raft_consensus.cc:493] T 12984db09f34472ba365fdf63a33b720 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:22.167136 20455 raft_consensus.cc:515] T 12984db09f34472ba365fdf63a33b720 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:22.169708 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12984db09f34472ba365fdf63a33b720" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:22.165103 19942 tablet_bootstrap.cc:492] T a26f5a02311c435a8c2434fd030c80a6 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:22.171062 19942 ts_tablet_manager.cc:1403] T a26f5a02311c435a8c2434fd030c80a6 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.126s user 0.023s sys 0.050s
I20260925 13:56:22.169868 20455 leader_election.cc:290] T 12984db09f34472ba365fdf63a33b720 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:22.172809 19955 ts_tablet_manager.cc:1929] T 47521fb74f9c40ee999a7f704aa551c9 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:22.173138 19955 log.cc:1199] T 47521fb74f9c40ee999a7f704aa551c9 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/47521fb74f9c40ee999a7f704aa551c9
I20260925 13:56:22.173530 19942 raft_consensus.cc:359] T a26f5a02311c435a8c2434fd030c80a6 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:22.174117 19942 raft_consensus.cc:385] T a26f5a02311c435a8c2434fd030c80a6 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:22.174360 19942 raft_consensus.cc:740] T a26f5a02311c435a8c2434fd030c80a6 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:22.163596 20011 raft_consensus.cc:2272] T e6e59d6f4f6e4de1be1cf450620cfa59 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:22.174932 19955 ts_tablet_manager.cc:1950] T 47521fb74f9c40ee999a7f704aa551c9 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:22.176332 20011 ts_tablet_manager.cc:1916] T e6e59d6f4f6e4de1be1cf450620cfa59 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:22.196707 19942 consensus_queue.cc:260] T a26f5a02311c435a8c2434fd030c80a6 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:22.199419 19939 tablet_bootstrap.cc:654] T 2ef3efe644364e93a089ddeed328389d P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
W20260925 13:56:22.195516 19773 leader_election.cc:343] T eb794cef052b43da831531455e2b5f15 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:22.170799 19832 raft_consensus.cc:2468] T 5788849b7336496488e5401b4ac91191 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
I20260925 13:56:22.201434 19773 leader_election.cc:304] T eb794cef052b43da831531455e2b5f15 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:22.202065 20478 raft_consensus.cc:2749] T eb794cef052b43da831531455e2b5f15 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:22.188108 19938 tablet_bootstrap.cc:492] T a26f5a02311c435a8c2434fd030c80a6 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:22.202626 19938 ts_tablet_manager.cc:1403] T a26f5a02311c435a8c2434fd030c80a6 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.157s user 0.011s sys 0.000s
I20260925 13:56:22.204264 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 92) took 5395 ms. Trace:
I20260925 13:56:22.204628 19955 rpcz_store.cc:276] 0925 13:56:16.809376 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:16.809916 (+ 540us) service_pool.cc:224] Handling call
0925 13:56:21.964743 (+5154827us) ts_tablet_manager.cc:1151] Deleting tablet 47521fb74f9c40ee999a7f704aa551c9
0925 13:56:21.964800 (+ 57us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:22.065218 (+100418us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:22.172692 (+107474us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:22.204209 (+ 31517us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5154291}
I20260925 13:56:22.204958 19938 raft_consensus.cc:359] T a26f5a02311c435a8c2434fd030c80a6 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:22.205586 19938 raft_consensus.cc:385] T a26f5a02311c435a8c2434fd030c80a6 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:22.205838 19938 raft_consensus.cc:740] T a26f5a02311c435a8c2434fd030c80a6 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:22.206521 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 94) took 5217 ms. Trace:
I20260925 13:56:22.206413 19938 consensus_queue.cc:260] T a26f5a02311c435a8c2434fd030c80a6 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:22.206831 19955 rpcz_store.cc:276] 0925 13:56:16.988739 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:16.988878 (+ 139us) service_pool.cc:224] Handling call
0925 13:56:22.206242 (+5217364us) ts_tablet_manager.cc:1151] Deleting tablet 8239d0cd3b0d40bdb721e807c0f403d3
0925 13:56:22.206288 (+ 46us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:22.206473 (+ 185us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5216824}
I20260925 13:56:22.208148 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet 47521fb74f9c40ee999a7f704aa551c9 (table default.Table_3a3e48a0641a43a1a783c6ba6374a708 [id=e64fe44a237f48859a288a87a565ee5e]) successfully deleted
I20260925 13:56:22.208604 19938 ts_tablet_manager.cc:1434] T a26f5a02311c435a8c2434fd030c80a6 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260925 13:56:22.208720 19942 ts_tablet_manager.cc:1434] T a26f5a02311c435a8c2434fd030c80a6 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.037s user 0.006s sys 0.001s
W20260925 13:56:22.208766 19264 catalog_manager.cc:5030] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): delete failed for tablet 8239d0cd3b0d40bdb721e807c0f403d3 because the tablet was not found. No further retry: Not found: Tablet not found: 8239d0cd3b0d40bdb721e807c0f403d3
I20260925 13:56:22.209311 19938 tablet_bootstrap.cc:492] T 2711e7a6973546b1822d93f91095dae2 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:22.212134 19942 tablet_bootstrap.cc:492] T ec7fe5c834be4a51b627b8b1c3ebe888 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
W20260925 13:56:22.216259 19631 consensus_peers.cc:597] T 137c43fb74ec4cc48d30ed3edac81d98 P 7c1b52895a14482fb54a6758f5054f8f -> Peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Couldn't send request to peer f790493ba7ad403993b678f2df2b55be. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:22.216672 19955 tablet_replica.cc:333] T f514fa52e7524cd3ba07f3f65de532d4 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:22.219197 19277 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 4d01a791445149e7bee9493e1eec5c6b
I20260925 13:56:22.224079 19818 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d01a791445149e7bee9493e1eec5c6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:21 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:22.225780 20407 raft_consensus.cc:493] T 2ef3efe644364e93a089ddeed328389d P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:22.226194 20407 raft_consensus.cc:515] T 2ef3efe644364e93a089ddeed328389d P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:22.228086 20407 leader_election.cc:290] T 2ef3efe644364e93a089ddeed328389d P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:22.227969 20455 consensus_queue.cc:1048] T 3e6c8b9bbf4549b3a8685f886973ed9c P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:22.229461 19833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ef3efe644364e93a089ddeed328389d" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:22.230028 19833 raft_consensus.cc:2468] T 2ef3efe644364e93a089ddeed328389d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:22.231055 19631 leader_election.cc:304] T 2ef3efe644364e93a089ddeed328389d P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:22.232024 20435 raft_consensus.cc:2804] T 2ef3efe644364e93a089ddeed328389d P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:22.232334 20435 raft_consensus.cc:493] T 2ef3efe644364e93a089ddeed328389d P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:22.232643 20435 raft_consensus.cc:3060] T 2ef3efe644364e93a089ddeed328389d P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:22.232748 19669 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d01a791445149e7bee9493e1eec5c6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:21 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:22.234133 19277 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_ff75f121abe34bc4a3a7338a2c5bb80c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:22.237275 19277 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ff75f121abe34bc4a3a7338a2c5bb80c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:22.238894 19631 leader_election.cc:304] T 5788849b7336496488e5401b4ac91191 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters: 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:22.239892 20407 raft_consensus.cc:2804] T 5788849b7336496488e5401b4ac91191 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:22.240361 20407 raft_consensus.cc:697] T 5788849b7336496488e5401b4ac91191 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Becoming Leader. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Running, Role: LEADER
I20260925 13:56:22.241153 20407 consensus_queue.cc:237] T 5788849b7336496488e5401b4ac91191 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:22.241895 19488 consensus_peers.cc:597] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3 -> Peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Couldn't send request to peer 7c1b52895a14482fb54a6758f5054f8f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1513c0048bd244028d3dfa8d118bc14e. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:22.243654 19522 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d01a791445149e7bee9493e1eec5c6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:21 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:22.248958 19818 tablet_service.cc:1511] Processing CreateTablet for tablet 505a24ebe7014f80b3c184329cc74a9a (DEFAULT_TABLE table=default.Table_5c6e964736a149d5b977b442a928a1ad [id=2e58540132af4ed298c50ddea3540cdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:22.250427 19818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 505a24ebe7014f80b3c184329cc74a9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:22.251444 20435 raft_consensus.cc:515] T 2ef3efe644364e93a089ddeed328389d P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:22.251739 19669 tablet_service.cc:1511] Processing CreateTablet for tablet 505a24ebe7014f80b3c184329cc74a9a (DEFAULT_TABLE table=default.Table_5c6e964736a149d5b977b442a928a1ad [id=2e58540132af4ed298c50ddea3540cdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:22.252767 19669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 505a24ebe7014f80b3c184329cc74a9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:22.203118 19278 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 8dc5ea164d6749b0926c63d6bda9dcc5 (table default.Table_e0f7b807e8884431898298a1c635c784 [id=e49e015b173c48c1b6ef8112f60b33e4]). Expected version 1 got 0
I20260925 13:56:22.253122 19938 tablet_bootstrap.cc:654] T 2711e7a6973546b1822d93f91095dae2 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:22.253494 19278 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 3e6c8b9bbf4549b3a8685f886973ed9c (table default.Table_d0dadd9b31b143fd8dd5adcb7f752c5f [id=beee0ac537e44859b0f7c76d6808b0e0]). Expected version 1 got 0
I20260925 13:56:22.259763 19522 tablet_service.cc:1511] Processing CreateTablet for tablet 505a24ebe7014f80b3c184329cc74a9a (DEFAULT_TABLE table=default.Table_5c6e964736a149d5b977b442a928a1ad [id=2e58540132af4ed298c50ddea3540cdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:22.265487 19275 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet f50ea68e3b3c4459b3f5e5b0df724724 (table default.Table_6d0ced912e4e4c188553d0839f794603 [id=f1353f53d28748c8a8a4a2c6259581bf]). Expected version 2 got 0
I20260925 13:56:22.266108 19275 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet e3cf9676160c4af08f1f62ae952b4c57 (table default.Table_4d6d747ea4034b3ab48710e10ff4544f [id=d6e6dd0772bf4901b696e11b10e41a48]). Expected version 1 got 0
W20260925 13:56:22.269165 19488 consensus_peers.cc:597] T bbfc4f99a5964e2ba6146e390e8dba44 P 8f578869bc084a5d8a9a69f143954eb3 -> Peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Couldn't send request to peer 7c1b52895a14482fb54a6758f5054f8f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbfc4f99a5964e2ba6146e390e8dba44. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:22.270500 19955 raft_consensus.cc:2243] T f514fa52e7524cd3ba07f3f65de532d4 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:22.271602 19813 tablet_service.cc:1511] Processing CreateTablet for tablet 11077cdb735840b5921e22f588059149 (DEFAULT_TABLE table=default.Table_a2effd0e8c1d4199b1f4192ab3d781e7 [id=c219cb8664c64ce5aba0ed135c171937]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:22.272696 19813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11077cdb735840b5921e22f588059149. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:22.275102 19959 ts_tablet_manager.cc:1929] T 137c43fb74ec4cc48d30ed3edac81d98 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:22.275442 19959 log.cc:1199] T 137c43fb74ec4cc48d30ed3edac81d98 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/137c43fb74ec4cc48d30ed3edac81d98
W20260925 13:56:22.275668 19490 leader_election.cc:343] T 12984db09f34472ba365fdf63a33b720 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 12984db09f34472ba365fdf63a33b720
I20260925 13:56:22.276031 19959 ts_tablet_manager.cc:1950] T 137c43fb74ec4cc48d30ed3edac81d98 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:22.276041 19490 leader_election.cc:304] T 12984db09f34472ba365fdf63a33b720 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3; no voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:22.276639 19279 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet f50ea68e3b3c4459b3f5e5b0df724724 (table default.Table_6d0ced912e4e4c188553d0839f794603 [id=f1353f53d28748c8a8a4a2c6259581bf]). Expected version 2 got 0
I20260925 13:56:22.277148 19279 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet eb794cef052b43da831531455e2b5f15 (table default.Table_c389f408e051410dac7a521879c22729 [id=d2bc135460ab4f2fad4740a281bfc6e2]). Expected version 2 got 0
I20260925 13:56:22.277869 19279 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet e3cf9676160c4af08f1f62ae952b4c57 (table default.Table_4d6d747ea4034b3ab48710e10ff4544f [id=d6e6dd0772bf4901b696e11b10e41a48]). Expected version 1 got 0
I20260925 13:56:22.279276 19273 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 2ef3efe644364e93a089ddeed328389d
I20260925 13:56:22.280978 19273 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:39288 (request call id 207) took 1361 ms. Trace:
I20260925 13:56:22.281527 20011 ts_tablet_manager.cc:1929] T e6e59d6f4f6e4de1be1cf450620cfa59 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:22.283921 20011 log.cc:1199] T e6e59d6f4f6e4de1be1cf450620cfa59 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/e6e59d6f4f6e4de1be1cf450620cfa59
I20260925 13:56:22.284641 19819 tablet_service.cc:1558] Processing DeleteTablet for tablet 2ef3efe644364e93a089ddeed328389d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:20 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:22.285277 20011 ts_tablet_manager.cc:1950] T e6e59d6f4f6e4de1be1cf450620cfa59 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:22.285494 19670 tablet_service.cc:1558] Processing DeleteTablet for tablet 2ef3efe644364e93a089ddeed328389d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:20 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:22.281306 19273 rpcz_store.cc:276] 0925 13:56:20.919477 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:20.919613 (+ 136us) service_pool.cc:224] Handling call
0925 13:56:20.920821 (+ 1208us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 13:56:20.921379 (+ 558us) catalog_manager.cc:2854] Modifying in-memory table state
0925 13:56:20.921492 (+ 113us) catalog_manager.cc:2863] Locking tablets
0925 13:56:21.484785 (+563293us) catalog_manager.cc:2877] Removing table and tablets from system table
0925 13:56:21.776203 (+291418us) catalog_manager.cc:2895] Removing table from by-name map
0925 13:56:21.776431 (+ 228us) catalog_manager.cc:2911] Committing in-memory state
0925 13:56:22.279161 (+502730us) catalog_manager.cc:2915] Aborting table tasks
0925 13:56:22.280927 (+ 1766us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:21.758174 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:21.758548 (+ 374us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:21.758570 (+ 22us) write_op.cc:435] Acquired schema lock
0925 13:56:21.758584 (+ 14us) tablet.cc:662] Decoding operations
0925 13:56:21.758898 (+ 314us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:21.758936 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:21.758959 (+ 23us) tablet.cc:685] Acquiring locks for 2 operations
0925 13:56:21.759200 (+ 241us) tablet.cc:701] Row locks acquired
0925 13:56:21.759215 (+ 15us) write_op.cc:260] PREPARE: finished
0925 13:56:21.759309 (+ 94us) write_op.cc:270] Start()
0925 13:56:21.759424 (+ 115us) write_op.cc:276] Timestamp: P: 1790344581759334 usec, L: 0
0925 13:56:21.759438 (+ 14us) op_driver.cc:348] REPLICATION: starting
0925 13:56:21.759900 (+ 462us) log.cc:844] Serialized 734 byte log entry
0925 13:56:21.774013 (+ 14113us) op_driver.cc:464] REPLICATION: finished
0925 13:56:21.774358 (+ 345us) write_op.cc:301] APPLY: starting
0925 13:56:21.774410 (+ 52us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:21.774513 (+ 103us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:21.774591 (+ 78us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:21.774845 (+ 254us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:21.774984 (+ 139us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 13:56:21.775028 (+ 44us) write_op.cc:312] APPLY: finished
0925 13:56:21.775208 (+ 180us) log.cc:844] Serialized 29 byte log entry
0925 13:56:21.775336 (+ 128us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:21.775422 (+ 86us) write_op.cc:454] Released schema lock
0925 13:56:21.775455 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0925 13:56:21.783668 (+ 8213us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:21.783703 (+ 35us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":221,"apply.run_cpu_time_us":1282,"apply.run_wall_time_us":1281,"num_ops":2,"prepare.queue_time_us":271769,"prepare.run_cpu_time_us":2081,"prepare.run_wall_time_us":25473,"raft.queue_time_us":138,"raft.run_cpu_time_us":656,"raft.run_wall_time_us":656,"replication_time_us":14493}]]}
I20260925 13:56:22.287101 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 147) took 1926 ms. Trace:
I20260925 13:56:22.288173 19273 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_1f476c49446b4488821bbbd7d02b71cd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:22.290992 19273 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1f476c49446b4488821bbbd7d02b71cd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:22.290644 20011 rpcz_store.cc:276] 0925 13:56:20.360598 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:20.361981 (+ 1383us) service_pool.cc:224] Handling call
0925 13:56:22.154351 (+1792370us) ts_tablet_manager.cc:1151] Deleting tablet e6e59d6f4f6e4de1be1cf450620cfa59
0925 13:56:22.154391 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:22.225085 (+ 70694us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:22.281401 (+ 56316us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:22.287047 (+ 5646us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1791849}
I20260925 13:56:22.291883 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet e6e59d6f4f6e4de1be1cf450620cfa59 (table default.Table_546097946d054781b0acae5186c5d43d [id=912de4c02216461f8a69b880207d8652]) successfully deleted
I20260925 13:56:22.293506 20011 tablet_replica.cc:333] T 12984db09f34472ba365fdf63a33b720 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:22.294814 20011 raft_consensus.cc:2243] T 12984db09f34472ba365fdf63a33b720 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:22.295537 20011 pending_rounds.cc:70] T 12984db09f34472ba365fdf63a33b720 P 7c1b52895a14482fb54a6758f5054f8f: Trying to abort 1 pending ops.
I20260925 13:56:22.295749 20011 pending_rounds.cc:77] T 12984db09f34472ba365fdf63a33b720 P 7c1b52895a14482fb54a6758f5054f8f: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251404679790592 op_type: NO_OP noop_request { }
I20260925 13:56:22.296048 20011 raft_consensus.cc:2889] T 12984db09f34472ba365fdf63a33b720 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:56:22.296339 20011 raft_consensus.cc:2272] T 12984db09f34472ba365fdf63a33b720 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:22.295221 19271 catalog_manager.cc:5838] T 3e6c8b9bbf4549b3a8685f886973ed9c P 8f578869bc084a5d8a9a69f143954eb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "8f578869bc084a5d8a9a69f143954eb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:22.272722 19832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ef3efe644364e93a089ddeed328389d" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:22.304863 19832 raft_consensus.cc:3060] T 2ef3efe644364e93a089ddeed328389d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:22.305121 19959 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 150) took 1423 ms. Trace:
I20260925 13:56:22.307713 20478 raft_consensus.cc:493] T a8a679a453e0430b866edc707fe7c234 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:22.307957 19959 rpcz_store.cc:276] 0925 13:56:20.881469 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:20.881592 (+ 123us) service_pool.cc:224] Handling call
0925 13:56:22.051608 (+1170016us) ts_tablet_manager.cc:1151] Deleting tablet 137c43fb74ec4cc48d30ed3edac81d98
0925 13:56:22.051653 (+ 45us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:22.161923 (+110270us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:22.274996 (+113073us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:22.305064 (+ 30068us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1169414}
I20260925 13:56:22.308154 20478 raft_consensus.cc:515] T a8a679a453e0430b866edc707fe7c234 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:22.309092 19959 tablet_replica.cc:333] T 5a9b3dfaa7474611a1824d63eb906b43 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:22.309082 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 137c43fb74ec4cc48d30ed3edac81d98 (table default.Table_5321051e9f6e4cd28b9b77619c908dc9 [id=9dac142084c948f9a50cfc9acf24d461]) successfully deleted
I20260925 13:56:22.303553 19271 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet 3e6c8b9bbf4549b3a8685f886973ed9c (table default.Table_d0dadd9b31b143fd8dd5adcb7f752c5f [id=beee0ac537e44859b0f7c76d6808b0e0]). Expected version 1 got 0
I20260925 13:56:22.322134 19271 catalog_manager.cc:5838] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "8f578869bc084a5d8a9a69f143954eb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:22.324697 19959 raft_consensus.cc:2243] T 5a9b3dfaa7474611a1824d63eb906b43 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:22.325153 19959 raft_consensus.cc:2272] T 5a9b3dfaa7474611a1824d63eb906b43 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:22.327317 20478 leader_election.cc:290] T a8a679a453e0430b866edc707fe7c234 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:22.329154 19959 ts_tablet_manager.cc:1916] T 5a9b3dfaa7474611a1824d63eb906b43 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:22.346330 19271 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet 8dc5ea164d6749b0926c63d6bda9dcc5 (table default.Table_e0f7b807e8884431898298a1c635c784 [id=e49e015b173c48c1b6ef8112f60b33e4]). Expected version 1 got 0
I20260925 13:56:22.354504 19277 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_a51e5b80e56c43cbbd853f7fdc740887"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:22.354919 19532 tablet_service.cc:1558] Processing DeleteTablet for tablet 2ef3efe644364e93a089ddeed328389d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:20 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:22.355665 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ef3efe644364e93a089ddeed328389d" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
W20260925 13:56:22.356074 19277 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a51e5b80e56c43cbbd853f7fdc740887 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:22.358248 19955 raft_consensus.cc:2272] T f514fa52e7524cd3ba07f3f65de532d4 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:22.358788 19553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ef3efe644364e93a089ddeed328389d" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:22.359965 19939 tablet_bootstrap.cc:492] T 2ef3efe644364e93a089ddeed328389d P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:22.350603 20455 raft_consensus.cc:2749] T 12984db09f34472ba365fdf63a33b720 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:22.360387 19939 ts_tablet_manager.cc:1403] T 2ef3efe644364e93a089ddeed328389d P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.230s user 0.012s sys 0.003s
I20260925 13:56:22.364583 19818 tablet_service.cc:1511] Processing CreateTablet for tablet 31b5f47966a74ec3b59c7c20479ba702 (DEFAULT_TABLE table=default.Table_ef5b8cfb027343188a849018298dbfa0 [id=e30a4ca9b0bf4c11a6691a896304c57e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:22.365872 19818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31b5f47966a74ec3b59c7c20479ba702. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:22.366336 20375 raft_consensus.cc:493] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:22.366755 20375 raft_consensus.cc:515] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:22.367823 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8a679a453e0430b866edc707fe7c234" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:22.368577 20375 leader_election.cc:290] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:22.368745 19939 raft_consensus.cc:359] T 2ef3efe644364e93a089ddeed328389d P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:22.369365 19939 raft_consensus.cc:385] T 2ef3efe644364e93a089ddeed328389d P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:22.369627 19939 raft_consensus.cc:740] T 2ef3efe644364e93a089ddeed328389d P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:22.369531 19672 tablet_service.cc:1511] Processing CreateTablet for tablet 31b5f47966a74ec3b59c7c20479ba702 (DEFAULT_TABLE table=default.Table_ef5b8cfb027343188a849018298dbfa0 [id=e30a4ca9b0bf4c11a6691a896304c57e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:22.370492 19672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31b5f47966a74ec3b59c7c20479ba702. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:22.370663 19955 ts_tablet_manager.cc:1916] T f514fa52e7524cd3ba07f3f65de532d4 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:22.370999 19522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 505a24ebe7014f80b3c184329cc74a9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:22.372851 20368 raft_consensus.cc:493] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:22.373252 20368 raft_consensus.cc:515] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:22.375100 20368 leader_election.cc:290] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
W20260925 13:56:22.379331 19773 leader_election.cc:343] T a8a679a453e0430b866edc707fe7c234 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:22.281888 20435 leader_election.cc:290] T 2ef3efe644364e93a089ddeed328389d P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:22.383031 19942 tablet_bootstrap.cc:700] T ec7fe5c834be4a51b627b8b1c3ebe888 P 7c1b52895a14482fb54a6758f5054f8f: Time spent opening tablet: real 0.120s user 0.001s sys 0.000s
I20260925 13:56:22.383072 20435 raft_consensus.cc:493] T a26f5a02311c435a8c2434fd030c80a6 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:22.383714 19942 tablet_bootstrap.cc:654] T ec7fe5c834be4a51b627b8b1c3ebe888 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:22.383850 19276 catalog_manager.cc:6391] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: default.Table_99593e98f3394811b4c91e234185cbef [id=0736e722eb8c43e9b6a151a7c2880fe3] alter complete (version 1)
W20260925 13:56:22.384902 19276 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.194:34125 (request call id 36) took 2018 ms (client timeout 1000 ms). Trace:
I20260925 13:56:22.385167 20011 ts_tablet_manager.cc:1916] T 12984db09f34472ba365fdf63a33b720 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:22.393116 19532 tablet_service.cc:1511] Processing CreateTablet for tablet 11077cdb735840b5921e22f588059149 (DEFAULT_TABLE table=default.Table_a2effd0e8c1d4199b1f4192ab3d781e7 [id=c219cb8664c64ce5aba0ed135c171937]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:22.393565 19819 tablet_service.cc:1558] Processing DeleteTablet for tablet e6e59d6f4f6e4de1be1cf450620cfa59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:20 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:22.394101 19532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11077cdb735840b5921e22f588059149. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:22.391320 19275 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.195:57799 (request call id 29) took 1403 ms (client timeout 1000 ms). Trace:
W20260925 13:56:22.391927 19274 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.193:37907 (request call id 30) took 1993 ms (client timeout 1000 ms). Trace:
W20260925 13:56:22.395804 19275 rpcz_store.cc:269] 0925 13:56:20.987505 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:20.987649 (+ 144us) service_pool.cc:224] Handling call
0925 13:56:22.391243 (+1403594us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":6}
I20260925 13:56:22.395900 19273 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_7786394fcec84a879a5da3eb2377ca74"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:22.395824 19274 rpcz_store.cc:269] 0925 13:56:20.393915 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:20.394027 (+ 112us) service_pool.cc:224] Handling call
0925 13:56:22.387307 (+1993280us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32682,"reported_tablets":15,"spinlock_wait_cycles":56192}
W20260925 13:56:22.397620 19490 leader_election.cc:343] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 7ea4fd6817654bff9c0fe8c6e42ed6b9
W20260925 13:56:22.398073 19273 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7786394fcec84a879a5da3eb2377ca74 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:22.389142 19520 tablet_service.cc:1511] Processing CreateTablet for tablet 31b5f47966a74ec3b59c7c20479ba702 (DEFAULT_TABLE table=default.Table_ef5b8cfb027343188a849018298dbfa0 [id=e30a4ca9b0bf4c11a6691a896304c57e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:22.400003 19520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31b5f47966a74ec3b59c7c20479ba702. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:22.402804 19490 leader_election.cc:343] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 0de8f2d2766c4ff197b552a1b5ed3d6b
I20260925 13:56:22.404197 19670 tablet_service.cc:1558] Processing DeleteTablet for tablet bbfc4f99a5964e2ba6146e390e8dba44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:19 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:22.412520 19819 tablet_service.cc:1511] Processing CreateTablet for tablet 37947005aaf14f27a6e5d1f30c67250c (DEFAULT_TABLE table=default.Table_e7478883ce9046579ee733f571ea95e9 [id=20666a4b905f4fe59476cfe6d86bbd78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:22.413468 19819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37947005aaf14f27a6e5d1f30c67250c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:22.414391 19518 tablet_service.cc:1558] Processing DeleteTablet for tablet dc14c37aa393407a87af0d0eb8b88d82 with delete_type TABLET_DATA_DELETED (replaced by tablet 5f8a96ae6eec4f15a221f9aad537c094) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:22.415326 19518 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d3dbd9796564472b44a31ccaf56d898 with delete_type TABLET_DATA_DELETED (replaced by tablet 0ad6442b2de249119ba0ef85dfc06da9) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:22.418459 19518 tablet_service.cc:1511] Processing CreateTablet for tablet 37947005aaf14f27a6e5d1f30c67250c (DEFAULT_TABLE table=default.Table_e7478883ce9046579ee733f571ea95e9 [id=20666a4b905f4fe59476cfe6d86bbd78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:22.419526 19518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37947005aaf14f27a6e5d1f30c67250c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:22.385200 19276 rpcz_store.cc:269] 0925 13:56:20.366094 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:20.366221 (+ 127us) service_pool.cc:224] Handling call
0925 13:56:20.387892 (+ 21671us) spinlock_profiling.cc:243] Waited 17.5 ms on lock 0x7b5400040454. stack: 00007fdf0a30b323 00007fdf0a30b220 00007fdf09eaf574 0000000000516964 00000000005168f9 00007fdf114095c0 00007fdf114a86fa 00007fdf13219d8c 00007fdf1331801e 00007fdf10ebcfb4 00007fdf10ebcf42 00007fdf10ebce71 00007fdf10ebcdec 00007fdf10ebc0a2 00007fdf0f16e09c 00007fdf0f16d4d6
0925 13:56:22.384838 (+1996946us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:20.925812 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:20.937586 (+ 11774us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:20.937613 (+ 27us) write_op.cc:435] Acquired schema lock
0925 13:56:20.938700 (+ 1087us) tablet.cc:662] Decoding operations
0925 13:56:20.938967 (+ 267us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:20.939011 (+ 44us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:20.939037 (+ 26us) tablet.cc:685] Acquiring locks for 1 operations
0925 13:56:20.939185 (+ 148us) tablet.cc:701] Row locks acquired
0925 13:56:20.939199 (+ 14us) write_op.cc:260] PREPARE: finished
0925 13:56:20.939312 (+ 113us) write_op.cc:270] Start()
0925 13:56:20.939425 (+ 113us) write_op.cc:276] Timestamp: P: 1790344580939391 usec, L: 0
0925 13:56:20.939440 (+ 15us) op_driver.cc:348] REPLICATION: starting
0925 13:56:20.939932 (+ 492us) log.cc:844] Serialized 356 byte log entry
0925 13:56:20.969711 (+ 29779us) op_driver.cc:464] REPLICATION: finished
0925 13:56:20.969985 (+ 274us) write_op.cc:301] APPLY: starting
0925 13:56:20.970042 (+ 57us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:20.970130 (+ 88us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:20.970232 (+ 102us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:20.970378 (+ 146us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:20.970514 (+ 136us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 13:56:20.970553 (+ 39us) write_op.cc:312] APPLY: finished
0925 13:56:20.970724 (+ 171us) log.cc:844] Serialized 23 byte log entry
0925 13:56:20.971288 (+ 564us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:20.971369 (+ 81us) write_op.cc:454] Released schema lock
0925 13:56:20.971400 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0925 13:56:20.992686 (+ 21286us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:20.992714 (+ 28us) write_op.cc:454] Released schema lock
Related trace 'op':
0925 13:56:21.223751 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:21.224120 (+ 369us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:21.224144 (+ 24us) write_op.cc:435] Acquired schema lock
0925 13:56:21.224158 (+ 14us) tablet.cc:662] Decoding operations
0925 13:56:21.224430 (+ 272us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:21.224470 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:21.224492 (+ 22us) tablet.cc:685] Acquiring locks for 1 operations
0925 13:56:21.244998 (+ 20506us) tablet.cc:701] Row locks acquired
0925 13:56:21.245014 (+ 16us) write_op.cc:260] PREPARE: finished
0925 13:56:21.245141 (+ 127us) write_op.cc:270] Start()
0925 13:56:21.245242 (+ 101us) write_op.cc:276] Timestamp: P: 1790344581245160 usec, L: 0
0925 13:56:21.245257 (+ 15us) op_driver.cc:348] REPLICATION: starting
0925 13:56:21.245946 (+ 689us) log.cc:844] Serialized 356 byte log entry
0925 13:56:21.275373 (+ 29427us) op_driver.cc:464] REPLICATION: finished
0925 13:56:21.276059 (+ 686us) write_op.cc:301] APPLY: starting
0925 13:56:21.276113 (+ 54us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:21.276213 (+ 100us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:21.276311 (+ 98us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:21.276465 (+ 154us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:21.276596 (+ 131us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 13:56:21.276655 (+ 59us) write_op.cc:312] APPLY: finished
0925 13:56:21.276846 (+ 191us) log.cc:844] Serialized 23 byte log entry
0925 13:56:21.276994 (+ 148us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:21.277056 (+ 62us) write_op.cc:454] Released schema lock
0925 13:56:21.277093 (+ 37us) write_op.cc:341] FINISH: Updating metrics
0925 13:56:21.277341 (+ 248us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:21.277365 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":4,"spinlock_wait_cycles":40220032,"child_traces":[["op",{"apply.queue_time_us":145,"apply.run_cpu_time_us":1229,"apply.run_wall_time_us":22656,"num_ops":1,"prepare.queue_time_us":405208,"prepare.run_cpu_time_us":2014,"prepare.run_wall_time_us":39172,"raft.queue_time_us":971,"raft.run_cpu_time_us":683,"raft.run_wall_time_us":683,"replication_time_us":30189}],["op",{"apply.queue_time_us":539,"apply.run_cpu_time_us":1236,"apply.run_wall_time_us":1235,"num_ops":1,"prepare.queue_time_us":250014,"prepare.run_cpu_time_us":2241,"prepare.run_wall_time_us":47134,"raft.queue_time_us":93,"raft.run_cpu_time_us":691,"raft.run_wall_time_us":688,"replication_time_us":30004}]]}
I20260925 13:56:22.407810 19274 master_service.cc:960] ReplaceTablet: received request to replace tablet c161ae3edfed4f8488c770ecc6332607 from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:22.438136 19274 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_86e7099f24054da589b5a6f7de9ab131" } modify_external_catalogs: true
I20260925 13:56:22.439029 19274 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_86e7099f24054da589b5a6f7de9ab131" } modify_external_catalogs: true
I20260925 13:56:22.440459 19272 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:39288 (request call id 215) took 1026 ms. Trace:
I20260925 13:56:22.283571 20413 raft_consensus.cc:493] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:22.440747 19272 rpcz_store.cc:276] 0925 13:56:21.413576 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:21.413687 (+ 111us) service_pool.cc:224] Handling call
0925 13:56:21.414454 (+ 767us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 13:56:21.415101 (+ 647us) catalog_manager.cc:3764] Apply alter schema
0925 13:56:21.415601 (+ 500us) catalog_manager.cc:3796] Acquired catalog manager lock
0925 13:56:21.415790 (+ 189us) catalog_manager.cc:3954] Updating metadata on disk
0925 13:56:21.717443 (+301653us) catalog_manager.cc:3985] Committing alterations to in-memory state
0925 13:56:22.440402 (+722959us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:21.464356 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:21.464742 (+ 386us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:21.464766 (+ 24us) write_op.cc:435] Acquired schema lock
0925 13:56:21.464780 (+ 14us) tablet.cc:662] Decoding operations
0925 13:56:21.465012 (+ 232us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:21.465044 (+ 32us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:21.465064 (+ 20us) tablet.cc:685] Acquiring locks for 1 operations
0925 13:56:21.465210 (+ 146us) tablet.cc:701] Row locks acquired
0925 13:56:21.465222 (+ 12us) write_op.cc:260] PREPARE: finished
0925 13:56:21.465319 (+ 97us) write_op.cc:270] Start()
0925 13:56:21.465391 (+ 72us) write_op.cc:276] Timestamp: P: 1790344581465397 usec, L: 0
0925 13:56:21.465405 (+ 14us) op_driver.cc:348] REPLICATION: starting
0925 13:56:21.625402 (+159997us) log.cc:844] Serialized 405 byte log entry
0925 13:56:21.713021 (+ 87619us) op_driver.cc:464] REPLICATION: finished
0925 13:56:21.714341 (+ 1320us) write_op.cc:301] APPLY: starting
0925 13:56:21.714411 (+ 70us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:21.714532 (+ 121us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:21.714682 (+ 150us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:21.714848 (+ 166us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:21.714967 (+ 119us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 13:56:21.715007 (+ 40us) write_op.cc:312] APPLY: finished
0925 13:56:21.715160 (+ 153us) log.cc:844] Serialized 23 byte log entry
0925 13:56:21.715304 (+ 144us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:21.715384 (+ 80us) write_op.cc:454] Released schema lock
0925 13:56:21.715426 (+ 42us) write_op.cc:341] FINISH: Updating metrics
0925 13:56:21.715706 (+ 280us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:21.715737 (+ 31us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1159,"apply.run_cpu_time_us":1324,"apply.run_wall_time_us":1323,"num_ops":1,"prepare.queue_time_us":47600,"prepare.run_cpu_time_us":1898,"prepare.run_wall_time_us":161693,"raft.queue_time_us":151,"raft.run_cpu_time_us":22,"raft.run_wall_time_us":25,"replication_time_us":247514}]]}
I20260925 13:56:22.453488 19669 tablet_service.cc:1558] Processing DeleteTablet for tablet c2d333f892f84a258f8a148a90ce3b06 with delete_type TABLET_DATA_DELETED (replaced by tablet 908df914cfec490cb7058dbe7b501c97) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:22.454236 20413 raft_consensus.cc:515] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:22.454874 19694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0de8f2d2766c4ff197b552a1b5ed3d6b" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:22.455495 19694 raft_consensus.cc:2468] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:22.456339 19668 tablet_service.cc:1511] Processing CreateTablet for tablet 11077cdb735840b5921e22f588059149 (DEFAULT_TABLE table=default.Table_a2effd0e8c1d4199b1f4192ab3d781e7 [id=c219cb8664c64ce5aba0ed135c171937]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:22.456470 19696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8a679a453e0430b866edc707fe7c234" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:22.457368 19668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11077cdb735840b5921e22f588059149. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:22.457329 19669 tablet_service.cc:1558] Processing DeleteTablet for tablet 0de8f2d2766c4ff197b552a1b5ed3d6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:18 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:22.457909 19693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea4fd6817654bff9c0fe8c6e42ed6b9" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:22.458261 19773 leader_election.cc:343] T a8a679a453e0430b866edc707fe7c234 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:22.458431 19693 raft_consensus.cc:2468] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:22.458649 19773 leader_election.cc:304] T a8a679a453e0430b866edc707fe7c234 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:22.459806 19670 tablet_service.cc:1558] Processing DeleteTablet for tablet 2a1f34dba13143b097a38bab3782c404 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:14 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:22.460062 20478 raft_consensus.cc:2749] T a8a679a453e0430b866edc707fe7c234 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:56:22.451002 19629 leader_election.cc:343] T 2ef3efe644364e93a089ddeed328389d P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:22.460975 19629 leader_election.cc:343] T 2ef3efe644364e93a089ddeed328389d P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:22.462234 19670 tablet_service.cc:1558] Processing DeleteTablet for tablet 7ea4fd6817654bff9c0fe8c6e42ed6b9 with delete_type TABLET_DATA_DELETED (replaced by tablet 321c740ba8eb4cbb8974305c5f8702af) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:22.463639 19670 tablet_service.cc:1558] Processing DeleteTablet for tablet e6e59d6f4f6e4de1be1cf450620cfa59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:20 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:22.464741 19488 leader_election.cc:304] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3; no voters: f790493ba7ad403993b678f2df2b55be
I20260925 13:56:22.465549 19670 tablet_service.cc:1558] Processing DeleteTablet for tablet f514fa52e7524cd3ba07f3f65de532d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:16 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:22.465758 19488 leader_election.cc:304] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3; no voters: f790493ba7ad403993b678f2df2b55be
W20260925 13:56:22.466964 19488 consensus_peers.cc:597] T c2d333f892f84a258f8a148a90ce3b06 P 8f578869bc084a5d8a9a69f143954eb3 -> Peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Couldn't send request to peer 7c1b52895a14482fb54a6758f5054f8f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c2d333f892f84a258f8a148a90ce3b06. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:22.467399 20338 raft_consensus.cc:2804] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:22.467516 20455 raft_consensus.cc:2804] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:22.467809 20338 raft_consensus.cc:493] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:22.467886 20455 raft_consensus.cc:493] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:22.468122 20338 raft_consensus.cc:3060] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:22.468216 20455 raft_consensus.cc:3060] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:22.469080 19670 tablet_service.cc:1558] Processing DeleteTablet for tablet 1832cff5eb3e4da4b9766ae1591df81a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:16 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:22.469164 19279 catalog_manager.cc:6391] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: default.Table_441a94250b5e4930a0b266c22acbda8a [id=d804d75fe5004ea4a1dc3245acb39ae7] alter complete (version 2)
W20260925 13:56:22.470697 19278 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.194:34125 (request call id 37) took 1093 ms (client timeout 1000 ms). Trace:
W20260925 13:56:22.471030 19278 rpcz_store.cc:269] 0925 13:56:21.376900 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:21.377504 (+ 604us) service_pool.cc:224] Handling call
0925 13:56:21.397693 (+ 20189us) spinlock_profiling.cc:243] Waited 17 ms on lock 0x7b5400040454. stack: 00007fdf0a30b323 00007fdf0a30b220 00007fdf09eaf574 0000000000516964 00000000005168f9 00007fdf114095c0 00007fdf114a86fa 00007fdf13219d8c 00007fdf1331801e 00007fdf10ebcfb4 00007fdf10ebcf42 00007fdf10ebce71 00007fdf10ebcdec 00007fdf10ebc0a2 00007fdf0f16e09c 00007fdf0f16d4d6
0925 13:56:22.470629 (+1072936us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":17,"spinlock_wait_cycles":39137024}
I20260925 13:56:22.475803 20455 raft_consensus.cc:515] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:22.383551 20435 raft_consensus.cc:515] T a26f5a02311c435a8c2434fd030c80a6 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:22.481388 20455 leader_election.cc:290] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:22.483529 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea4fd6817654bff9c0fe8c6e42ed6b9" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:22.484643 19690 raft_consensus.cc:3060] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:22.487095 20435 leader_election.cc:290] T a26f5a02311c435a8c2434fd030c80a6 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:22.487577 20435 consensus_queue.cc:1048] T 137c43fb74ec4cc48d30ed3edac81d98 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:22.488973 20413 leader_election.cc:290] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:22.491174 19278 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_c90b2c8024944acb9ffe76b506fff5a2" } modify_external_catalogs: true
I20260925 13:56:22.491725 19278 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_c90b2c8024944acb9ffe76b506fff5a2" } modify_external_catalogs: true
I20260925 13:56:22.493005 19690 raft_consensus.cc:2468] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:22.370273 19939 consensus_queue.cc:260] T 2ef3efe644364e93a089ddeed328389d P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:22.500661 20338 raft_consensus.cc:515] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:22.501379 19939 ts_tablet_manager.cc:1434] T 2ef3efe644364e93a089ddeed328389d P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.140s user 0.006s sys 0.000s
I20260925 13:56:22.502272 19939 tablet_bootstrap.cc:492] T 28219970809145bc9d4e2a3289a9479a P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:22.505308 19274 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet ec7fe5c834be4a51b627b8b1c3ebe888
W20260925 13:56:22.507171 19265 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet a0109e4115174d8eb8174a1a2236150f (table default.Table_92f5e6ebf342495b8333c8da779d6998 [id=af68d1e4a6874f999362917b2feb6efe], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet a0109e4115174d8eb8174a1a2236150f consensus state has no leader
I20260925 13:56:22.508658 19812 tablet_service.cc:1558] Processing DeleteTablet for tablet ec7fe5c834be4a51b627b8b1c3ebe888 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:22 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:22.509426 19533 tablet_service.cc:1558] Processing DeleteTablet for tablet ec7fe5c834be4a51b627b8b1c3ebe888 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:22 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:22.513954 19812 tablet_service.cc:1511] Processing CreateTablet for tablet 72183473a5c94629a58c2cd8e1c56f32 (DEFAULT_TABLE table=default.Table_0b05542218754c9c8bbf04c76ba736b5 [id=5965407f056548edadae1a81a5c56103]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:22.514609 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a86ea84d1d49d1b0950b4ce3a3d770" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:22.514994 19812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72183473a5c94629a58c2cd8e1c56f32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:22.515935 19938 tablet_bootstrap.cc:492] T 2711e7a6973546b1822d93f91095dae2 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:22.516279 19938 ts_tablet_manager.cc:1403] T 2711e7a6973546b1822d93f91095dae2 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.307s user 0.008s sys 0.001s
I20260925 13:56:22.517897 19533 tablet_service.cc:1511] Processing CreateTablet for tablet 72183473a5c94629a58c2cd8e1c56f32 (DEFAULT_TABLE table=default.Table_0b05542218754c9c8bbf04c76ba736b5 [id=5965407f056548edadae1a81a5c56103]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:22.518855 19533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72183473a5c94629a58c2cd8e1c56f32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:22.520948 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a26f5a02311c435a8c2434fd030c80a6" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:22.527763 19939 tablet_bootstrap.cc:654] T 28219970809145bc9d4e2a3289a9479a P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:22.529424 19938 raft_consensus.cc:359] T 2711e7a6973546b1822d93f91095dae2 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:22.530063 19938 raft_consensus.cc:385] T 2711e7a6973546b1822d93f91095dae2 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:22.530383 19938 raft_consensus.cc:740] T 2711e7a6973546b1822d93f91095dae2 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:22.533152 19278 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 908df914cfec490cb7058dbe7b501c97
I20260925 13:56:22.540637 19801 tablet_service.cc:1558] Processing DeleteTablet for tablet 908df914cfec490cb7058dbe7b501c97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:22 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:22.507440 19670 tablet_service.cc:1558] Processing DeleteTablet for tablet ec7fe5c834be4a51b627b8b1c3ebe888 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:22 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:22.512559 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0de8f2d2766c4ff197b552a1b5ed3d6b" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:22.544066 19532 tablet_service.cc:1558] Processing DeleteTablet for tablet 908df914cfec490cb7058dbe7b501c97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:22 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:22.544404 19692 raft_consensus.cc:3060] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:22.545131 19801 tablet_service.cc:1558] Processing DeleteTablet for tablet c161ae3edfed4f8488c770ecc6332607 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:17 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:22.531397 20413 raft_consensus.cc:493] T e3cf9676160c4af08f1f62ae952b4c57 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:22.547539 20413 raft_consensus.cc:515] T e3cf9676160c4af08f1f62ae952b4c57 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:22.547863 20338 leader_election.cc:290] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:22.549489 20413 leader_election.cc:290] T e3cf9676160c4af08f1f62ae952b4c57 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:22.553613 19692 raft_consensus.cc:2468] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:22.556630 19278 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_231f1e829ede440cae21c914d61e62d9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:22.558185 19278 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_231f1e829ede440cae21c914d61e62d9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:22.557896 19670 tablet_service.cc:1511] Processing CreateTablet for tablet 37947005aaf14f27a6e5d1f30c67250c (DEFAULT_TABLE table=default.Table_e7478883ce9046579ee733f571ea95e9 [id=20666a4b905f4fe59476cfe6d86bbd78]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 13:56:22.560662 19490 leader_election.cc:343] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 7ea4fd6817654bff9c0fe8c6e42ed6b9
W20260925 13:56:22.562803 19629 leader_election.cc:343] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:22.563455 19629 leader_election.cc:343] T a26f5a02311c435a8c2434fd030c80a6 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:22.564781 19670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37947005aaf14f27a6e5d1f30c67250c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:22.567158 19518 tablet_service.cc:1511] Processing CreateTablet for tablet a218f033a132482a82bd1f60126fa0f9 (DEFAULT_TABLE table=default.Table_67306a96449b4a7bbb4b96498a396cea [id=bcf9d7c5e4f44267b9204bc3c2d90584]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 13:56:22.568838 19271 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.193:37907 (request call id 31) took 837 ms (client timeout 1000 ms). Trace:
W20260925 13:56:22.569689 19271 rpcz_store.cc:269] 0925 13:56:21.731742 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:21.751514 (+ 19772us) service_pool.cc:224] Handling call
0925 13:56:22.568769 (+817255us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:22.476231 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:22.476607 (+ 376us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:22.476629 (+ 22us) write_op.cc:435] Acquired schema lock
0925 13:56:22.476641 (+ 12us) tablet.cc:662] Decoding operations
0925 13:56:22.476914 (+ 273us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:22.476946 (+ 32us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:22.476978 (+ 32us) tablet.cc:685] Acquiring locks for 2 operations
0925 13:56:22.477364 (+ 386us) tablet.cc:701] Row locks acquired
0925 13:56:22.477378 (+ 14us) write_op.cc:260] PREPARE: finished
0925 13:56:22.477473 (+ 95us) write_op.cc:270] Start()
0925 13:56:22.477544 (+ 71us) write_op.cc:276] Timestamp: P: 1790344582477540 usec, L: 0
0925 13:56:22.477561 (+ 17us) op_driver.cc:348] REPLICATION: starting
0925 13:56:22.477976 (+ 415us) log.cc:844] Serialized 919 byte log entry
0925 13:56:22.492664 (+ 14688us) op_driver.cc:464] REPLICATION: finished
0925 13:56:22.495288 (+ 2624us) write_op.cc:301] APPLY: starting
0925 13:56:22.495339 (+ 51us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:22.495488 (+ 149us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:22.495618 (+ 130us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:22.495843 (+ 225us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:22.495980 (+ 137us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 13:56:22.496021 (+ 41us) write_op.cc:312] APPLY: finished
0925 13:56:22.496201 (+ 180us) log.cc:844] Serialized 29 byte log entry
0925 13:56:22.496332 (+ 131us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:22.496401 (+ 69us) write_op.cc:454] Released schema lock
0925 13:56:22.496434 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0925 13:56:22.496651 (+ 217us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:22.496675 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":21,"spinlock_wait_cycles":1401728,"child_traces":[["op",{"apply.queue_time_us":2485,"apply.run_cpu_time_us":1335,"apply.run_wall_time_us":1334,"num_ops":2,"prepare.queue_time_us":580,"prepare.run_cpu_time_us":1809,"prepare.run_wall_time_us":3404,"replication_time_us":15024}]]}
I20260925 13:56:22.578280 19801 tablet_service.cc:1511] Processing CreateTablet for tablet a218f033a132482a82bd1f60126fa0f9 (DEFAULT_TABLE table=default.Table_67306a96449b4a7bbb4b96498a396cea [id=bcf9d7c5e4f44267b9204bc3c2d90584]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:22.578745 19832 raft_consensus.cc:2468] T 2ef3efe644364e93a089ddeed328389d P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
I20260925 13:56:22.579403 19801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a218f033a132482a82bd1f60126fa0f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:22.579766 19669 tablet_service.cc:1558] Processing DeleteTablet for tablet 908df914cfec490cb7058dbe7b501c97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:22 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:22.581655 19669 tablet_service.cc:1511] Processing CreateTablet for tablet 72183473a5c94629a58c2cd8e1c56f32 (DEFAULT_TABLE table=default.Table_0b05542218754c9c8bbf04c76ba736b5 [id=5965407f056548edadae1a81a5c56103]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:22.582805 19669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72183473a5c94629a58c2cd8e1c56f32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:22.598973 19488 leader_election.cc:304] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3; no voters:
I20260925 13:56:22.599231 19673 tablet_service.cc:1511] Processing CreateTablet for tablet a218f033a132482a82bd1f60126fa0f9 (DEFAULT_TABLE table=default.Table_67306a96449b4a7bbb4b96498a396cea [id=bcf9d7c5e4f44267b9204bc3c2d90584]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:22.600314 19673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a218f033a132482a82bd1f60126fa0f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:22.601713 20338 raft_consensus.cc:2804] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:22.602308 20338 raft_consensus.cc:697] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Running, Role: LEADER
I20260925 13:56:22.603140 20338 consensus_queue.cc:237] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:22.606727 19518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a218f033a132482a82bd1f60126fa0f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:22.606833 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3cf9676160c4af08f1f62ae952b4c57" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
W20260925 13:56:22.610569 19490 leader_election.cc:343] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 0de8f2d2766c4ff197b552a1b5ed3d6b
W20260925 13:56:22.612548 19629 leader_election.cc:343] T e3cf9676160c4af08f1f62ae952b4c57 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:22.613809 19813 tablet_service.cc:1511] Processing CreateTablet for tablet f146a9aee3054d7491e74101df0e28e1 (DEFAULT_TABLE table=default.Table_f00423c3addb4d2a87e8d419f72a686c [id=88175c51667a4a42b556e232630db485]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:22.615131 19813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f146a9aee3054d7491e74101df0e28e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:22.617560 19524 tablet_service.cc:1511] Processing CreateTablet for tablet f146a9aee3054d7491e74101df0e28e1 (DEFAULT_TABLE table=default.Table_f00423c3addb4d2a87e8d419f72a686c [id=88175c51667a4a42b556e232630db485]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:22.618626 19524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f146a9aee3054d7491e74101df0e28e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:22.625365 19804 tablet_service.cc:1511] Processing CreateTablet for tablet 45ab2817d7e84185b7ec2bfddcf4997f (DEFAULT_TABLE table=default.Table_ab754389c42247ccbed64c8f7c539d90 [id=9cb2dc12a91e4de087414b492cd0dcfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:22.626434 19804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45ab2817d7e84185b7ec2bfddcf4997f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:22.628917 19488 leader_election.cc:304] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3; no voters: f790493ba7ad403993b678f2df2b55be
I20260925 13:56:22.531193 19938 consensus_queue.cc:260] T 2711e7a6973546b1822d93f91095dae2 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:22.635447 19667 tablet_service.cc:1511] Processing CreateTablet for tablet 45ab2817d7e84185b7ec2bfddcf4997f (DEFAULT_TABLE table=default.Table_ab754389c42247ccbed64c8f7c539d90 [id=9cb2dc12a91e4de087414b492cd0dcfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:22.636459 19667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45ab2817d7e84185b7ec2bfddcf4997f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:22.637823 19672 tablet_service.cc:1511] Processing CreateTablet for tablet f146a9aee3054d7491e74101df0e28e1 (DEFAULT_TABLE table=default.Table_f00423c3addb4d2a87e8d419f72a686c [id=88175c51667a4a42b556e232630db485]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:22.639109 19672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f146a9aee3054d7491e74101df0e28e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:22.641525 20478 raft_consensus.cc:493] T ec7fe5c834be4a51b627b8b1c3ebe888 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:22.641924 20478 raft_consensus.cc:515] T ec7fe5c834be4a51b627b8b1c3ebe888 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:22.648491 19532 tablet_service.cc:1511] Processing CreateTablet for tablet 45ab2817d7e84185b7ec2bfddcf4997f (DEFAULT_TABLE table=default.Table_ab754389c42247ccbed64c8f7c539d90 [id=9cb2dc12a91e4de087414b492cd0dcfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:22.649293 20369 raft_consensus.cc:2804] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:22.649514 19532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45ab2817d7e84185b7ec2bfddcf4997f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:22.649732 20369 raft_consensus.cc:697] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Running, Role: LEADER
I20260925 13:56:22.650480 20369 consensus_queue.cc:237] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:22.652549 20441 raft_consensus.cc:493] T 3aaf29a2310a41d0971eded7ef51fdc9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:22.652590 19529 tablet_service.cc:1511] Processing CreateTablet for tablet e4b29a3d0edc481cba6c61cfb78f8f40 (DEFAULT_TABLE table=default.Table_240f5ce2f31541eda914442e2cab887a [id=45fc848c759b4365a5c69016b6aa3309]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:22.652976 20441 raft_consensus.cc:515] T 3aaf29a2310a41d0971eded7ef51fdc9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:22.653592 19529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4b29a3d0edc481cba6c61cfb78f8f40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:22.655121 19519 tablet_service.cc:1511] Processing CreateTablet for tablet 540132db164c403da72b9377078c4bf0 (DEFAULT_TABLE table=default.Table_5321051e9f6e4cd28b9b77619c908dc9 [id=9dac142084c948f9a50cfc9acf24d461]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:22.655915 19523 tablet_service.cc:1511] Processing CreateTablet for tablet de2b3e07bd634bc08368989a9062ba08 (DEFAULT_TABLE table=default.Table_dffaefeede134bde97c6bcdff8bfc010 [id=be2ddd2647094a8788d52ef46500426f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:22.656239 19519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 540132db164c403da72b9377078c4bf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:22.656934 19523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de2b3e07bd634bc08368989a9062ba08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:22.672080 20441 leader_election.cc:290] T 3aaf29a2310a41d0971eded7ef51fdc9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:22.676545 19959 ts_tablet_manager.cc:1929] T 5a9b3dfaa7474611a1824d63eb906b43 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:22.676882 19959 log.cc:1199] T 5a9b3dfaa7474611a1824d63eb906b43 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/5a9b3dfaa7474611a1824d63eb906b43
I20260925 13:56:22.677469 19959 ts_tablet_manager.cc:1950] T 5a9b3dfaa7474611a1824d63eb906b43 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:22.678843 20478 leader_election.cc:290] T ec7fe5c834be4a51b627b8b1c3ebe888 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:22.679971 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaf29a2310a41d0971eded7ef51fdc9" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:22.701224 19942 tablet_bootstrap.cc:492] T ec7fe5c834be4a51b627b8b1c3ebe888 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:22.701669 19942 ts_tablet_manager.cc:1403] T ec7fe5c834be4a51b627b8b1c3ebe888 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.490s user 0.008s sys 0.011s
I20260925 13:56:22.703859 19938 ts_tablet_manager.cc:1434] T 2711e7a6973546b1822d93f91095dae2 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.187s user 0.007s sys 0.000s
I20260925 13:56:22.704913 19938 tablet_bootstrap.cc:492] T 9059655f2c3d4e86b3b389b5b9a68891 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:22.574842 19274 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 8dc5ea164d6749b0926c63d6bda9dcc5 (table default.Table_e0f7b807e8884431898298a1c635c784 [id=e49e015b173c48c1b6ef8112f60b33e4]). Expected version 1 got 0
I20260925 13:56:22.724309 19274 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 3e6c8b9bbf4549b3a8685f886973ed9c (table default.Table_d0dadd9b31b143fd8dd5adcb7f752c5f [id=beee0ac537e44859b0f7c76d6808b0e0]). Expected version 1 got 0
I20260925 13:56:22.597918 19273 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_86568b68e691429192052565e933af79"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:22.651064 19278 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_ef5b8cfb027343188a849018298dbfa0" } new_table_name: "default.Table_bd9c5f2059394324857aa7fef30188f7" modify_external_catalogs: true
I20260925 13:56:22.671114 19279 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_750d58e5a30a4b43ba8d84fb7821273e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:22.730901 19279 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_750d58e5a30a4b43ba8d84fb7821273e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:22.676155 19280 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_a2effd0e8c1d4199b1f4192ab3d781e7" } new_table_name: "default.Table_e4f4f8c05f904aa485bd1b4c5a96d9c8" modify_external_catalogs: true
I20260925 13:56:22.711809 19277 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_231f1e829ede440cae21c914d61e62d9" } modify_external_catalogs: true
I20260925 13:56:22.736115 19277 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_231f1e829ede440cae21c914d61e62d9" } modify_external_catalogs: true
I20260925 13:56:22.739985 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7fe5c834be4a51b627b8b1c3ebe888" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
W20260925 13:56:22.741941 19273 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_86568b68e691429192052565e933af79 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:22.750844 19274 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_9cdc814c021f49dea82875834691ba9e" } modify_external_catalogs: true
I20260925 13:56:22.758447 19274 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_9cdc814c021f49dea82875834691ba9e" } modify_external_catalogs: true
I20260925 13:56:22.774438 19525 tablet_service.cc:1511] Processing CreateTablet for tablet a0109e4115174d8eb8174a1a2236150f (DEFAULT_TABLE table=default.Table_92f5e6ebf342495b8333c8da779d6998 [id=af68d1e4a6874f999362917b2feb6efe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:22.775689 19525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0109e4115174d8eb8174a1a2236150f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:22.781769 19271 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_5f090a5fffe7463c9dcac5a5023da7d1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:22.783874 19271 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5f090a5fffe7463c9dcac5a5023da7d1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 13:56:22.789011 19773 leader_election.cc:343] T 3aaf29a2310a41d0971eded7ef51fdc9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:22.789619 19773 leader_election.cc:343] T ec7fe5c834be4a51b627b8b1c3ebe888 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:22.789243 19818 tablet_service.cc:1511] Processing CreateTablet for tablet e4b29a3d0edc481cba6c61cfb78f8f40 (DEFAULT_TABLE table=default.Table_240f5ce2f31541eda914442e2cab887a [id=45fc848c759b4365a5c69016b6aa3309]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:22.790987 19818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4b29a3d0edc481cba6c61cfb78f8f40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:22.793607 19819 tablet_service.cc:1511] Processing CreateTablet for tablet 540132db164c403da72b9377078c4bf0 (DEFAULT_TABLE table=default.Table_5321051e9f6e4cd28b9b77619c908dc9 [id=9dac142084c948f9a50cfc9acf24d461]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:22.793879 19817 tablet_service.cc:1511] Processing CreateTablet for tablet de2b3e07bd634bc08368989a9062ba08 (DEFAULT_TABLE table=default.Table_dffaefeede134bde97c6bcdff8bfc010 [id=be2ddd2647094a8788d52ef46500426f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:22.796077 19805 tablet_service.cc:1511] Processing CreateTablet for tablet a0109e4115174d8eb8174a1a2236150f (DEFAULT_TABLE table=default.Table_92f5e6ebf342495b8333c8da779d6998 [id=af68d1e4a6874f999362917b2feb6efe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:22.797224 19803 tablet_service.cc:1558] Processing DeleteTablet for tablet a26f5a02311c435a8c2434fd030c80a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:19 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:22.804790 19959 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 151) took 1782 ms. Trace:
I20260925 13:56:22.805179 19959 rpcz_store.cc:276] 0925 13:56:21.022268 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:21.022376 (+ 108us) service_pool.cc:224] Handling call
0925 13:56:22.308948 (+1286572us) ts_tablet_manager.cc:1151] Deleting tablet 5a9b3dfaa7474611a1824d63eb906b43
0925 13:56:22.308989 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:22.348852 (+ 39863us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:22.676437 (+327585us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:22.804707 (+128270us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1286027}
I20260925 13:56:22.806671 19959 tablet_replica.cc:333] T 5788849b7336496488e5401b4ac91191 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:22.807387 19959 raft_consensus.cc:2243] T 5788849b7336496488e5401b4ac91191 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:22.807802 19959 raft_consensus.cc:2272] T 5788849b7336496488e5401b4ac91191 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:22.807756 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 5a9b3dfaa7474611a1824d63eb906b43 (table default.Table_67306a96449b4a7bbb4b96498a396cea [id=bcf9d7c5e4f44267b9204bc3c2d90584]) successfully deleted
I20260925 13:56:22.807653 19272 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_2b6499b3f48241f58bae5603d9f439c8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:22.809075 19272 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2b6499b3f48241f58bae5603d9f439c8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 13:56:22.823173 19278 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 31b5f47966a74ec3b59c7c20479ba702 consensus state has no leader
I20260925 13:56:22.841161 19938 tablet_bootstrap.cc:654] T 9059655f2c3d4e86b3b389b5b9a68891 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:22.843500 19959 ts_tablet_manager.cc:1916] T 5788849b7336496488e5401b4ac91191 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 13:56:22.848587 19280 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 11077cdb735840b5921e22f588059149 consensus state has no leader
I20260925 13:56:22.853664 19817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de2b3e07bd634bc08368989a9062ba08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:22.858126 19805 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0109e4115174d8eb8174a1a2236150f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:22.859727 19819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 540132db164c403da72b9377078c4bf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:22.865412 19959 ts_tablet_manager.cc:1929] T 5788849b7336496488e5401b4ac91191 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:22.865813 19959 log.cc:1199] T 5788849b7336496488e5401b4ac91191 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/5788849b7336496488e5401b4ac91191
I20260925 13:56:22.866465 19959 ts_tablet_manager.cc:1950] T 5788849b7336496488e5401b4ac91191 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:22.868269 19959 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 152) took 1804 ms. Trace:
I20260925 13:56:22.868559 19959 rpcz_store.cc:276] 0925 13:56:21.063670 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:21.063772 (+ 102us) service_pool.cc:224] Handling call
0925 13:56:22.806526 (+1742754us) ts_tablet_manager.cc:1151] Deleting tablet 5788849b7336496488e5401b4ac91191
0925 13:56:22.806568 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:22.858466 (+ 51898us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:22.865281 (+ 6815us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:22.868211 (+ 2930us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1742262}
I20260925 13:56:22.869570 19959 tablet_replica.cc:333] T c1f15b9de572479aae6c2e644da3392a P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:22.870472 19959 raft_consensus.cc:2243] T c1f15b9de572479aae6c2e644da3392a P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:22.870991 19959 raft_consensus.cc:2272] T c1f15b9de572479aae6c2e644da3392a P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:22.872323 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 5788849b7336496488e5401b4ac91191 (table default.Table_16c91b8d55be449899b5e3a1884d81b4 [id=72cd050d1582435e8a7e8c101df3e117]) successfully deleted
I20260925 13:56:22.873147 19959 ts_tablet_manager.cc:1916] T c1f15b9de572479aae6c2e644da3392a P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:22.889581 19959 ts_tablet_manager.cc:1929] T c1f15b9de572479aae6c2e644da3392a P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:22.890168 19959 log.cc:1199] T c1f15b9de572479aae6c2e644da3392a P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/c1f15b9de572479aae6c2e644da3392a
I20260925 13:56:22.891324 19959 ts_tablet_manager.cc:1950] T c1f15b9de572479aae6c2e644da3392a P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:22.893455 19959 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 153) took 1497 ms. Trace:
I20260925 13:56:22.893822 19959 rpcz_store.cc:276] 0925 13:56:21.395661 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:21.395771 (+ 110us) service_pool.cc:224] Handling call
0925 13:56:22.869414 (+1473643us) ts_tablet_manager.cc:1151] Deleting tablet c1f15b9de572479aae6c2e644da3392a
0925 13:56:22.869455 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:22.881035 (+ 11580us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:22.889434 (+ 8399us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:22.893391 (+ 3957us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1473134}
I20260925 13:56:22.895097 19959 tablet_replica.cc:333] T 2711e7a6973546b1822d93f91095dae2 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:22.895992 19959 raft_consensus.cc:2243] T 2711e7a6973546b1822d93f91095dae2 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:22.896456 19959 raft_consensus.cc:2272] T 2711e7a6973546b1822d93f91095dae2 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:22.897269 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet c1f15b9de572479aae6c2e644da3392a (table default.Table_ab754389c42247ccbed64c8f7c539d90 [id=9cb2dc12a91e4de087414b492cd0dcfb]) successfully deleted
I20260925 13:56:22.898837 19959 ts_tablet_manager.cc:1916] T 2711e7a6973546b1822d93f91095dae2 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:22.914892 19959 ts_tablet_manager.cc:1929] T 2711e7a6973546b1822d93f91095dae2 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:22.915289 19959 log.cc:1199] T 2711e7a6973546b1822d93f91095dae2 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/2711e7a6973546b1822d93f91095dae2
I20260925 13:56:22.915927 19959 ts_tablet_manager.cc:1950] T 2711e7a6973546b1822d93f91095dae2 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:22.917868 19959 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 158) took 1160 ms. Trace:
I20260925 13:56:22.918239 19959 rpcz_store.cc:276] 0925 13:56:21.757257 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:21.757357 (+ 100us) service_pool.cc:224] Handling call
0925 13:56:22.894935 (+1137578us) ts_tablet_manager.cc:1151] Deleting tablet 2711e7a6973546b1822d93f91095dae2
0925 13:56:22.894981 (+ 46us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:22.907066 (+ 12085us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:22.914768 (+ 7702us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:22.917804 (+ 3036us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1137052}
I20260925 13:56:22.919468 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 2711e7a6973546b1822d93f91095dae2 (table default.Table_83acf26db75e4c02b312cef4df19901f [id=7ec3d38b986747ca99d6a49e6a7dbe1a]) successfully deleted
I20260925 13:56:22.920017 19959 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 159) took 1100 ms. Trace:
I20260925 13:56:22.920374 19959 rpcz_store.cc:276] 0925 13:56:21.819741 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:21.825019 (+ 5278us) service_pool.cc:224] Handling call
0925 13:56:22.919644 (+1094625us) ts_tablet_manager.cc:1151] Deleting tablet 78d8c01c88d24deeae21d647e7f205ba
0925 13:56:22.919692 (+ 48us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:22.919961 (+ 269us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1094105}
W20260925 13:56:22.921519 19266 catalog_manager.cc:5043] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): delete failed for tablet 78d8c01c88d24deeae21d647e7f205ba because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 78d8c01c88d24deeae21d647e7f205ba already in progress: creating tablet
W20260925 13:56:22.922837 19266 catalog_manager.cc:5043] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): delete failed for tablet 4d01a791445149e7bee9493e1eec5c6b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4d01a791445149e7bee9493e1eec5c6b already in progress: creating tablet
I20260925 13:56:22.923688 19959 tablet_replica.cc:333] T 2ef3efe644364e93a089ddeed328389d P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:22.924365 19959 raft_consensus.cc:2243] T 2ef3efe644364e93a089ddeed328389d P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:22.924765 19959 raft_consensus.cc:2272] T 2ef3efe644364e93a089ddeed328389d P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:22.926306 19959 ts_tablet_manager.cc:1916] T 2ef3efe644364e93a089ddeed328389d P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:22.934372 20513 tablet.cc:1751] T b0a293e0094a4d9e937af6454e1ea16c P f790493ba7ad403993b678f2df2b55be: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 13:56:22.941184 19959 ts_tablet_manager.cc:1929] T 2ef3efe644364e93a089ddeed328389d P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:22.941548 19959 log.cc:1199] T 2ef3efe644364e93a089ddeed328389d P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/2ef3efe644364e93a089ddeed328389d
I20260925 13:56:22.942166 19959 ts_tablet_manager.cc:1950] T 2ef3efe644364e93a089ddeed328389d P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:22.944931 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 2ef3efe644364e93a089ddeed328389d (table default.Table_dfe9d9863d364dfc8df4c405e87dd635 [id=7b4bc216530346b6ab44c1ce25fe2112]) successfully deleted
I20260925 13:56:22.945534 19274 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 953d41f26f5f4027acfd374c6ec4ee75
W20260925 13:56:22.946694 19266 catalog_manager.cc:5030] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): delete failed for tablet e6e59d6f4f6e4de1be1cf450620cfa59 because the tablet was not found. No further retry: Not found: Tablet not found: e6e59d6f4f6e4de1be1cf450620cfa59
I20260925 13:56:22.946995 20455 raft_consensus.cc:493] T 2ef3efe644364e93a089ddeed328389d P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:22.947563 20455 raft_consensus.cc:515] T 2ef3efe644364e93a089ddeed328389d P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:22.948341 19804 tablet_service.cc:1558] Processing DeleteTablet for tablet 953d41f26f5f4027acfd374c6ec4ee75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:22 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:22.949210 19959 tablet_replica.cc:333] T ec7fe5c834be4a51b627b8b1c3ebe888 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
W20260925 13:56:22.951272 19490 leader_election.cc:343] T 2ef3efe644364e93a089ddeed328389d P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 2ef3efe644364e93a089ddeed328389d
I20260925 13:56:22.952853 19520 tablet_service.cc:1558] Processing DeleteTablet for tablet 953d41f26f5f4027acfd374c6ec4ee75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:22 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:22.959308 19938 tablet_bootstrap.cc:492] T 9059655f2c3d4e86b3b389b5b9a68891 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:22.959826 19938 ts_tablet_manager.cc:1403] T 9059655f2c3d4e86b3b389b5b9a68891 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.255s user 0.003s sys 0.014s
I20260925 13:56:22.962334 19938 raft_consensus.cc:359] T 9059655f2c3d4e86b3b389b5b9a68891 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:22.963011 19938 raft_consensus.cc:385] T 9059655f2c3d4e86b3b389b5b9a68891 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:22.997319 20478 raft_consensus.cc:493] T a26f5a02311c435a8c2434fd030c80a6 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:22.998004 20478 raft_consensus.cc:515] T a26f5a02311c435a8c2434fd030c80a6 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:23.001084 20478 leader_election.cc:290] T a26f5a02311c435a8c2434fd030c80a6 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:22.963310 19938 raft_consensus.cc:740] T 9059655f2c3d4e86b3b389b5b9a68891 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:23.006400 19938 consensus_queue.cc:260] T 9059655f2c3d4e86b3b389b5b9a68891 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:23.007876 19938 ts_tablet_manager.cc:1434] T 9059655f2c3d4e86b3b389b5b9a68891 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.048s user 0.005s sys 0.001s
I20260925 13:56:23.009430 19938 tablet_bootstrap.cc:492] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:23.020955 19277 catalog_manager.cc:6128] Not sending DeleteTablet requests; no consensus state for tablet fb71ac4a12b34449a6ad9d29263429b5
I20260925 13:56:23.025226 19955 ts_tablet_manager.cc:1929] T f514fa52e7524cd3ba07f3f65de532d4 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:23.025594 19955 log.cc:1199] T f514fa52e7524cd3ba07f3f65de532d4 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/f514fa52e7524cd3ba07f3f65de532d4
I20260925 13:56:23.026190 19955 ts_tablet_manager.cc:1950] T f514fa52e7524cd3ba07f3f65de532d4 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:22.704443 19942 raft_consensus.cc:359] T ec7fe5c834be4a51b627b8b1c3ebe888 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:23.043576 19942 raft_consensus.cc:385] T ec7fe5c834be4a51b627b8b1c3ebe888 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:23.043951 19942 raft_consensus.cc:740] T ec7fe5c834be4a51b627b8b1c3ebe888 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:23.044574 19942 consensus_queue.cc:260] T ec7fe5c834be4a51b627b8b1c3ebe888 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
W20260925 13:56:22.975730 19631 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.633s user 0.000s sys 0.015s
W20260925 13:56:23.045361 19631 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.703s user 0.000s sys 0.017s
W20260925 13:56:23.053643 19629 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.193:38269, user_credentials={real_user=slave}} blocked reactor thread for 268863us
I20260925 13:56:23.058490 19838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a86ea84d1d49d1b0950b4ce3a3d770" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:23.059077 19838 raft_consensus.cc:2468] T 70a86ea84d1d49d1b0950b4ce3a3d770 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:23.072227 19832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a26f5a02311c435a8c2434fd030c80a6" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:23.076465 19838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3cf9676160c4af08f1f62ae952b4c57" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:23.076699 19832 raft_consensus.cc:2468] T a26f5a02311c435a8c2434fd030c80a6 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:23.077013 19838 raft_consensus.cc:2468] T e3cf9676160c4af08f1f62ae952b4c57 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:23.094692 19277 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_6d0ced912e4e4c188553d0839f794603" } new_table_name: "default.Table_01f0109a149148dfaf1974cee0c3e42b" modify_external_catalogs: true
I20260925 13:56:23.096709 20011 ts_tablet_manager.cc:1929] T 12984db09f34472ba365fdf63a33b720 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:23.097040 20011 log.cc:1199] T 12984db09f34472ba365fdf63a33b720 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/12984db09f34472ba365fdf63a33b720
I20260925 13:56:23.097697 20011 ts_tablet_manager.cc:1950] T 12984db09f34472ba365fdf63a33b720 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:23.099339 19271 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_f1564f324ef645f9a62c24fb5d1ebd06" } new_table_name: "default.Table_17642b3f89974e5c97036637fe6d04d3" modify_external_catalogs: true
I20260925 13:56:23.109205 19272 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_83fc37dc7db944d0951289432d899876"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:23.118743 19272 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_83fc37dc7db944d0951289432d899876 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:23.120712 19939 tablet_bootstrap.cc:492] T 28219970809145bc9d4e2a3289a9479a P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:23.121146 19939 ts_tablet_manager.cc:1403] T 28219970809145bc9d4e2a3289a9479a P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.619s user 0.017s sys 0.017s
I20260925 13:56:23.123967 19939 raft_consensus.cc:359] T 28219970809145bc9d4e2a3289a9479a P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:23.124665 19939 raft_consensus.cc:385] T 28219970809145bc9d4e2a3289a9479a P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:23.124954 19939 raft_consensus.cc:740] T 28219970809145bc9d4e2a3289a9479a P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:23.125613 19939 consensus_queue.cc:260] T 28219970809145bc9d4e2a3289a9479a P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:23.126996 19939 ts_tablet_manager.cc:1434] T 28219970809145bc9d4e2a3289a9479a P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.006s user 0.001s sys 0.003s
I20260925 13:56:23.131547 19938 tablet_bootstrap.cc:654] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
W20260925 13:56:23.135752 19277 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f50ea68e3b3c4459b3f5e5b0df724724 consensus state has no leader
I20260925 13:56:23.149819 19277 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_1e8d4bb8dcbb45a3b882ed664417c23b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:23.151336 19277 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1e8d4bb8dcbb45a3b882ed664417c23b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:23.155702 19279 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_079723ff92144ba293733e54ab25d782" } modify_external_catalogs: true
I20260925 13:56:23.156205 19279 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_079723ff92144ba293733e54ab25d782" } modify_external_catalogs: true
I20260925 13:56:23.158167 19552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a26f5a02311c435a8c2434fd030c80a6" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
W20260925 13:56:23.159200 19490 consensus_peers.cc:597] T c1f15b9de572479aae6c2e644da3392a P 8f578869bc084a5d8a9a69f143954eb3 -> Peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Couldn't send request to peer f790493ba7ad403993b678f2df2b55be. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c1f15b9de572479aae6c2e644da3392a. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:23.165818 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 96) took 6003 ms. Trace:
I20260925 13:56:23.166805 19938 tablet_bootstrap.cc:492] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:23.167212 19938 ts_tablet_manager.cc:1403] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.158s user 0.006s sys 0.002s
I20260925 13:56:23.173334 19938 raft_consensus.cc:359] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:23.173941 19938 raft_consensus.cc:385] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:23.174196 19938 raft_consensus.cc:740] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:23.175210 19938 consensus_queue.cc:260] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:23.177098 19938 ts_tablet_manager.cc:1434] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.010s user 0.004s sys 0.000s
I20260925 13:56:23.177914 19938 tablet_bootstrap.cc:492] T f16050752ad44720ba8703ef10bbb359 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:23.094645 20455 leader_election.cc:290] T 2ef3efe644364e93a089ddeed328389d P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:23.175083 19939 tablet_bootstrap.cc:492] T 0ad6442b2de249119ba0ef85dfc06da9 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:23.184641 19942 ts_tablet_manager.cc:1434] T ec7fe5c834be4a51b627b8b1c3ebe888 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.483s user 0.005s sys 0.001s
I20260925 13:56:23.199326 19959 raft_consensus.cc:2243] T ec7fe5c834be4a51b627b8b1c3ebe888 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:23.181540 19955 rpcz_store.cc:276] 0925 13:56:17.162681 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:17.162803 (+ 122us) service_pool.cc:224] Handling call
0925 13:56:22.216296 (+5053493us) ts_tablet_manager.cc:1151] Deleting tablet f514fa52e7524cd3ba07f3f65de532d4
0925 13:56:22.216364 (+ 68us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:22.440540 (+224176us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:23.025092 (+584552us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:23.165728 (+140636us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28594,"tablet-delete.queue_time_us":5052915}
I20260925 13:56:23.200058 19959 raft_consensus.cc:2272] T ec7fe5c834be4a51b627b8b1c3ebe888 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:23.201745 19959 ts_tablet_manager.cc:1916] T ec7fe5c834be4a51b627b8b1c3ebe888 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:23.207340 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet f514fa52e7524cd3ba07f3f65de532d4 (table default.Table_ba97a54a19744b058cffd5638ffb97db [id=7e7ee975d4bb457eb3cc4632e12f3cd5]) successfully deleted
I20260925 13:56:23.211454 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ef3efe644364e93a089ddeed328389d" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:23.212167 19695 raft_consensus.cc:2393] T 2ef3efe644364e93a089ddeed328389d P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f578869bc084a5d8a9a69f143954eb3 in current term 1: Already voted for candidate 7c1b52895a14482fb54a6758f5054f8f in this term.
I20260925 13:56:23.208884 20511 tablet.cc:1751] T b0a293e0094a4d9e937af6454e1ea16c P 8f578869bc084a5d8a9a69f143954eb3: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 13:56:23.219249 19942 tablet_bootstrap.cc:492] T 436b8d46e1d244deb8bb9dc128ee6279 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
W20260925 13:56:23.227722 19773 leader_election.cc:343] T a26f5a02311c435a8c2434fd030c80a6 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:23.232565 19280 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_5f090a5fffe7463c9dcac5a5023da7d1" } modify_external_catalogs: true
I20260925 13:56:23.233052 19280 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_5f090a5fffe7463c9dcac5a5023da7d1" } modify_external_catalogs: true
I20260925 13:56:23.234674 20513 tablet.cc:1751] T b12a6c6cff024d70b53cc2c9169dd48b P f790493ba7ad403993b678f2df2b55be: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:23.236213 19959 ts_tablet_manager.cc:1929] T ec7fe5c834be4a51b627b8b1c3ebe888 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:23.236544 19959 log.cc:1199] T ec7fe5c834be4a51b627b8b1c3ebe888 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/ec7fe5c834be4a51b627b8b1c3ebe888
I20260925 13:56:23.237133 19959 ts_tablet_manager.cc:1950] T ec7fe5c834be4a51b627b8b1c3ebe888 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:23.237488 19276 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_c77ea16965bc41eca6f65334f0219a09"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:23.238600 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet ec7fe5c834be4a51b627b8b1c3ebe888 (table default.Table_86e7099f24054da589b5a6f7de9ab131 [id=191bb6f9e3b14882aabe499934d8eb57]) successfully deleted
W20260925 13:56:23.238886 19276 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c77ea16965bc41eca6f65334f0219a09 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 13:56:23.250649 19266 catalog_manager.cc:5043] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): delete failed for tablet 908df914cfec490cb7058dbe7b501c97 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 908df914cfec490cb7058dbe7b501c97 already in progress: creating tablet
I20260925 13:56:23.232450 19955 tablet_replica.cc:333] T 1da6eadb1648479889027fa911f4676b P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:23.251848 19955 raft_consensus.cc:2243] T 1da6eadb1648479889027fa911f4676b P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:23.237398 19278 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_1de14a8309044a888188fc1c228700fe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:23.254189 19278 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1de14a8309044a888188fc1c228700fe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:23.254751 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 151) took 2541 ms. Trace:
I20260925 13:56:23.256348 19552 raft_consensus.cc:3060] T 28219970809145bc9d4e2a3289a9479a P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:23.256455 19959 tablet_replica.cc:333] T c161ae3edfed4f8488c770ecc6332607 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:23.257246 19959 raft_consensus.cc:2243] T c161ae3edfed4f8488c770ecc6332607 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:23.257745 19959 raft_consensus.cc:2272] T c161ae3edfed4f8488c770ecc6332607 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:23.260140 19959 ts_tablet_manager.cc:1916] T c161ae3edfed4f8488c770ecc6332607 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:23.275535 19672 tablet_service.cc:1511] Processing CreateTablet for tablet de2b3e07bd634bc08368989a9062ba08 (DEFAULT_TABLE table=default.Table_dffaefeede134bde97c6bcdff8bfc010 [id=be2ddd2647094a8788d52ef46500426f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:23.275858 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a26f5a02311c435a8c2434fd030c80a6" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:23.276425 19692 raft_consensus.cc:2468] T a26f5a02311c435a8c2434fd030c80a6 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 0.
I20260925 13:56:23.276854 19672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de2b3e07bd634bc08368989a9062ba08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:23.277411 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7fe5c834be4a51b627b8b1c3ebe888" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:23.277935 19695 raft_consensus.cc:2468] T ec7fe5c834be4a51b627b8b1c3ebe888 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 0.
I20260925 13:56:23.278645 19668 tablet_service.cc:1558] Processing DeleteTablet for tablet 953d41f26f5f4027acfd374c6ec4ee75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:22 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:23.280365 19673 tablet_service.cc:1511] Processing CreateTablet for tablet a0109e4115174d8eb8174a1a2236150f (DEFAULT_TABLE table=default.Table_92f5e6ebf342495b8333c8da779d6998 [id=af68d1e4a6874f999362917b2feb6efe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:23.281193 19673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0109e4115174d8eb8174a1a2236150f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:23.281523 19773 leader_election.cc:304] T a26f5a02311c435a8c2434fd030c80a6 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters: 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:23.282763 19773 leader_election.cc:304] T ec7fe5c834be4a51b627b8b1c3ebe888 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters: 8f578869bc084a5d8a9a69f143954eb3
W20260925 13:56:23.283620 19773 raft_consensus.cc:2686] T ec7fe5c834be4a51b627b8b1c3ebe888 P f790493ba7ad403993b678f2df2b55be: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260925 13:56:23.289162 20478 raft_consensus.cc:493] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:23.289582 20478 raft_consensus.cc:515] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:23.292217 19670 tablet_service.cc:1511] Processing CreateTablet for tablet e4b29a3d0edc481cba6c61cfb78f8f40 (DEFAULT_TABLE table=default.Table_240f5ce2f31541eda914442e2cab887a [id=45fc848c759b4365a5c69016b6aa3309]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:23.293290 19670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4b29a3d0edc481cba6c61cfb78f8f40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:23.302055 20407 consensus_queue.cc:1048] T 28219970809145bc9d4e2a3289a9479a P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:23.305832 19955 raft_consensus.cc:2272] T 1da6eadb1648479889027fa911f4676b P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:23.309032 19488 leader_election.cc:304] T 2ef3efe644364e93a089ddeed328389d P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3; no voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:23.316576 20478 leader_election.cc:290] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:23.317037 20478 raft_consensus.cc:2804] T a26f5a02311c435a8c2434fd030c80a6 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:23.317322 20478 raft_consensus.cc:493] T a26f5a02311c435a8c2434fd030c80a6 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:23.317611 20478 raft_consensus.cc:3060] T a26f5a02311c435a8c2434fd030c80a6 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
W20260925 13:56:23.318079 19271 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ab1ba6ef73a046b59ffac150f7c73953 consensus state has no leader
I20260925 13:56:23.255088 20011 rpcz_store.cc:276] 0925 13:56:20.713816 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:20.736511 (+ 22695us) service_pool.cc:224] Handling call
0925 13:56:22.293366 (+1556855us) ts_tablet_manager.cc:1151] Deleting tablet 12984db09f34472ba365fdf63a33b720
0925 13:56:22.293406 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:22.423399 (+129993us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:23.096594 (+673195us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:23.254677 (+158083us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1556282}
I20260925 13:56:23.320363 20011 tablet_replica.cc:333] T 137c43fb74ec4cc48d30ed3edac81d98 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:23.321072 20011 raft_consensus.cc:2243] T 137c43fb74ec4cc48d30ed3edac81d98 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:23.322324 20011 raft_consensus.cc:2272] T 137c43fb74ec4cc48d30ed3edac81d98 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:23.323433 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 12984db09f34472ba365fdf63a33b720 (table default.Table_10868b99c6ec43b2a38a3a4a5bdcf960 [id=46b6ad1a631c4c08a05ef5a1b5f62ab9]) successfully deleted
I20260925 13:56:23.325515 19939 tablet_bootstrap.cc:654] T 0ad6442b2de249119ba0ef85dfc06da9 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:23.329255 19942 tablet_bootstrap.cc:654] T 436b8d46e1d244deb8bb9dc128ee6279 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:23.333223 19805 tablet_service.cc:1511] Processing CreateTablet for tablet fa8330217e794edf9e26a368d8b34685 (DEFAULT_TABLE table=default.Table_7786394fcec84a879a5da3eb2377ca74 [id=e633d43cec1d40f1b2df3ee09a1ca3d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:23.334280 19805 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa8330217e794edf9e26a368d8b34685. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:23.337172 19812 tablet_service.cc:1511] Processing CreateTablet for tablet fd448eab74b344838b3b92cad83a1294 (DEFAULT_TABLE table=default.Table_a51e5b80e56c43cbbd853f7fdc740887 [id=1dbe345269274eb9bf93b7658f9a41d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:23.338169 19812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd448eab74b344838b3b92cad83a1294. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:23.312840 20455 raft_consensus.cc:3060] T 2ef3efe644364e93a089ddeed328389d P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:23.348889 20011 ts_tablet_manager.cc:1916] T 137c43fb74ec4cc48d30ed3edac81d98 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:23.352874 19523 tablet_service.cc:1511] Processing CreateTablet for tablet 80ded435c38e4a84a639cd9dcad25cde (DEFAULT_TABLE table=default.Table_ff75f121abe34bc4a3a7338a2c5bb80c [id=51b0fcb6d89e40bd980250cc79165291]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:23.357661 19273 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet b12a6c6cff024d70b53cc2c9169dd48b (table default.Table_05aaf80be87b4d8489b392a7b7a382ea [id=54685374ee92481fbbf6543c920b782d]). Expected version 1 got 0
I20260925 13:56:23.358428 19273 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 3e6c8b9bbf4549b3a8685f886973ed9c (table default.Table_d0dadd9b31b143fd8dd5adcb7f752c5f [id=beee0ac537e44859b0f7c76d6808b0e0]). Expected version 1 got 0
I20260925 13:56:23.358922 19273 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 8dc5ea164d6749b0926c63d6bda9dcc5 (table default.Table_e0f7b807e8884431898298a1c635c784 [id=e49e015b173c48c1b6ef8112f60b33e4]). Expected version 1 got 0
I20260925 13:56:23.359979 19819 tablet_service.cc:1511] Processing CreateTablet for tablet 69934b9a49354c92b5b13e182247ef77 (DEFAULT_TABLE table=default.Table_1f476c49446b4488821bbbd7d02b71cd [id=50c3843b240b47e1bfd1bfd07e3d4875]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:23.360090 20524 tablet.cc:1751] T b12a6c6cff024d70b53cc2c9169dd48b P 7c1b52895a14482fb54a6758f5054f8f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:23.360329 20375 raft_consensus.cc:493] T 28219970809145bc9d4e2a3289a9479a P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:23.360966 19819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69934b9a49354c92b5b13e182247ef77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:23.360738 20375 raft_consensus.cc:515] T 28219970809145bc9d4e2a3289a9479a P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:23.364564 19817 tablet_service.cc:1511] Processing CreateTablet for tablet 80ded435c38e4a84a639cd9dcad25cde (DEFAULT_TABLE table=default.Table_ff75f121abe34bc4a3a7338a2c5bb80c [id=51b0fcb6d89e40bd980250cc79165291]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:23.365475 19817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80ded435c38e4a84a639cd9dcad25cde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:23.365998 20375 leader_election.cc:290] T 28219970809145bc9d4e2a3289a9479a P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:23.366214 19959 ts_tablet_manager.cc:1929] T c161ae3edfed4f8488c770ecc6332607 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:23.366544 19959 log.cc:1199] T c161ae3edfed4f8488c770ecc6332607 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/c161ae3edfed4f8488c770ecc6332607
I20260925 13:56:23.370620 19523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80ded435c38e4a84a639cd9dcad25cde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:23.373869 19519 tablet_service.cc:1511] Processing CreateTablet for tablet fd448eab74b344838b3b92cad83a1294 (DEFAULT_TABLE table=default.Table_a51e5b80e56c43cbbd853f7fdc740887 [id=1dbe345269274eb9bf93b7658f9a41d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:23.378780 20455 raft_consensus.cc:2749] T 2ef3efe644364e93a089ddeed328389d P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:23.382689 19693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8aa12e95dc434a98a426499593d9a5" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:23.383255 19673 tablet_service.cc:1511] Processing CreateTablet for tablet fa8330217e794edf9e26a368d8b34685 (DEFAULT_TABLE table=default.Table_7786394fcec84a879a5da3eb2377ca74 [id=e633d43cec1d40f1b2df3ee09a1ca3d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:23.383702 19669 tablet_service.cc:1511] Processing CreateTablet for tablet 69934b9a49354c92b5b13e182247ef77 (DEFAULT_TABLE table=default.Table_1f476c49446b4488821bbbd7d02b71cd [id=50c3843b240b47e1bfd1bfd07e3d4875]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:23.384254 19673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa8330217e794edf9e26a368d8b34685. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:23.370435 19818 tablet_service.cc:1511] Processing CreateTablet for tablet 9c5ef5b050624127990737acab3a8e9f (DEFAULT_TABLE table=default.Table_cbf20322808c49ef86dff2c576d5654c [id=64639a1e491042f7b89cd60d23d68a6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:23.387605 19672 tablet_service.cc:1511] Processing CreateTablet for tablet fd448eab74b344838b3b92cad83a1294 (DEFAULT_TABLE table=default.Table_a51e5b80e56c43cbbd853f7fdc740887 [id=1dbe345269274eb9bf93b7658f9a41d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:23.387620 19669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69934b9a49354c92b5b13e182247ef77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:23.370645 19804 tablet_service.cc:1511] Processing CreateTablet for tablet 869bc08603b444b4885603d81b97a1db (DEFAULT_TABLE table=default.Table_45345226ecf3438eb41e5b811058cfee [id=6790a5ada93349b6a8d75b7d149b6add]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:23.391407 19818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c5ef5b050624127990737acab3a8e9f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:23.391178 19488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.StartTabletCopy -> {remote=127.18.182.194:38641, user_credentials={real_user=slave}} blocked reactor thread for 78057.2us
I20260925 13:56:23.395802 19518 tablet_service.cc:1511] Processing CreateTablet for tablet 869bc08603b444b4885603d81b97a1db (DEFAULT_TABLE table=default.Table_45345226ecf3438eb41e5b811058cfee [id=6790a5ada93349b6a8d75b7d149b6add]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:23.396183 19804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 869bc08603b444b4885603d81b97a1db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:23.399039 19524 tablet_service.cc:1511] Processing CreateTablet for tablet fa8330217e794edf9e26a368d8b34685 (DEFAULT_TABLE table=default.Table_7786394fcec84a879a5da3eb2377ca74 [id=e633d43cec1d40f1b2df3ee09a1ca3d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:23.399677 19672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd448eab74b344838b3b92cad83a1294. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:23.401305 19529 tablet_service.cc:1511] Processing CreateTablet for tablet 69934b9a49354c92b5b13e182247ef77 (DEFAULT_TABLE table=default.Table_1f476c49446b4488821bbbd7d02b71cd [id=50c3843b240b47e1bfd1bfd07e3d4875]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:23.403340 19519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd448eab74b344838b3b92cad83a1294. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:23.410811 19532 tablet_service.cc:1511] Processing CreateTablet for tablet 77f3683ff34342e8bc8ad5a4a349da6d (DEFAULT_TABLE table=default.Table_42018ccf33f9446eaa36e1a3a38dc1ef [id=1feb9ec286c146bab7265ace28d4f3ca]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 13:56:23.413987 19629 consensus_peers.cc:597] T f514fa52e7524cd3ba07f3f65de532d4 P 7c1b52895a14482fb54a6758f5054f8f -> Peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Couldn't send request to peer 8f578869bc084a5d8a9a69f143954eb3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f514fa52e7524cd3ba07f3f65de532d4. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:23.413414 19533 tablet_service.cc:1511] Processing CreateTablet for tablet 8e5f54ef08054e7fb231c361615e2ab2 (DEFAULT_TABLE table=default.Table_7046f5be3ed44b3e94bd4d14e34c25ee [id=d7aaf1f0836e45f7a6d666f4f430c6f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:23.367457 19959 ts_tablet_manager.cc:1950] T c161ae3edfed4f8488c770ecc6332607 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:23.418377 19959 tablet_replica.cc:333] T a26f5a02311c435a8c2434fd030c80a6 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:23.418208 19668 tablet_service.cc:1511] Processing CreateTablet for tablet 80ded435c38e4a84a639cd9dcad25cde (DEFAULT_TABLE table=default.Table_ff75f121abe34bc4a3a7338a2c5bb80c [id=51b0fcb6d89e40bd980250cc79165291]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:23.419215 19668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80ded435c38e4a84a639cd9dcad25cde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:23.424345 19674 tablet_service.cc:1511] Processing CreateTablet for tablet 9c5ef5b050624127990737acab3a8e9f (DEFAULT_TABLE table=default.Table_cbf20322808c49ef86dff2c576d5654c [id=64639a1e491042f7b89cd60d23d68a6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:23.425205 19674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c5ef5b050624127990737acab3a8e9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:23.428577 19676 tablet_service.cc:1511] Processing CreateTablet for tablet 869bc08603b444b4885603d81b97a1db (DEFAULT_TABLE table=default.Table_45345226ecf3438eb41e5b811058cfee [id=6790a5ada93349b6a8d75b7d149b6add]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:23.429512 19676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 869bc08603b444b4885603d81b97a1db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:23.430115 19525 tablet_service.cc:1511] Processing CreateTablet for tablet 9c5ef5b050624127990737acab3a8e9f (DEFAULT_TABLE table=default.Table_cbf20322808c49ef86dff2c576d5654c [id=64639a1e491042f7b89cd60d23d68a6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:23.438905 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaf29a2310a41d0971eded7ef51fdc9" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:23.440119 20407 raft_consensus.cc:493] T c161ae3edfed4f8488c770ecc6332607 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:23.440569 20407 raft_consensus.cc:515] T c161ae3edfed4f8488c770ecc6332607 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:23.442480 20407 leader_election.cc:290] T c161ae3edfed4f8488c770ecc6332607 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:23.444207 19667 tablet_service.cc:1511] Processing CreateTablet for tablet 540132db164c403da72b9377078c4bf0 (DEFAULT_TABLE table=default.Table_5321051e9f6e4cd28b9b77619c908dc9 [id=9dac142084c948f9a50cfc9acf24d461]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:23.445327 19667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 540132db164c403da72b9377078c4bf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:23.458166 19518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 869bc08603b444b4885603d81b97a1db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:23.467104 19533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e5f54ef08054e7fb231c361615e2ab2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:23.468181 19520 tablet_service.cc:1511] Processing CreateTablet for tablet 43692c9ae2dc460d9e32efad41c263c1 (DEFAULT_TABLE table=default.Table_9ae8fc862af64663b8f5f175a4085a03 [id=fa78293b520f449fa0e961b2a14a373b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:23.469179 19520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43692c9ae2dc460d9e32efad41c263c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:23.475270 19532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77f3683ff34342e8bc8ad5a4a349da6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:23.482653 19529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69934b9a49354c92b5b13e182247ef77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:23.485235 19522 tablet_service.cc:1511] Processing CreateTablet for tablet 65ad0197267d47158751bc44c539b555 (DEFAULT_TABLE table=default.Table_727855fadd224779a3638b48e6b0266f [id=4e57e9392b3f43e3a767b7c4b641d498]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:23.513628 19526 tablet_service.cc:1511] Processing CreateTablet for tablet ab1ba6ef73a046b59ffac150f7c73953 (DEFAULT_TABLE table=default.Table_17642b3f89974e5c97036637fe6d04d3 [id=4ca863b59dfd4410822e15705934b503]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:23.388242 19274 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_ab754389c42247ccbed64c8f7c539d90" } new_table_name: "default.Table_8b37ba6f398943df8ab95b19f0b7dec6" modify_external_catalogs: true
I20260925 13:56:23.514981 19832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28219970809145bc9d4e2a3289a9479a" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:23.515690 19832 raft_consensus.cc:2468] T 28219970809145bc9d4e2a3289a9479a P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:23.518774 19955 ts_tablet_manager.cc:1916] T 1da6eadb1648479889027fa911f4676b P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:23.519907 19524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa8330217e794edf9e26a368d8b34685. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:23.520907 19522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65ad0197267d47158751bc44c539b555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:23.521785 19526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab1ba6ef73a046b59ffac150f7c73953. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:23.522294 19273 catalog_manager.cc:5838] T 28219970809145bc9d4e2a3289a9479a P 7c1b52895a14482fb54a6758f5054f8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194). New cstate: current_term: 1 leader_uuid: "7c1b52895a14482fb54a6758f5054f8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:23.447256 19631 leader_election.cc:304] T 2ef3efe644364e93a089ddeed328389d P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters: 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:23.454851 20011 ts_tablet_manager.cc:1929] T 137c43fb74ec4cc48d30ed3edac81d98 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:23.492933 19672 tablet_service.cc:1511] Processing CreateTablet for tablet 43692c9ae2dc460d9e32efad41c263c1 (DEFAULT_TABLE table=default.Table_9ae8fc862af64663b8f5f175a4085a03 [id=fa78293b520f449fa0e961b2a14a373b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:23.531068 20011 log.cc:1199] T 137c43fb74ec4cc48d30ed3edac81d98 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/137c43fb74ec4cc48d30ed3edac81d98
I20260925 13:56:23.531600 19672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43692c9ae2dc460d9e32efad41c263c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:23.531939 20011 ts_tablet_manager.cc:1950] T 137c43fb74ec4cc48d30ed3edac81d98 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:23.532015 20407 raft_consensus.cc:2804] T 2ef3efe644364e93a089ddeed328389d P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:23.532562 20407 raft_consensus.cc:697] T 2ef3efe644364e93a089ddeed328389d P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Becoming Leader. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Running, Role: LEADER
I20260925 13:56:23.533792 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 152) took 2651 ms. Trace:
I20260925 13:56:23.533484 20407 consensus_queue.cc:237] T 2ef3efe644364e93a089ddeed328389d P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:23.534092 20011 rpcz_store.cc:276] 0925 13:56:20.882165 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:20.956025 (+ 73860us) service_pool.cc:224] Handling call
0925 13:56:23.320197 (+2364172us) ts_tablet_manager.cc:1151] Deleting tablet 137c43fb74ec4cc48d30ed3edac81d98
0925 13:56:23.320255 (+ 58us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:23.376353 (+ 56098us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:23.454743 (+ 78390us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:23.533733 (+ 78990us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2363508}
I20260925 13:56:23.535233 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 153) took 2512 ms. Trace:
I20260925 13:56:23.535553 20011 rpcz_store.cc:276] 0925 13:56:21.023267 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:21.023407 (+ 140us) service_pool.cc:224] Handling call
0925 13:56:23.534924 (+2511517us) ts_tablet_manager.cc:1151] Deleting tablet 5a9b3dfaa7474611a1824d63eb906b43
0925 13:56:23.534961 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:23.535188 (+ 227us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2509798}
I20260925 13:56:23.536438 20011 tablet_replica.cc:333] T 5788849b7336496488e5401b4ac91191 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
W20260925 13:56:23.535827 19631 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 88702us
I20260925 13:56:23.537101 20011 raft_consensus.cc:2243] T 5788849b7336496488e5401b4ac91191 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:23.537276 19631 leader_election.cc:304] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters: 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:23.537570 20011 pending_rounds.cc:70] T 5788849b7336496488e5401b4ac91191 P 7c1b52895a14482fb54a6758f5054f8f: Trying to abort 1 pending ops.
I20260925 13:56:23.537786 20011 pending_rounds.cc:77] T 5788849b7336496488e5401b4ac91191 P 7c1b52895a14482fb54a6758f5054f8f: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251408872312832 op_type: NO_OP noop_request { }
I20260925 13:56:23.538076 20011 raft_consensus.cc:2889] T 5788849b7336496488e5401b4ac91191 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:56:23.538400 20011 raft_consensus.cc:2272] T 5788849b7336496488e5401b4ac91191 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 13:56:23.541576 19264 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet ab1ba6ef73a046b59ffac150f7c73953 (table default.Table_17642b3f89974e5c97036637fe6d04d3 [id=4ca863b59dfd4410822e15705934b503], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet ab1ba6ef73a046b59ffac150f7c73953 consensus state has no leader
I20260925 13:56:23.543700 20526 raft_consensus.cc:493] T 9059655f2c3d4e86b3b389b5b9a68891 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:23.544206 20526 raft_consensus.cc:515] T 9059655f2c3d4e86b3b389b5b9a68891 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:23.546066 20526 leader_election.cc:290] T 9059655f2c3d4e86b3b389b5b9a68891 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
W20260925 13:56:23.546523 19773 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.194:38641, user_credentials={real_user=slave}} blocked reactor thread for 154850us
I20260925 13:56:23.547174 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet c161ae3edfed4f8488c770ecc6332607 (table default.Table_475efa7896f84e658eb81238ceb58d8f [id=5e975bd1dd6e487e97377bc2896e6a3f]) successfully deleted
I20260925 13:56:23.494814 19691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28219970809145bc9d4e2a3289a9479a" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:23.496697 19673 tablet_service.cc:1511] Processing CreateTablet for tablet 8e5f54ef08054e7fb231c361615e2ab2 (DEFAULT_TABLE table=default.Table_7046f5be3ed44b3e94bd4d14e34c25ee [id=d7aaf1f0836e45f7a6d666f4f430c6f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:23.549757 19673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e5f54ef08054e7fb231c361615e2ab2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:23.549630 19804 tablet_service.cc:1511] Processing CreateTablet for tablet 8e5f54ef08054e7fb231c361615e2ab2 (DEFAULT_TABLE table=default.Table_7046f5be3ed44b3e94bd4d14e34c25ee [id=d7aaf1f0836e45f7a6d666f4f430c6f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:23.550642 19804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e5f54ef08054e7fb231c361615e2ab2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:23.556355 19818 tablet_service.cc:1511] Processing CreateTablet for tablet 43692c9ae2dc460d9e32efad41c263c1 (DEFAULT_TABLE table=default.Table_9ae8fc862af64663b8f5f175a4085a03 [id=fa78293b520f449fa0e961b2a14a373b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:23.557551 19818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43692c9ae2dc460d9e32efad41c263c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:23.500152 19674 tablet_service.cc:1511] Processing CreateTablet for tablet 77f3683ff34342e8bc8ad5a4a349da6d (DEFAULT_TABLE table=default.Table_42018ccf33f9446eaa36e1a3a38dc1ef [id=1feb9ec286c146bab7265ace28d4f3ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:23.559916 19674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77f3683ff34342e8bc8ad5a4a349da6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:23.568197 19812 tablet_service.cc:1511] Processing CreateTablet for tablet 65ad0197267d47158751bc44c539b555 (DEFAULT_TABLE table=default.Table_727855fadd224779a3638b48e6b0266f [id=4e57e9392b3f43e3a767b7c4b641d498]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:23.569259 19812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65ad0197267d47158751bc44c539b555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:23.577833 19515 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:40014 (request call id 26) took 1273 ms. Trace:
I20260925 13:56:23.578250 19515 rpcz_store.cc:276] 0925 13:56:22.256023 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:22.256138 (+ 115us) service_pool.cc:224] Handling call
0925 13:56:22.937933 (+681795us) spinlock_profiling.cc:243] Waited 559 ms on lock 0x7b5400083ed4. stack: 00007f1358052323 00007f1358052220 00007f1357bf6574 0000000000516964 00000000005168f9 00007f135f14faa1 00007f135f1e06b8 00007f136025198d 00007f135e9a1104 00007f135e9a1092 00007f135e9a0fc1 00007f135e9a0f3c 00007f135e9a01f2 00007f135ceb509c 00007f135ceb44d6 00007f135ceb3e7f
0925 13:56:23.230973 (+293040us) spinlock_profiling.cc:243] Waited 291 ms on lock 0x7b54000d11d4. stack: 00007f1358052323 00007f1358052220 00007f1357bf6574 0000000000516964 00000000005168f9 00007f135f14faa1 00007f135f1e06b8 00007f136025198d 00007f135e9a1104 00007f135e9a1092 00007f135e9a0fc1 00007f135e9a0f3c 00007f135e9a01f2 00007f135ceb509c 00007f135ceb44d6 00007f135ceb3e7f
0925 13:56:23.335439 (+104466us) spinlock_profiling.cc:243] Waited 71.8 ms on lock 0x7b54000c2954. stack: 00007f1358052323 00007f1358052220 00007f1357bf6574 0000000000516964 00000000005168f9 00007f135f14faa1 00007f135f1e06b8 00007f136025198d 00007f135e9a1104 00007f135e9a1092 00007f135e9a0fc1 00007f135e9a0f3c 00007f135e9a01f2 00007f135ceb509c 00007f135ceb44d6 00007f135ceb3e7f
0925 13:56:23.528942 (+193503us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2121155072}
I20260925 13:56:23.584304 20511 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:40018 (request call id 173) took 1106 ms. Trace:
W20260925 13:56:23.595798 19773 leader_election.cc:343] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:23.596403 19773 leader_election.cc:343] T 3aaf29a2310a41d0971eded7ef51fdc9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:23.596680 19801 tablet_service.cc:1511] Processing CreateTablet for tablet ab1ba6ef73a046b59ffac150f7c73953 (DEFAULT_TABLE table=default.Table_17642b3f89974e5c97036637fe6d04d3 [id=4ca863b59dfd4410822e15705934b503]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:23.598287 19801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab1ba6ef73a046b59ffac150f7c73953. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:23.599129 20011 ts_tablet_manager.cc:1916] T 5788849b7336496488e5401b4ac91191 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:23.558214 19525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c5ef5b050624127990737acab3a8e9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:23.603891 20435 raft_consensus.cc:2804] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:23.604254 20435 raft_consensus.cc:493] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:23.604573 20435 raft_consensus.cc:3060] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
W20260925 13:56:23.619737 19631 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 82565.6us
I20260925 13:56:23.620435 19631 leader_election.cc:304] T a26f5a02311c435a8c2434fd030c80a6 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters: 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:23.646126 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c161ae3edfed4f8488c770ecc6332607" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:23.647091 19551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9059655f2c3d4e86b3b389b5b9a68891" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:23.648655 19552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8aa12e95dc434a98a426499593d9a5" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:23.656629 19773 leader_election.cc:304] T 3aaf29a2310a41d0971eded7ef51fdc9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:23.657812 20522 raft_consensus.cc:2749] T 3aaf29a2310a41d0971eded7ef51fdc9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:23.659076 20491 raft_consensus.cc:2804] T a26f5a02311c435a8c2434fd030c80a6 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:23.659497 20491 raft_consensus.cc:493] T a26f5a02311c435a8c2434fd030c80a6 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:23.659881 20491 raft_consensus.cc:3060] T a26f5a02311c435a8c2434fd030c80a6 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:23.662138 19939 tablet_bootstrap.cc:492] T 0ad6442b2de249119ba0ef85dfc06da9 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:23.666747 19939 ts_tablet_manager.cc:1403] T 0ad6442b2de249119ba0ef85dfc06da9 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.492s user 0.005s sys 0.007s
I20260925 13:56:23.667783 20491 raft_consensus.cc:515] T a26f5a02311c435a8c2434fd030c80a6 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:23.669833 20491 leader_election.cc:290] T a26f5a02311c435a8c2434fd030c80a6 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
W20260925 13:56:23.672940 19631 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 52603.7us
I20260925 13:56:23.676998 19273 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet e3cf9676160c4af08f1f62ae952b4c57 (table default.Table_4d6d747ea4034b3ab48710e10ff4544f [id=d6e6dd0772bf4901b696e11b10e41a48]). Expected version 1 got 0
I20260925 13:56:23.697297 20478 raft_consensus.cc:515] T a26f5a02311c435a8c2434fd030c80a6 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:23.698158 19959 raft_consensus.cc:2243] T a26f5a02311c435a8c2434fd030c80a6 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:23.699038 19959 raft_consensus.cc:2272] T a26f5a02311c435a8c2434fd030c80a6 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:23.701117 20478 leader_election.cc:290] T a26f5a02311c435a8c2434fd030c80a6 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:23.703984 19631 leader_election.cc:304] T e3cf9676160c4af08f1f62ae952b4c57 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters: 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:23.705880 20533 tablet.cc:1751] T b12a6c6cff024d70b53cc2c9169dd48b P 8f578869bc084a5d8a9a69f143954eb3: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:23.706136 20535 tablet.cc:1751] T b0a293e0094a4d9e937af6454e1ea16c P 7c1b52895a14482fb54a6758f5054f8f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 13:56:23.706669 20407 raft_consensus.cc:2804] T e3cf9676160c4af08f1f62ae952b4c57 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:23.707000 20407 raft_consensus.cc:493] T e3cf9676160c4af08f1f62ae952b4c57 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:23.707315 20407 raft_consensus.cc:3060] T e3cf9676160c4af08f1f62ae952b4c57 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:23.693467 19942 tablet_bootstrap.cc:492] T 436b8d46e1d244deb8bb9dc128ee6279 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:23.719632 19942 ts_tablet_manager.cc:1403] T 436b8d46e1d244deb8bb9dc128ee6279 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.534s user 0.009s sys 0.003s
I20260925 13:56:23.722009 19942 raft_consensus.cc:359] T 436b8d46e1d244deb8bb9dc128ee6279 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:23.722599 19942 raft_consensus.cc:385] T 436b8d46e1d244deb8bb9dc128ee6279 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:23.722842 19942 raft_consensus.cc:740] T 436b8d46e1d244deb8bb9dc128ee6279 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:23.584758 20511 rpcz_store.cc:276] 0925 13:56:22.478055 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:22.479227 (+ 1172us) service_pool.cc:224] Handling call
0925 13:56:23.584229 (+1105002us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:22.571536 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0925 13:56:22.571908 (+ 372us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0925 13:56:22.571937 (+ 29us) alter_schema_op.cc:66] Acquired schema lock
0925 13:56:22.571979 (+ 42us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0925 13:56:22.572086 (+ 107us) alter_schema_op.cc:128] START. Timestamp: 1790344582572101
0925 13:56:22.572105 (+ 19us) op_driver.cc:348] REPLICATION: starting
0925 13:56:22.572659 (+ 554us) log.cc:844] Serialized 173 byte log entry
0925 13:56:22.851855 (+279196us) op_driver.cc:464] REPLICATION: finished
0925 13:56:23.208788 (+356933us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0925 13:56:23.216338 (+ 7550us) log.cc:844] Serialized 5 byte log entry
0925 13:56:23.526007 (+309669us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:23.583983 (+ 57976us) log.cc:844] Serialized 14 byte log entry
0925 13:56:23.584075 (+ 92us) alter_schema_op.cc:73] Released schema lock
0925 13:56:23.584106 (+ 31us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":356733,"prepare.queue_time_us":91323,"prepare.run_cpu_time_us":1877,"prepare.run_wall_time_us":34394,"raft.queue_time_us":322,"raft.run_cpu_time_us":1482,"raft.run_wall_time_us":1562,"replication_time_us":279616,"thread_start_us":457765,"threads_started":3,"wal-append.queue_time_us":27962}]]}
I20260925 13:56:23.732414 19273 catalog_manager.cc:5838] T 0ad6442b2de249119ba0ef85dfc06da9 P 7c1b52895a14482fb54a6758f5054f8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194). New cstate: current_term: 1 leader_uuid: "7c1b52895a14482fb54a6758f5054f8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:23.747793 19280 catalog_manager.cc:6128] Not sending DeleteTablet requests; no consensus state for tablet 511688e3ddf34dc6b433e42a555d076f
I20260925 13:56:23.760567 19939 raft_consensus.cc:359] T 0ad6442b2de249119ba0ef85dfc06da9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:23.761637 19939 raft_consensus.cc:385] T 0ad6442b2de249119ba0ef85dfc06da9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:23.762676 19939 raft_consensus.cc:740] T 0ad6442b2de249119ba0ef85dfc06da9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:23.763268 19939 consensus_queue.cc:260] T 0ad6442b2de249119ba0ef85dfc06da9 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:23.763649 19272 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_f20a7a770a834948b4fb54d6d19ae070"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:23.765161 19272 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f20a7a770a834948b4fb54d6d19ae070 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:23.618868 20435 raft_consensus.cc:515] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:23.772598 19276 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_af816cb5cc774d51813360f38299d90a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:23.774026 19276 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_af816cb5cc774d51813360f38299d90a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:23.783800 19278 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_5c6e964736a149d5b977b442a928a1ad" } new_table_name: "default.Table_f071a5a526034801828291eb010c40c0" modify_external_catalogs: true
I20260925 13:56:23.794804 19280 master_service.cc:960] ReplaceTablet: received request to replace tablet 8dc5ea164d6749b0926c63d6bda9dcc5 from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:23.796105 19275 master_service.cc:960] ReplaceTablet: received request to replace tablet c1f15b9de572479aae6c2e644da3392a from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:23.797062 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 137c43fb74ec4cc48d30ed3edac81d98 (table default.Table_5321051e9f6e4cd28b9b77619c908dc9 [id=9dac142084c948f9a50cfc9acf24d461]) successfully deleted
W20260925 13:56:23.797662 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 5a9b3dfaa7474611a1824d63eb906b43 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5a9b3dfaa7474611a1824d63eb906b43 already in progress: creating tablet
I20260925 13:56:23.809602 20435 leader_election.cc:290] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:23.828737 19490 leader_election.cc:304] T 28219970809145bc9d4e2a3289a9479a P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
W20260925 13:56:23.815665 19629 consensus_peers.cc:597] T 0ad6442b2de249119ba0ef85dfc06da9 P 7c1b52895a14482fb54a6758f5054f8f -> Peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Couldn't send request to peer 8f578869bc084a5d8a9a69f143954eb3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:23.823864 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9059655f2c3d4e86b3b389b5b9a68891" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:23.826901 19668 tablet_service.cc:1511] Processing CreateTablet for tablet 65ad0197267d47158751bc44c539b555 (DEFAULT_TABLE table=default.Table_727855fadd224779a3638b48e6b0266f [id=4e57e9392b3f43e3a767b7c4b641d498]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:23.832221 19668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65ad0197267d47158751bc44c539b555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:23.835390 19959 ts_tablet_manager.cc:1916] T a26f5a02311c435a8c2434fd030c80a6 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 13:56:23.735509 19773 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.194:38641, user_credentials={real_user=slave}} blocked reactor thread for 139162us
I20260925 13:56:23.655215 19938 tablet_bootstrap.cc:700] T f16050752ad44720ba8703ef10bbb359 P f790493ba7ad403993b678f2df2b55be: Time spent opening tablet: real 0.204s user 0.004s sys 0.000s
I20260925 13:56:23.845536 19938 tablet_bootstrap.cc:654] T f16050752ad44720ba8703ef10bbb359 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:23.848975 19279 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 436b8d46e1d244deb8bb9dc128ee6279
I20260925 13:56:23.811427 19819 tablet_service.cc:1511] Processing CreateTablet for tablet 77f3683ff34342e8bc8ad5a4a349da6d (DEFAULT_TABLE table=default.Table_42018ccf33f9446eaa36e1a3a38dc1ef [id=1feb9ec286c146bab7265ace28d4f3ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:23.858155 19819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77f3683ff34342e8bc8ad5a4a349da6d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:23.856235 19488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.194:38641, user_credentials={real_user=slave}} blocked reactor thread for 92016.2us
I20260925 13:56:23.883291 19938 tablet_bootstrap.cc:492] T f16050752ad44720ba8703ef10bbb359 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:23.883760 19938 ts_tablet_manager.cc:1403] T f16050752ad44720ba8703ef10bbb359 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.706s user 0.009s sys 0.002s
I20260925 13:56:23.884233 20375 raft_consensus.cc:2804] T 28219970809145bc9d4e2a3289a9479a P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 13:56:23.884886 20375 raft_consensus.cc:493] T 28219970809145bc9d4e2a3289a9479a P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7c1b52895a14482fb54a6758f5054f8f)
I20260925 13:56:23.885304 20375 raft_consensus.cc:3060] T 28219970809145bc9d4e2a3289a9479a P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Advancing to term 2
W20260925 13:56:23.886277 19278 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 505a24ebe7014f80b3c184329cc74a9a consensus state has no leader
I20260925 13:56:23.886868 19938 raft_consensus.cc:359] T f16050752ad44720ba8703ef10bbb359 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:23.887493 19938 raft_consensus.cc:385] T f16050752ad44720ba8703ef10bbb359 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:23.887749 19938 raft_consensus.cc:740] T f16050752ad44720ba8703ef10bbb359 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:23.888298 19938 consensus_queue.cc:260] T f16050752ad44720ba8703ef10bbb359 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:23.892835 19955 ts_tablet_manager.cc:1929] T 1da6eadb1648479889027fa911f4676b P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:23.893193 19955 log.cc:1199] T 1da6eadb1648479889027fa911f4676b P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/1da6eadb1648479889027fa911f4676b
I20260925 13:56:23.893859 19955 ts_tablet_manager.cc:1950] T 1da6eadb1648479889027fa911f4676b P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:23.901793 19938 ts_tablet_manager.cc:1434] T f16050752ad44720ba8703ef10bbb359 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.018s user 0.005s sys 0.000s
I20260925 13:56:23.902320 19942 consensus_queue.cc:260] T 436b8d46e1d244deb8bb9dc128ee6279 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:23.904449 19942 ts_tablet_manager.cc:1434] T 436b8d46e1d244deb8bb9dc128ee6279 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.185s user 0.003s sys 0.002s
I20260925 13:56:23.905617 19942 tablet_bootstrap.cc:492] T f16050752ad44720ba8703ef10bbb359 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
W20260925 13:56:23.906087 19490 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 77456.6us
I20260925 13:56:23.916935 19959 ts_tablet_manager.cc:1929] T a26f5a02311c435a8c2434fd030c80a6 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:23.917289 19959 log.cc:1199] T a26f5a02311c435a8c2434fd030c80a6 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/a26f5a02311c435a8c2434fd030c80a6
I20260925 13:56:23.917898 19959 ts_tablet_manager.cc:1950] T a26f5a02311c435a8c2434fd030c80a6 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:23.930347 19833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a26f5a02311c435a8c2434fd030c80a6" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
W20260925 13:56:23.936358 19773 leader_election.cc:343] T 9059655f2c3d4e86b3b389b5b9a68891 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:23.936921 19773 leader_election.cc:343] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:23.937259 19773 leader_election.cc:304] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:23.937906 20478 raft_consensus.cc:2749] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:23.940810 19836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a86ea84d1d49d1b0950b4ce3a3d770" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:23.941360 19836 raft_consensus.cc:3060] T 70a86ea84d1d49d1b0950b4ce3a3d770 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:23.952939 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a26f5a02311c435a8c2434fd030c80a6" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:23.956329 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a26f5a02311c435a8c2434fd030c80a6" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
W20260925 13:56:23.958408 19273 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.194:34125 (request call id 38) took 878 ms (client timeout 1000 ms). Trace:
I20260925 13:56:23.959218 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a86ea84d1d49d1b0950b4ce3a3d770" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
W20260925 13:56:23.968077 19773 leader_election.cc:343] T a26f5a02311c435a8c2434fd030c80a6 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:23.968533 19273 rpcz_store.cc:269] 0925 13:56:23.080338 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:23.080864 (+ 526us) service_pool.cc:224] Handling call
0925 13:56:23.111575 (+ 30711us) spinlock_profiling.cc:243] Waited 27.1 ms on lock 0x7b5400040454. stack: 00007fdf0a30b323 00007fdf0a30b220 00007fdf09eaf574 0000000000516964 00000000005168f9 00007fdf114095c0 00007fdf114a86fa 00007fdf13219d8c 00007fdf1331801e 00007fdf10ebcfb4 00007fdf10ebcf42 00007fdf10ebce71 00007fdf10ebcdec 00007fdf10ebc0a2 00007fdf0f16e09c 00007fdf0f16d4d6
0925 13:56:23.958338 (+846763us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:23.896024 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:23.896381 (+ 357us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:23.896405 (+ 24us) write_op.cc:435] Acquired schema lock
0925 13:56:23.896417 (+ 12us) tablet.cc:662] Decoding operations
0925 13:56:23.896686 (+ 269us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:23.896721 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:23.896743 (+ 22us) tablet.cc:685] Acquiring locks for 2 operations
0925 13:56:23.896984 (+ 241us) tablet.cc:701] Row locks acquired
0925 13:56:23.896998 (+ 14us) write_op.cc:260] PREPARE: finished
0925 13:56:23.897099 (+ 101us) write_op.cc:270] Start()
0925 13:56:23.897171 (+ 72us) write_op.cc:276] Timestamp: P: 1790344583897137 usec, L: 0
0925 13:56:23.897185 (+ 14us) op_driver.cc:348] REPLICATION: starting
0925 13:56:23.897638 (+ 453us) log.cc:844] Serialized 919 byte log entry
0925 13:56:23.923827 (+ 26189us) op_driver.cc:464] REPLICATION: finished
0925 13:56:23.925399 (+ 1572us) write_op.cc:301] APPLY: starting
0925 13:56:23.925475 (+ 76us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:23.925662 (+ 187us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:23.925786 (+ 124us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:23.926138 (+ 352us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:23.926364 (+ 226us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 13:56:23.926429 (+ 65us) write_op.cc:312] APPLY: finished
0925 13:56:23.926690 (+ 261us) log.cc:844] Serialized 29 byte log entry
0925 13:56:23.926890 (+ 200us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:23.927006 (+ 116us) write_op.cc:454] Released schema lock
0925 13:56:23.927053 (+ 47us) write_op.cc:341] FINISH: Updating metrics
0925 13:56:23.928359 (+ 1306us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:23.928399 (+ 40us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":22,"spinlock_wait_cycles":62259968,"child_traces":[["op",{"apply.queue_time_us":1354,"apply.run_cpu_time_us":1954,"apply.run_wall_time_us":1954,"num_ops":2,"prepare.queue_time_us":89,"prepare.run_cpu_time_us":1811,"prepare.run_wall_time_us":1810,"replication_time_us":26513}]]}
W20260925 13:56:23.974992 19490 consensus_peers.cc:597] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 8f578869bc084a5d8a9a69f143954eb3 -> Peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Couldn't send request to peer f790493ba7ad403993b678f2df2b55be. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0de8f2d2766c4ff197b552a1b5ed3d6b. This is attempt 1: this message will repeat every 5th retry.
W20260925 13:56:23.987869 19490 consensus_peers.cc:597] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 8f578869bc084a5d8a9a69f143954eb3 -> Peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Couldn't send request to peer f790493ba7ad403993b678f2df2b55be. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7ea4fd6817654bff9c0fe8c6e42ed6b9. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:23.992372 19280 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 8dc5ea164d6749b0926c63d6bda9dcc5
I20260925 13:56:23.994252 19280 catalog_manager.cc:6934] ReplaceTablet: tablet 8dc5ea164d6749b0926c63d6bda9dcc5 deleted and replaced by tablet d0caac86b259433e9c490aa71ac9d329
I20260925 13:56:23.999608 19817 tablet_service.cc:1558] Processing DeleteTablet for tablet 8dc5ea164d6749b0926c63d6bda9dcc5 with delete_type TABLET_DATA_DELETED (replaced by tablet d0caac86b259433e9c490aa71ac9d329) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:24.002395 19836 raft_consensus.cc:2468] T 70a86ea84d1d49d1b0950b4ce3a3d770 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
I20260925 13:56:24.030582 19812 tablet_service.cc:1558] Processing DeleteTablet for tablet 436b8d46e1d244deb8bb9dc128ee6279 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:23 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:23.858631 19939 ts_tablet_manager.cc:1434] T 0ad6442b2de249119ba0ef85dfc06da9 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.181s user 0.005s sys 0.000s
I20260925 13:56:24.042249 19939 tablet_bootstrap.cc:492] T c161ae3edfed4f8488c770ecc6332607 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:24.046675 19938 tablet_bootstrap.cc:492] T 908df914cfec490cb7058dbe7b501c97 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:24.047019 19959 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 179) took 1251 ms. Trace:
I20260925 13:56:24.047330 19959 rpcz_store.cc:276] 0925 13:56:22.795706 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:22.797084 (+ 1378us) service_pool.cc:224] Handling call
0925 13:56:23.418234 (+621150us) ts_tablet_manager.cc:1151] Deleting tablet a26f5a02311c435a8c2434fd030c80a6
0925 13:56:23.418271 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:23.698825 (+280554us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b54001a0254. stack: 00007f6c49bb9323 00007f6c49bb9220 00007f6c4975d574 0000000000516964 00000000005168f9 00007f6c50cb6aa1 00007f6c50d55b22 00007f6c5119650c 00007f6c51dffc3a 00007f6c51e30960 00007f6c51e16e6d 00007f6c51e16e19 00007f6c51e16da9 00007f6c51e16d71 00007f6c51e15f9d 00000000005bd4e4
0925 13:56:23.842324 (+143499us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:23.916818 (+ 74494us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:24.046955 (+130137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":642017408,"tablet-delete.queue_time_us":618241}
I20260925 13:56:24.048972 19959 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 180) took 1101 ms. Trace:
I20260925 13:56:24.049247 19959 rpcz_store.cc:276] 0925 13:56:22.948010 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:22.948167 (+ 157us) service_pool.cc:224] Handling call
0925 13:56:24.048651 (+1100484us) ts_tablet_manager.cc:1151] Deleting tablet 953d41f26f5f4027acfd374c6ec4ee75
0925 13:56:24.048700 (+ 49us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:24.048926 (+ 226us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1099858}
I20260925 13:56:24.052336 19938 tablet_bootstrap.cc:654] T 908df914cfec490cb7058dbe7b501c97 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:24.053280 19959 tablet_replica.cc:333] T 8dc5ea164d6749b0926c63d6bda9dcc5 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:24.054745 19959 raft_consensus.cc:2243] T 8dc5ea164d6749b0926c63d6bda9dcc5 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:24.055553 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet a26f5a02311c435a8c2434fd030c80a6 (table default.Table_65608937f2cd4986b2588a4831750601 [id=7079cc0c725b4578a1b11c7dd3ef9342]) successfully deleted
I20260925 13:56:24.055927 19959 raft_consensus.cc:2272] T 8dc5ea164d6749b0926c63d6bda9dcc5 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 13:56:24.056233 19266 catalog_manager.cc:5043] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): delete failed for tablet 953d41f26f5f4027acfd374c6ec4ee75 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 953d41f26f5f4027acfd374c6ec4ee75 already in progress: creating tablet
I20260925 13:56:24.014854 19277 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet 3e6c8b9bbf4549b3a8685f886973ed9c (table default.Table_d0dadd9b31b143fd8dd5adcb7f752c5f [id=beee0ac537e44859b0f7c76d6808b0e0]). Expected version 1 got 0
I20260925 13:56:24.059460 19959 ts_tablet_manager.cc:1916] T 8dc5ea164d6749b0926c63d6bda9dcc5 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 13:56:24.060822 19773 leader_election.cc:343] T 9059655f2c3d4e86b3b389b5b9a68891 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:24.061215 19773 leader_election.cc:304] T 9059655f2c3d4e86b3b389b5b9a68891 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:24.062099 20478 raft_consensus.cc:2749] T 9059655f2c3d4e86b3b389b5b9a68891 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:24.062898 20533 tablet.cc:1751] T 3e6c8b9bbf4549b3a8685f886973ed9c P 8f578869bc084a5d8a9a69f143954eb3: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:24.073138 19277 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet b12a6c6cff024d70b53cc2c9169dd48b (table default.Table_05aaf80be87b4d8489b392a7b7a382ea [id=54685374ee92481fbbf6543c920b782d]). Expected version 1 got 0
I20260925 13:56:24.074486 19277 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet b0a293e0094a4d9e937af6454e1ea16c (table default.Table_d65fe24e58a949bca116a094fbb025c5 [id=b673ac6542c5403fbf0fad16c4917dfd]). Expected version 2 got 1
I20260925 13:56:23.827877 19676 tablet_service.cc:1511] Processing CreateTablet for tablet ab1ba6ef73a046b59ffac150f7c73953 (DEFAULT_TABLE table=default.Table_17642b3f89974e5c97036637fe6d04d3 [id=4ca863b59dfd4410822e15705934b503]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:24.102836 19676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab1ba6ef73a046b59ffac150f7c73953. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:23.830215 20011 ts_tablet_manager.cc:1929] T 5788849b7336496488e5401b4ac91191 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:24.108456 20011 log.cc:1199] T 5788849b7336496488e5401b4ac91191 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/5788849b7336496488e5401b4ac91191
I20260925 13:56:24.109339 20011 ts_tablet_manager.cc:1950] T 5788849b7336496488e5401b4ac91191 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
W20260925 13:56:24.110126 19490 consensus_peers.cc:597] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3 -> Peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Couldn't send request to peer f790493ba7ad403993b678f2df2b55be. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:24.111487 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 154) took 3048 ms. Trace:
W20260925 13:56:23.928421 19629 leader_election.cc:343] T c161ae3edfed4f8488c770ecc6332607 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:24.112028 19629 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.193:38269, user_credentials={real_user=slave}} blocked reactor thread for 183676us
W20260925 13:56:24.113072 19629 leader_election.cc:343] T a26f5a02311c435a8c2434fd030c80a6 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:24.113664 19629 leader_election.cc:343] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:24.114509 20011 rpcz_store.cc:276] 0925 13:56:21.063274 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:21.065030 (+ 1756us) service_pool.cc:224] Handling call
0925 13:56:23.536316 (+2471286us) ts_tablet_manager.cc:1151] Deleting tablet 5788849b7336496488e5401b4ac91191
0925 13:56:23.536351 (+ 35us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:23.610555 (+ 74204us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:23.830109 (+219554us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:24.111429 (+281320us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59040,"tablet-delete.queue_time_us":2470689}
I20260925 13:56:24.115849 20011 tablet_replica.cc:333] T c1f15b9de572479aae6c2e644da3392a P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:24.118182 20523 consensus_queue.cc:1048] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:24.120687 20523 consensus_queue.cc:1048] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:24.123695 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a26f5a02311c435a8c2434fd030c80a6" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:24.124503 19695 raft_consensus.cc:2393] T a26f5a02311c435a8c2434fd030c80a6 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f790493ba7ad403993b678f2df2b55be in current term 1: Already voted for candidate 7c1b52895a14482fb54a6758f5054f8f in this term.
I20260925 13:56:24.129480 19938 tablet_bootstrap.cc:492] T 908df914cfec490cb7058dbe7b501c97 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:24.129891 19938 ts_tablet_manager.cc:1403] T 908df914cfec490cb7058dbe7b501c97 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.083s user 0.034s sys 0.019s
I20260925 13:56:24.133369 19938 raft_consensus.cc:359] T 908df914cfec490cb7058dbe7b501c97 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:24.133970 19938 raft_consensus.cc:385] T 908df914cfec490cb7058dbe7b501c97 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:24.134214 19938 raft_consensus.cc:740] T 908df914cfec490cb7058dbe7b501c97 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:24.135043 19938 consensus_queue.cc:260] T 908df914cfec490cb7058dbe7b501c97 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:24.141687 19938 ts_tablet_manager.cc:1434] T 908df914cfec490cb7058dbe7b501c97 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.011s user 0.007s sys 0.000s
I20260925 13:56:24.147696 20375 raft_consensus.cc:515] T 28219970809145bc9d4e2a3289a9479a P 8f578869bc084a5d8a9a69f143954eb3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:24.151286 20375 leader_election.cc:290] T 28219970809145bc9d4e2a3289a9479a P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 2 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:24.157205 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 97) took 6447 ms. Trace:
I20260925 13:56:24.157514 19955 rpcz_store.cc:276] 0925 13:56:17.710219 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:17.710342 (+ 123us) service_pool.cc:224] Handling call
0925 13:56:23.208231 (+5497889us) ts_tablet_manager.cc:1151] Deleting tablet 1da6eadb1648479889027fa911f4676b
0925 13:56:23.232334 (+ 24103us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:23.675174 (+442840us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:23.892725 (+217551us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:24.157137 (+264412us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":261256,"spinlock_wait_cycles":20224,"tablet-delete.queue_time_us":5497350}
I20260925 13:56:24.050675 20407 raft_consensus.cc:515] T e3cf9676160c4af08f1f62ae952b4c57 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:23.920238 20491 consensus_queue.cc:1048] T 28219970809145bc9d4e2a3289a9479a P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 13:56:23.922184 19631 consensus_peers.cc:597] T f514fa52e7524cd3ba07f3f65de532d4 P 7c1b52895a14482fb54a6758f5054f8f -> Peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Couldn't send request to peer f790493ba7ad403993b678f2df2b55be. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f514fa52e7524cd3ba07f3f65de532d4. This is attempt 1: this message will repeat every 5th retry.
W20260925 13:56:24.165217 19631 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 243147us
I20260925 13:56:24.166939 20491 consensus_queue.cc:1048] T 0ad6442b2de249119ba0ef85dfc06da9 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 13:56:24.168521 19631 leader_election.cc:343] T c161ae3edfed4f8488c770ecc6332607 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: c161ae3edfed4f8488c770ecc6332607
I20260925 13:56:24.161567 19773 leader_election.cc:304] T a26f5a02311c435a8c2434fd030c80a6 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:24.151090 19836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28219970809145bc9d4e2a3289a9479a" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
W20260925 13:56:24.186357 19773 raft_consensus.cc:2686] T a26f5a02311c435a8c2434fd030c80a6 P f790493ba7ad403993b678f2df2b55be: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260925 13:56:24.186779 19836 raft_consensus.cc:3060] T 28219970809145bc9d4e2a3289a9479a P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Advancing to term 2
I20260925 13:56:24.146253 19938 tablet_bootstrap.cc:492] T 0fdb45b31f93466c8bed1b848c6cdf15 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:24.193778 19631 leader_election.cc:304] T c161ae3edfed4f8488c770ecc6332607 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:24.194515 20435 raft_consensus.cc:2749] T c161ae3edfed4f8488c770ecc6332607 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:24.196702 20524 tablet.cc:1751] T 3e6c8b9bbf4549b3a8685f886973ed9c P 7c1b52895a14482fb54a6758f5054f8f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:24.199875 19554 raft_consensus.cc:1240] T 28219970809145bc9d4e2a3289a9479a P 8f578869bc084a5d8a9a69f143954eb3 [term 2 FOLLOWER]: Rejecting Update request from peer 7c1b52895a14482fb54a6758f5054f8f for earlier term 1. Current term is 2. Ops: []
I20260925 13:56:24.200250 19553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3cf9676160c4af08f1f62ae952b4c57" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:24.200956 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 98) took 6304 ms. Trace:
I20260925 13:56:24.201476 19955 rpcz_store.cc:276] 0925 13:56:17.896643 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:17.896773 (+ 130us) service_pool.cc:224] Handling call
0925 13:56:24.159257 (+6262484us) ts_tablet_manager.cc:1151] Deleting tablet c161ae3edfed4f8488c770ecc6332607
0925 13:56:24.159386 (+ 129us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:24.200880 (+ 41494us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6261871}
W20260925 13:56:24.208331 19629 leader_election.cc:343] T e3cf9676160c4af08f1f62ae952b4c57 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:24.211103 20435 consensus_queue.cc:1059] T 28219970809145bc9d4e2a3289a9479a P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:24.212687 20491 raft_consensus.cc:3055] T 28219970809145bc9d4e2a3289a9479a P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Stepping down as leader of term 1
I20260925 13:56:24.213131 20491 raft_consensus.cc:740] T 28219970809145bc9d4e2a3289a9479a P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Running, Role: LEADER
I20260925 13:56:24.214237 20491 consensus_queue.cc:260] T 28219970809145bc9d4e2a3289a9479a P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:24.223627 19955 tablet_replica.cc:333] T c2d333f892f84a258f8a148a90ce3b06 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
W20260925 13:56:24.225569 19274 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 45ab2817d7e84185b7ec2bfddcf4997f consensus state has no leader
I20260925 13:56:24.225620 19938 tablet_bootstrap.cc:654] T 0fdb45b31f93466c8bed1b848c6cdf15 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:24.225083 19955 raft_consensus.cc:2243] T c2d333f892f84a258f8a148a90ce3b06 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:24.231992 19955 raft_consensus.cc:2272] T c2d333f892f84a258f8a148a90ce3b06 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:24.233922 19959 ts_tablet_manager.cc:1929] T 8dc5ea164d6749b0926c63d6bda9dcc5 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:24.234447 19959 log.cc:1199] T 8dc5ea164d6749b0926c63d6bda9dcc5 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/8dc5ea164d6749b0926c63d6bda9dcc5
I20260925 13:56:24.238476 19955 ts_tablet_manager.cc:1916] T c2d333f892f84a258f8a148a90ce3b06 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:24.242452 19836 raft_consensus.cc:2468] T 28219970809145bc9d4e2a3289a9479a P f790493ba7ad403993b678f2df2b55be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 2.
I20260925 13:56:24.247687 19939 tablet_bootstrap.cc:654] T c161ae3edfed4f8488c770ecc6332607 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:24.264191 19672 tablet_service.cc:1558] Processing DeleteTablet for tablet 12984db09f34472ba365fdf63a33b720 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:20 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:24.264706 19669 tablet_service.cc:1558] Processing DeleteTablet for tablet c161ae3edfed4f8488c770ecc6332607 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:17 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:24.177894 20011 raft_consensus.cc:2243] T c1f15b9de572479aae6c2e644da3392a P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:24.265412 20011 raft_consensus.cc:2272] T c1f15b9de572479aae6c2e644da3392a P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:24.260881 19670 tablet_service.cc:1558] Processing DeleteTablet for tablet 436b8d46e1d244deb8bb9dc128ee6279 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:23 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:24.263415 19673 tablet_service.cc:1558] Processing DeleteTablet for tablet 137c43fb74ec4cc48d30ed3edac81d98 with delete_type TABLET_DATA_DELETED (replaced by tablet 540132db164c403da72b9377078c4bf0) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:24.263825 19674 tablet_service.cc:1558] Processing DeleteTablet for tablet a26f5a02311c435a8c2434fd030c80a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:19 UTC) from {username='slave'} at 127.0.0.1:45948
W20260925 13:56:24.231753 19631 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 63276.1us
I20260925 13:56:24.268699 19691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28219970809145bc9d4e2a3289a9479a" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:24.184304 20407 leader_election.cc:290] T e3cf9676160c4af08f1f62ae952b4c57 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:24.269960 19671 tablet_service.cc:1558] Processing DeleteTablet for tablet 8dc5ea164d6749b0926c63d6bda9dcc5 with delete_type TABLET_DATA_DELETED (replaced by tablet d0caac86b259433e9c490aa71ac9d329) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:24.273984 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 5788849b7336496488e5401b4ac91191 (table default.Table_16c91b8d55be449899b5e3a1884d81b4 [id=72cd050d1582435e8a7e8c101df3e117]) successfully deleted
I20260925 13:56:24.280187 19833 raft_consensus.cc:1240] T 28219970809145bc9d4e2a3289a9479a P f790493ba7ad403993b678f2df2b55be [term 2 FOLLOWER]: Rejecting Update request from peer 7c1b52895a14482fb54a6758f5054f8f for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260925 13:56:24.238703 19959 ts_tablet_manager.cc:1950] T 8dc5ea164d6749b0926c63d6bda9dcc5 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:24.287312 19959 tablet_replica.cc:333] T 436b8d46e1d244deb8bb9dc128ee6279 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:24.289381 19959 raft_consensus.cc:2243] T 436b8d46e1d244deb8bb9dc128ee6279 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:24.296608 19959 raft_consensus.cc:2272] T 436b8d46e1d244deb8bb9dc128ee6279 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:24.299878 19938 tablet_bootstrap.cc:492] T 0fdb45b31f93466c8bed1b848c6cdf15 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:24.262338 20547 tablet.cc:1751] T 3e6c8b9bbf4549b3a8685f886973ed9c P f790493ba7ad403993b678f2df2b55be: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:24.300289 19938 ts_tablet_manager.cc:1403] T 0fdb45b31f93466c8bed1b848c6cdf15 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.154s user 0.009s sys 0.000s
I20260925 13:56:24.302593 19938 raft_consensus.cc:359] T 0fdb45b31f93466c8bed1b848c6cdf15 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:24.303262 19938 raft_consensus.cc:385] T 0fdb45b31f93466c8bed1b848c6cdf15 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:24.303570 19938 raft_consensus.cc:740] T 0fdb45b31f93466c8bed1b848c6cdf15 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:24.303884 19274 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_0b05542218754c9c8bbf04c76ba736b5" } modify_external_catalogs: true
I20260925 13:56:24.304373 19274 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_0b05542218754c9c8bbf04c76ba736b5" } modify_external_catalogs: true
I20260925 13:56:24.304119 19938 consensus_queue.cc:260] T 0fdb45b31f93466c8bed1b848c6cdf15 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:24.306747 19273 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_6800d8124538458699f660740653e710"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:24.279629 19631 leader_election.cc:343] T a26f5a02311c435a8c2434fd030c80a6 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Illegal state: Tablet not RUNNING: SHUTDOWN
I20260925 13:56:24.308336 19631 leader_election.cc:304] T a26f5a02311c435a8c2434fd030c80a6 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:24.309289 20011 ts_tablet_manager.cc:1916] T c1f15b9de572479aae6c2e644da3392a P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:24.290674 19942 tablet_bootstrap.cc:654] T f16050752ad44720ba8703ef10bbb359 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:24.310931 19278 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_660b4f2919ba4fd4916ff027d5e01ac9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:24.312935 19488 consensus_peers.cc:597] T c1f15b9de572479aae6c2e644da3392a P 8f578869bc084a5d8a9a69f143954eb3 -> Peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Couldn't send request to peer 7c1b52895a14482fb54a6758f5054f8f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:24.312441 19490 leader_election.cc:304] T 28219970809145bc9d4e2a3289a9479a P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:24.317078 20491 raft_consensus.cc:3060] T 28219970809145bc9d4e2a3289a9479a P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Advancing to term 2
I20260925 13:56:24.310221 20407 raft_consensus.cc:493] T 436b8d46e1d244deb8bb9dc128ee6279 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:24.317895 20407 raft_consensus.cc:515] T 436b8d46e1d244deb8bb9dc128ee6279 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:24.319779 20407 leader_election.cc:290] T 436b8d46e1d244deb8bb9dc128ee6279 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:24.320212 20407 raft_consensus.cc:2749] T a26f5a02311c435a8c2434fd030c80a6 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260925 13:56:24.336074 19667 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45948 (request call id 184) took 1061 ms. Trace:
I20260925 13:56:24.340587 19667 rpcz_store.cc:276] 0925 13:56:23.274484 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:23.443512 (+169028us) service_pool.cc:224] Handling call
0925 13:56:23.444893 (+ 1381us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0925 13:56:23.444970 (+ 77us) ts_tablet_manager.cc:682] Creating new metadata...
0925 13:56:23.822800 (+377830us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:24.335608 (+512808us) tablet_replica.cc:206] Creating consensus instance
0925 13:56:24.336013 (+ 405us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260925 13:56:24.349452 19691 raft_consensus.cc:2468] T 28219970809145bc9d4e2a3289a9479a P 7c1b52895a14482fb54a6758f5054f8f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 2.
W20260925 13:56:24.350823 19631 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 71231us
I20260925 13:56:24.353384 19631 leader_election.cc:304] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters: 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:24.354085 20435 raft_consensus.cc:2804] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:24.354497 20435 raft_consensus.cc:697] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Becoming Leader. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Running, Role: LEADER
I20260925 13:56:24.355245 20435 consensus_queue.cc:237] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:24.360349 19525 tablet_service.cc:1558] Processing DeleteTablet for tablet 436b8d46e1d244deb8bb9dc128ee6279 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:23 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:24.361071 19553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "436b8d46e1d244deb8bb9dc128ee6279" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:24.349494 19959 ts_tablet_manager.cc:1916] T 436b8d46e1d244deb8bb9dc128ee6279 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:24.386720 19833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3cf9676160c4af08f1f62ae952b4c57" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:24.387498 19833 raft_consensus.cc:3060] T e3cf9676160c4af08f1f62ae952b4c57 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:24.388983 19837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "436b8d46e1d244deb8bb9dc128ee6279" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:24.392578 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet 1da6eadb1648479889027fa911f4676b (table default.Table_d3801e388b7b4b4d95188dc60386cce9 [id=479b481f813f4a60b2700734d4f51da3]) successfully deleted
I20260925 13:56:24.345482 19524 tablet_service.cc:1558] Processing DeleteTablet for tablet 8dc5ea164d6749b0926c63d6bda9dcc5 with delete_type TABLET_DATA_DELETED (replaced by tablet d0caac86b259433e9c490aa71ac9d329) from {username='slave'} at 127.0.0.1:40018
W20260925 13:56:24.393285 19264 catalog_manager.cc:5043] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): delete failed for tablet c161ae3edfed4f8488c770ecc6332607 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c161ae3edfed4f8488c770ecc6332607 already in progress: creating tablet
W20260925 13:56:24.403889 19629 leader_election.cc:343] T 436b8d46e1d244deb8bb9dc128ee6279 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:24.405409 19553 raft_consensus.cc:3060] T 0ad6442b2de249119ba0ef85dfc06da9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
W20260925 13:56:24.414398 19273 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6800d8124538458699f660740653e710 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:24.420595 20011 ts_tablet_manager.cc:1929] T c1f15b9de572479aae6c2e644da3392a P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:24.348085 20375 raft_consensus.cc:2804] T 28219970809145bc9d4e2a3289a9479a P 8f578869bc084a5d8a9a69f143954eb3 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 13:56:24.432758 20375 raft_consensus.cc:697] T 28219970809145bc9d4e2a3289a9479a P 8f578869bc084a5d8a9a69f143954eb3 [term 2 LEADER]: Becoming Leader. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Running, Role: LEADER
I20260925 13:56:24.441025 20375 consensus_queue.cc:237] T 28219970809145bc9d4e2a3289a9479a P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:24.451651 19938 ts_tablet_manager.cc:1434] T 0fdb45b31f93466c8bed1b848c6cdf15 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.151s user 0.007s sys 0.000s
I20260925 13:56:24.460500 19938 tablet_bootstrap.cc:492] T f3b46e4fac694084b06b7086bcc596b9 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
W20260925 13:56:24.414021 19278 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_660b4f2919ba4fd4916ff027d5e01ac9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:24.462159 19272 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_261023ab15a845df82deea31f0eac8c8" } modify_external_catalogs: true
I20260925 13:56:24.462639 19272 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_261023ab15a845df82deea31f0eac8c8" } modify_external_catalogs: true
I20260925 13:56:24.466156 19275 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_b8c44d75312d4cf48bedf12f8f6a948f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:24.467936 19275 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b8c44d75312d4cf48bedf12f8f6a948f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:24.464977 19279 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_7e877880587848638cd8f4b2a05e0723"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:24.469762 19279 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7e877880587848638cd8f4b2a05e0723 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:24.472451 19833 raft_consensus.cc:2468] T e3cf9676160c4af08f1f62ae952b4c57 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
I20260925 13:56:24.477020 19955 ts_tablet_manager.cc:1929] T c2d333f892f84a258f8a148a90ce3b06 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:24.483215 19955 log.cc:1199] T c2d333f892f84a258f8a148a90ce3b06 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/c2d333f892f84a258f8a148a90ce3b06
I20260925 13:56:24.483951 19955 ts_tablet_manager.cc:1950] T c2d333f892f84a258f8a148a90ce3b06 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:24.494002 20011 log.cc:1199] T c1f15b9de572479aae6c2e644da3392a P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/c1f15b9de572479aae6c2e644da3392a
I20260925 13:56:24.494755 20011 ts_tablet_manager.cc:1950] T c1f15b9de572479aae6c2e644da3392a P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:24.497730 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 156) took 3101 ms. Trace:
I20260925 13:56:24.501936 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 112) took 6129 ms. Trace:
I20260925 13:56:24.502373 19955 rpcz_store.cc:276] 0925 13:56:18.372245 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:18.395544 (+ 23299us) service_pool.cc:224] Handling call
0925 13:56:24.203093 (+5807549us) ts_tablet_manager.cc:1151] Deleting tablet c2d333f892f84a258f8a148a90ce3b06
0925 13:56:24.223428 (+ 20335us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:24.349843 (+126415us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:24.472234 (+122391us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:24.501878 (+ 29644us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5806941}
W20260925 13:56:24.511893 19266 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.182.252:36181: connect: Connection refused (error 111) [suppressed 5 similar messages]
W20260925 13:56:24.492654 19631 leader_election.cc:343] T 436b8d46e1d244deb8bb9dc128ee6279 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Illegal state: Tablet not RUNNING: STOPPED
I20260925 13:56:24.519526 19955 tablet_replica.cc:333] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:24.520097 19631 leader_election.cc:304] T 436b8d46e1d244deb8bb9dc128ee6279 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:24.519078 20011 rpcz_store.cc:276] 0925 13:56:21.396466 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:21.397515 (+ 1049us) service_pool.cc:224] Handling call
0925 13:56:24.115714 (+2718199us) ts_tablet_manager.cc:1151] Deleting tablet c1f15b9de572479aae6c2e644da3392a
0925 13:56:24.115753 (+ 39us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:24.330468 (+214715us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:24.420486 (+ 90018us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:24.497680 (+ 77194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61185,"tablet-delete.queue_time_us":2717673}
I20260925 13:56:24.525043 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet c1f15b9de572479aae6c2e644da3392a (table default.Table_8b37ba6f398943df8ab95b19f0b7dec6 [id=9cb2dc12a91e4de087414b492cd0dcfb]) successfully deleted
I20260925 13:56:24.525110 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 161) took 2769 ms. Trace:
I20260925 13:56:24.525398 20435 raft_consensus.cc:2749] T 436b8d46e1d244deb8bb9dc128ee6279 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:24.525473 20011 rpcz_store.cc:276] 0925 13:56:21.755339 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:21.755486 (+ 147us) service_pool.cc:224] Handling call
0925 13:56:24.524751 (+2769265us) ts_tablet_manager.cc:1151] Deleting tablet 2711e7a6973546b1822d93f91095dae2
0925 13:56:24.524790 (+ 39us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:24.525057 (+ 267us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2768706}
I20260925 13:56:24.526604 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 162) took 2707 ms. Trace:
I20260925 13:56:24.526880 20011 rpcz_store.cc:276] 0925 13:56:21.818927 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:21.819061 (+ 134us) service_pool.cc:224] Handling call
0925 13:56:24.526274 (+2707213us) ts_tablet_manager.cc:1151] Deleting tablet 78d8c01c88d24deeae21d647e7f205ba
0925 13:56:24.526309 (+ 35us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:24.526548 (+ 239us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2706667}
W20260925 13:56:24.532898 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 2711e7a6973546b1822d93f91095dae2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2711e7a6973546b1822d93f91095dae2 already in progress: creating tablet
I20260925 13:56:24.535702 19955 raft_consensus.cc:2243] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Raft consensus shutting down.
W20260925 13:56:24.527806 19631 consensus_peers.cc:597] T 2ef3efe644364e93a089ddeed328389d P 7c1b52895a14482fb54a6758f5054f8f -> Peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Couldn't send request to peer f790493ba7ad403993b678f2df2b55be. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2ef3efe644364e93a089ddeed328389d. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:24.543495 19631 leader_election.cc:304] T e3cf9676160c4af08f1f62ae952b4c57 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters: 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:24.545074 20407 raft_consensus.cc:2804] T e3cf9676160c4af08f1f62ae952b4c57 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election won for term 1
W20260925 13:56:24.546087 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 78d8c01c88d24deeae21d647e7f205ba because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 78d8c01c88d24deeae21d647e7f205ba already in progress: creating tablet
I20260925 13:56:24.547187 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 163) took 2314 ms. Trace:
I20260925 13:56:24.549230 19959 ts_tablet_manager.cc:1929] T 436b8d46e1d244deb8bb9dc128ee6279 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:24.550247 20011 rpcz_store.cc:276] 0925 13:56:22.232484 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:22.232615 (+ 131us) service_pool.cc:224] Handling call
0925 13:56:24.546859 (+2314244us) ts_tablet_manager.cc:1151] Deleting tablet 4d01a791445149e7bee9493e1eec5c6b
0925 13:56:24.546902 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:24.547137 (+ 235us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2313709}
W20260925 13:56:24.552099 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 4d01a791445149e7bee9493e1eec5c6b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4d01a791445149e7bee9493e1eec5c6b already in progress: creating tablet
I20260925 13:56:24.553099 20011 tablet_replica.cc:333] T 2ef3efe644364e93a089ddeed328389d P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:24.553874 19959 log.cc:1199] T 436b8d46e1d244deb8bb9dc128ee6279 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/436b8d46e1d244deb8bb9dc128ee6279
I20260925 13:56:24.554744 20011 raft_consensus.cc:2243] T 2ef3efe644364e93a089ddeed328389d P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:24.555944 19959 ts_tablet_manager.cc:1950] T 436b8d46e1d244deb8bb9dc128ee6279 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:24.563712 20011 pending_rounds.cc:70] T 2ef3efe644364e93a089ddeed328389d P 7c1b52895a14482fb54a6758f5054f8f: Trying to abort 1 pending ops.
W20260925 13:56:24.565923 19264 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 505a24ebe7014f80b3c184329cc74a9a (table default.Table_f071a5a526034801828291eb010c40c0 [id=2e58540132af4ed298c50ddea3540cdf], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 505a24ebe7014f80b3c184329cc74a9a consensus state has no leader
I20260925 13:56:24.566820 20011 pending_rounds.cc:77] T 2ef3efe644364e93a089ddeed328389d P 7c1b52895a14482fb54a6758f5054f8f: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251414172733440 op_type: NO_OP noop_request { }
I20260925 13:56:24.568559 20011 raft_consensus.cc:2889] T 2ef3efe644364e93a089ddeed328389d P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:56:24.570770 20011 raft_consensus.cc:2272] T 2ef3efe644364e93a089ddeed328389d P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:24.587455 20011 ts_tablet_manager.cc:1916] T 2ef3efe644364e93a089ddeed328389d P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:24.593142 19939 tablet_bootstrap.cc:492] T c161ae3edfed4f8488c770ecc6332607 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:24.593566 19939 ts_tablet_manager.cc:1403] T c161ae3edfed4f8488c770ecc6332607 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.551s user 0.009s sys 0.002s
I20260925 13:56:24.595978 19939 raft_consensus.cc:359] T c161ae3edfed4f8488c770ecc6332607 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:24.596706 19939 raft_consensus.cc:385] T c161ae3edfed4f8488c770ecc6332607 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:24.597152 19939 raft_consensus.cc:740] T c161ae3edfed4f8488c770ecc6332607 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:24.598241 19939 consensus_queue.cc:260] T c161ae3edfed4f8488c770ecc6332607 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
W20260925 13:56:24.599291 19266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.252:36181, user_credentials={real_user=slave}} blocked reactor thread for 87418.2us
I20260925 13:56:24.600147 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 8dc5ea164d6749b0926c63d6bda9dcc5 (table default.Table_e0f7b807e8884431898298a1c635c784 [id=e49e015b173c48c1b6ef8112f60b33e4]) successfully deleted
I20260925 13:56:24.600780 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 436b8d46e1d244deb8bb9dc128ee6279 (table default.Table_079723ff92144ba293733e54ab25d782 [id=4b0bfb9726854123864f8fe743a16a33]) successfully deleted
I20260925 13:56:24.602947 19939 ts_tablet_manager.cc:1434] T c161ae3edfed4f8488c770ecc6332607 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.009s user 0.006s sys 0.001s
I20260925 13:56:24.603961 19939 tablet_bootstrap.cc:492] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:24.617563 20478 raft_consensus.cc:493] T f16050752ad44720ba8703ef10bbb359 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:24.618000 20478 raft_consensus.cc:515] T f16050752ad44720ba8703ef10bbb359 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:24.619867 20478 leader_election.cc:290] T f16050752ad44720ba8703ef10bbb359 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:24.623728 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f16050752ad44720ba8703ef10bbb359" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:24.625407 19773 leader_election.cc:343] T f16050752ad44720ba8703ef10bbb359 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:24.626754 19553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f16050752ad44720ba8703ef10bbb359" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
W20260925 13:56:24.628173 19773 leader_election.cc:343] T f16050752ad44720ba8703ef10bbb359 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:24.628512 19773 leader_election.cc:304] T f16050752ad44720ba8703ef10bbb359 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:24.629451 20478 raft_consensus.cc:2749] T f16050752ad44720ba8703ef10bbb359 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:24.629567 20557 raft_consensus.cc:493] T c161ae3edfed4f8488c770ecc6332607 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:24.630053 20557 raft_consensus.cc:515] T c161ae3edfed4f8488c770ecc6332607 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:24.636314 20557 leader_election.cc:290] T c161ae3edfed4f8488c770ecc6332607 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
W20260925 13:56:24.638185 19490 leader_election.cc:343] T c161ae3edfed4f8488c770ecc6332607 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: c161ae3edfed4f8488c770ecc6332607
I20260925 13:56:24.637054 19939 tablet_bootstrap.cc:654] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:24.643803 20011 ts_tablet_manager.cc:1929] T 2ef3efe644364e93a089ddeed328389d P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:24.647701 19955 raft_consensus.cc:2272] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:24.648169 20011 log.cc:1199] T 2ef3efe644364e93a089ddeed328389d P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/2ef3efe644364e93a089ddeed328389d
I20260925 13:56:24.648988 20011 ts_tablet_manager.cc:1950] T 2ef3efe644364e93a089ddeed328389d P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:24.649976 19955 ts_tablet_manager.cc:1916] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:24.650897 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 165) took 2365 ms. Trace:
W20260925 13:56:24.655539 19488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.194:38641, user_credentials={real_user=slave}} blocked reactor thread for 202039us
I20260925 13:56:24.656091 20011 rpcz_store.cc:276] 0925 13:56:22.285247 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:22.285381 (+ 134us) service_pool.cc:224] Handling call
0925 13:56:24.552944 (+2267563us) ts_tablet_manager.cc:1151] Deleting tablet 2ef3efe644364e93a089ddeed328389d
0925 13:56:24.552985 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:24.615533 (+ 62548us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:24.643687 (+ 28154us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:24.650847 (+ 7160us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2169015}
I20260925 13:56:24.658090 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 2ef3efe644364e93a089ddeed328389d (table default.Table_dfe9d9863d364dfc8df4c405e87dd635 [id=7b4bc216530346b6ab44c1ce25fe2112]) successfully deleted
I20260925 13:56:24.660100 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 168) took 2256 ms. Trace:
I20260925 13:56:24.660449 20011 rpcz_store.cc:276] 0925 13:56:22.403942 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:22.404073 (+ 131us) service_pool.cc:224] Handling call
0925 13:56:24.659860 (+2255787us) ts_tablet_manager.cc:1151] Deleting tablet bbfc4f99a5964e2ba6146e390e8dba44
0925 13:56:24.659902 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:24.660053 (+ 151us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2255275}
W20260925 13:56:24.662092 19264 catalog_manager.cc:5030] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet bbfc4f99a5964e2ba6146e390e8dba44 because the tablet was not found. No further retry: Not found: Tablet not found: bbfc4f99a5964e2ba6146e390e8dba44
I20260925 13:56:24.663295 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 169) took 2210 ms. Trace:
I20260925 13:56:24.663779 20011 rpcz_store.cc:276] 0925 13:56:22.453193 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:22.453321 (+ 128us) service_pool.cc:224] Handling call
0925 13:56:24.663015 (+2209694us) ts_tablet_manager.cc:1151] Deleting tablet c2d333f892f84a258f8a148a90ce3b06
0925 13:56:24.663066 (+ 51us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:24.663234 (+ 168us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2209049}
I20260925 13:56:24.668572 20011 tablet_replica.cc:333] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:24.669458 20011 raft_consensus.cc:2243] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:24.669764 19942 tablet_bootstrap.cc:492] T f16050752ad44720ba8703ef10bbb359 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:24.670120 20011 raft_consensus.cc:2272] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:24.671622 19942 ts_tablet_manager.cc:1403] T f16050752ad44720ba8703ef10bbb359 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.766s user 0.036s sys 0.016s
I20260925 13:56:24.674435 20011 ts_tablet_manager.cc:1916] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:24.674587 19942 raft_consensus.cc:359] T f16050752ad44720ba8703ef10bbb359 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:24.678021 19942 raft_consensus.cc:385] T f16050752ad44720ba8703ef10bbb359 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:24.678572 19942 raft_consensus.cc:740] T f16050752ad44720ba8703ef10bbb359 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:24.685506 19942 consensus_queue.cc:260] T f16050752ad44720ba8703ef10bbb359 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:24.702881 20011 ts_tablet_manager.cc:1929] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:24.703081 19942 ts_tablet_manager.cc:1434] T f16050752ad44720ba8703ef10bbb359 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.031s user 0.004s sys 0.001s
I20260925 13:56:24.704041 19942 tablet_bootstrap.cc:492] T 1215d875209a40d38cc527e2795d6c75 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:24.707206 19693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c161ae3edfed4f8488c770ecc6332607" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:24.707788 19693 raft_consensus.cc:2468] T c161ae3edfed4f8488c770ecc6332607 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:24.711963 20407 raft_consensus.cc:697] T e3cf9676160c4af08f1f62ae952b4c57 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Becoming Leader. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Running, Role: LEADER
I20260925 13:56:24.715059 19942 tablet_bootstrap.cc:654] T 1215d875209a40d38cc527e2795d6c75 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:24.718191 20011 log.cc:1199] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/0de8f2d2766c4ff197b552a1b5ed3d6b
I20260925 13:56:24.718969 20011 ts_tablet_manager.cc:1950] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:24.720846 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 170) took 2266 ms. Trace:
I20260925 13:56:24.721387 20011 rpcz_store.cc:276] 0925 13:56:22.454914 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:22.457221 (+ 2307us) service_pool.cc:224] Handling call
0925 13:56:24.668408 (+2211187us) ts_tablet_manager.cc:1151] Deleting tablet 0de8f2d2766c4ff197b552a1b5ed3d6b
0925 13:56:24.668456 (+ 48us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:24.696741 (+ 28285us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:24.702771 (+ 6030us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:24.720788 (+ 18017us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2210624}
I20260925 13:56:24.722759 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 0de8f2d2766c4ff197b552a1b5ed3d6b (table default.Table_052e4e4ee53e4e408b18cd0ce8063d31 [id=993a34f0c22c4555834fb5cf42da8579]) successfully deleted
I20260925 13:56:24.725652 20011 tablet_replica.cc:333] T 2a1f34dba13143b097a38bab3782c404 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:24.726438 20011 raft_consensus.cc:2243] T 2a1f34dba13143b097a38bab3782c404 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:24.738054 19942 tablet_bootstrap.cc:492] T 1215d875209a40d38cc527e2795d6c75 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:24.738476 19942 ts_tablet_manager.cc:1403] T 1215d875209a40d38cc527e2795d6c75 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.035s user 0.009s sys 0.000s
I20260925 13:56:24.712963 20407 consensus_queue.cc:237] T e3cf9676160c4af08f1f62ae952b4c57 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:24.751565 20011 raft_consensus.cc:2272] T 2a1f34dba13143b097a38bab3782c404 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:24.753371 20011 ts_tablet_manager.cc:1916] T 2a1f34dba13143b097a38bab3782c404 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:24.759897 19942 raft_consensus.cc:359] T 1215d875209a40d38cc527e2795d6c75 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:24.760542 19942 raft_consensus.cc:385] T 1215d875209a40d38cc527e2795d6c75 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:24.760778 19942 raft_consensus.cc:740] T 1215d875209a40d38cc527e2795d6c75 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:24.763660 19942 consensus_queue.cc:260] T 1215d875209a40d38cc527e2795d6c75 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:24.770916 19942 ts_tablet_manager.cc:1434] T 1215d875209a40d38cc527e2795d6c75 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.032s user 0.008s sys 0.000s
I20260925 13:56:24.774507 19942 tablet_bootstrap.cc:492] T 9059655f2c3d4e86b3b389b5b9a68891 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:24.781653 20491 raft_consensus.cc:493] T f16050752ad44720ba8703ef10bbb359 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:24.782105 20491 raft_consensus.cc:515] T f16050752ad44720ba8703ef10bbb359 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:24.783990 20491 leader_election.cc:290] T f16050752ad44720ba8703ef10bbb359 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:24.786391 19553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f16050752ad44720ba8703ef10bbb359" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
W20260925 13:56:24.789855 19629 leader_election.cc:343] T f16050752ad44720ba8703ef10bbb359 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:24.799166 19833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f16050752ad44720ba8703ef10bbb359" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:24.799903 19833 raft_consensus.cc:2468] T f16050752ad44720ba8703ef10bbb359 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:24.803071 20011 ts_tablet_manager.cc:1929] T 2a1f34dba13143b097a38bab3782c404 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:24.803438 20011 log.cc:1199] T 2a1f34dba13143b097a38bab3782c404 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/2a1f34dba13143b097a38bab3782c404
I20260925 13:56:24.804167 20011 ts_tablet_manager.cc:1950] T 2a1f34dba13143b097a38bab3782c404 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
W20260925 13:56:24.806224 19488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.194:38641, user_credentials={real_user=slave}} blocked reactor thread for 58983.2us
I20260925 13:56:24.811888 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 171) took 2354 ms. Trace:
I20260925 13:56:24.812179 20011 rpcz_store.cc:276] 0925 13:56:22.456995 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:22.459702 (+ 2707us) service_pool.cc:224] Handling call
0925 13:56:24.725494 (+2265792us) ts_tablet_manager.cc:1151] Deleting tablet 2a1f34dba13143b097a38bab3782c404
0925 13:56:24.725537 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:24.778470 (+ 52933us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:24.802946 (+ 24476us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:24.811829 (+ 8883us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24264,"tablet-delete.queue_time_us":2265329}
I20260925 13:56:24.814288 20011 tablet_replica.cc:333] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:24.814509 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 2a1f34dba13143b097a38bab3782c404 (table default.Table_3cf69a468c014c279a928211b6ff89a2 [id=fe0f78d1c6e44f589aede852f77f88fc]) successfully deleted
I20260925 13:56:24.816098 19631 leader_election.cc:304] T f16050752ad44720ba8703ef10bbb359 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters: 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:24.818470 20571 raft_consensus.cc:2804] T f16050752ad44720ba8703ef10bbb359 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:24.818846 20571 raft_consensus.cc:493] T f16050752ad44720ba8703ef10bbb359 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:24.816959 20011 raft_consensus.cc:2243] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:24.819923 20571 raft_consensus.cc:3060] T f16050752ad44720ba8703ef10bbb359 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:24.819943 20011 raft_consensus.cc:2272] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:24.824359 20011 ts_tablet_manager.cc:1916] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:24.826263 19942 tablet_bootstrap.cc:654] T 9059655f2c3d4e86b3b389b5b9a68891 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:24.832442 20338 raft_consensus.cc:493] T e6e59d6f4f6e4de1be1cf450620cfa59 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c1b52895a14482fb54a6758f5054f8f)
I20260925 13:56:24.832937 20338 raft_consensus.cc:515] T e6e59d6f4f6e4de1be1cf450620cfa59 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:24.834873 20338 leader_election.cc:290] T e6e59d6f4f6e4de1be1cf450620cfa59 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
W20260925 13:56:24.845587 19490 leader_election.cc:343] T e6e59d6f4f6e4de1be1cf450620cfa59 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: e6e59d6f4f6e4de1be1cf450620cfa59
I20260925 13:56:24.850160 19488 leader_election.cc:304] T c161ae3edfed4f8488c770ecc6332607 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3; no voters: f790493ba7ad403993b678f2df2b55be
I20260925 13:56:24.862198 20011 ts_tablet_manager.cc:1929] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:24.862582 20011 log.cc:1199] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/7ea4fd6817654bff9c0fe8c6e42ed6b9
I20260925 13:56:24.863318 20011 ts_tablet_manager.cc:1950] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:24.865124 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 172) took 2403 ms. Trace:
I20260925 13:56:24.865527 20011 rpcz_store.cc:276] 0925 13:56:22.462018 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:22.462102 (+ 84us) service_pool.cc:224] Handling call
0925 13:56:24.814150 (+2352048us) ts_tablet_manager.cc:1151] Deleting tablet 7ea4fd6817654bff9c0fe8c6e42ed6b9
0925 13:56:24.814188 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:24.847939 (+ 33751us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:24.862084 (+ 14145us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:24.865070 (+ 2986us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2351498}
I20260925 13:56:24.867596 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 173) took 2404 ms. Trace:
I20260925 13:56:24.867879 20011 rpcz_store.cc:276] 0925 13:56:22.463416 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:22.463512 (+ 96us) service_pool.cc:224] Handling call
0925 13:56:24.867343 (+2403831us) ts_tablet_manager.cc:1151] Deleting tablet e6e59d6f4f6e4de1be1cf450620cfa59
0925 13:56:24.867400 (+ 57us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:24.867551 (+ 151us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2403296}
I20260925 13:56:24.868789 20011 tablet_replica.cc:333] T f514fa52e7524cd3ba07f3f65de532d4 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:24.869647 20011 raft_consensus.cc:2243] T f514fa52e7524cd3ba07f3f65de532d4 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:24.870462 20011 pending_rounds.cc:70] T f514fa52e7524cd3ba07f3f65de532d4 P 7c1b52895a14482fb54a6758f5054f8f: Trying to abort 1 pending ops.
I20260925 13:56:24.870671 20011 pending_rounds.cc:77] T f514fa52e7524cd3ba07f3f65de532d4 P 7c1b52895a14482fb54a6758f5054f8f: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251406896893952 op_type: NO_OP noop_request { }
I20260925 13:56:24.870998 20011 raft_consensus.cc:2889] T f514fa52e7524cd3ba07f3f65de532d4 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:56:24.871318 20011 raft_consensus.cc:2272] T f514fa52e7524cd3ba07f3f65de532d4 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:24.875810 20558 raft_consensus.cc:2804] T c161ae3edfed4f8488c770ecc6332607 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:24.860248 20571 raft_consensus.cc:515] T f16050752ad44720ba8703ef10bbb359 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:24.876169 20558 raft_consensus.cc:493] T c161ae3edfed4f8488c770ecc6332607 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:24.876459 20558 raft_consensus.cc:3060] T c161ae3edfed4f8488c770ecc6332607 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:24.877395 20571 leader_election.cc:290] T f16050752ad44720ba8703ef10bbb359 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:24.884537 20558 raft_consensus.cc:515] T c161ae3edfed4f8488c770ecc6332607 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:24.887612 20558 leader_election.cc:290] T c161ae3edfed4f8488c770ecc6332607 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:24.890190 19278 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_af816cb5cc774d51813360f38299d90a" } new_table_name: "default.Table_353d24d106dc42b7b8085d28496b3058" modify_external_catalogs: true
I20260925 13:56:24.899183 19938 tablet_bootstrap.cc:654] T f3b46e4fac694084b06b7086bcc596b9 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:24.905117 20584 tablet.cc:1751] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:24.908591 19273 master_service.cc:960] ReplaceTablet: received request to replace tablet c5a9df833271480aa49feb7d9f925459 from {username='slave'} at 127.0.0.1:39288
W20260925 13:56:24.906818 19490 leader_election.cc:343] T c161ae3edfed4f8488c770ecc6332607 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: c161ae3edfed4f8488c770ecc6332607
I20260925 13:56:24.914431 19280 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 3e6c8b9bbf4549b3a8685f886973ed9c (table default.Table_d0dadd9b31b143fd8dd5adcb7f752c5f [id=beee0ac537e44859b0f7c76d6808b0e0]). Expected version 1 got 0
I20260925 13:56:24.923216 19279 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_1930b257c6824b38ab3c8cba088f9c42" } new_table_name: "default.Table_1b1f36048d2a4a3c86e513e46bcf5471" modify_external_catalogs: true
I20260925 13:56:24.924993 19955 ts_tablet_manager.cc:1929] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:24.927006 19955 log.cc:1199] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/7ea4fd6817654bff9c0fe8c6e42ed6b9
I20260925 13:56:24.930817 19955 ts_tablet_manager.cc:1950] T 7ea4fd6817654bff9c0fe8c6e42ed6b9 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:24.950637 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 7ea4fd6817654bff9c0fe8c6e42ed6b9 (table default.Table_d647554488064f3ea60ad9b397cabe30 [id=b889cefa13cb486f9fb18991d8b6a9ba]) successfully deleted
W20260925 13:56:24.952996 19264 catalog_manager.cc:5030] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet e6e59d6f4f6e4de1be1cf450620cfa59 because the tablet was not found. No further retry: Not found: Tablet not found: e6e59d6f4f6e4de1be1cf450620cfa59
W20260925 13:56:24.959786 19278 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 77a401585d3f4c28ac75b9096c9c378f has no consensus state
I20260925 13:56:24.960114 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c161ae3edfed4f8488c770ecc6332607" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:24.961622 19692 raft_consensus.cc:3060] T c161ae3edfed4f8488c770ecc6332607 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:24.962052 19278 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_5564a650590f413580292ea7e803b417"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:24.963819 19278 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5564a650590f413580292ea7e803b417 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 13:56:24.968747 19488 leader_election.cc:343] T e6e59d6f4f6e4de1be1cf450620cfa59 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Not found: Tablet not found: e6e59d6f4f6e4de1be1cf450620cfa59
I20260925 13:56:24.969226 19488 leader_election.cc:304] T e6e59d6f4f6e4de1be1cf450620cfa59 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3; no voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:24.969903 20455 raft_consensus.cc:2749] T e6e59d6f4f6e4de1be1cf450620cfa59 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:56:24.973008 19938 tablet_bootstrap.cc:492] T f3b46e4fac694084b06b7086bcc596b9 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:24.973532 19942 tablet_bootstrap.cc:492] T 9059655f2c3d4e86b3b389b5b9a68891 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:24.974238 19942 ts_tablet_manager.cc:1403] T 9059655f2c3d4e86b3b389b5b9a68891 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.201s user 0.007s sys 0.004s
I20260925 13:56:24.975096 19938 ts_tablet_manager.cc:1403] T f3b46e4fac694084b06b7086bcc596b9 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.515s user 0.015s sys 0.001s
I20260925 13:56:24.977685 19942 raft_consensus.cc:359] T 9059655f2c3d4e86b3b389b5b9a68891 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:24.978384 19942 raft_consensus.cc:385] T 9059655f2c3d4e86b3b389b5b9a68891 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:24.978633 19942 raft_consensus.cc:740] T 9059655f2c3d4e86b3b389b5b9a68891 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:24.979386 19942 consensus_queue.cc:260] T 9059655f2c3d4e86b3b389b5b9a68891 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:24.981012 19942 ts_tablet_manager.cc:1434] T 9059655f2c3d4e86b3b389b5b9a68891 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260925 13:56:24.984376 19942 tablet_bootstrap.cc:492] T f50ea68e3b3c4459b3f5e5b0df724724 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:24.989471 20011 ts_tablet_manager.cc:1916] T f514fa52e7524cd3ba07f3f65de532d4 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:24.990062 20478 raft_consensus.cc:493] T 908df914cfec490cb7058dbe7b501c97 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:24.990438 20478 raft_consensus.cc:515] T 908df914cfec490cb7058dbe7b501c97 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:24.992646 20478 leader_election.cc:290] T 908df914cfec490cb7058dbe7b501c97 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:24.995422 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "908df914cfec490cb7058dbe7b501c97" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:25.000459 19773 leader_election.cc:343] T 908df914cfec490cb7058dbe7b501c97 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:25.003629 19276 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet f16050752ad44720ba8703ef10bbb359 (table default.Table_1da8b04f68ab4682a8ec32ad47854930 [id=b954461264164e969a40af810627fcb2]). Expected version 1 got 0
I20260925 13:56:25.010070 19524 tablet_service.cc:1511] Processing CreateTablet for tablet f01fc02d12c8461f96ef0a93412ac641 (DEFAULT_TABLE table=default.Table_1e8d4bb8dcbb45a3b882ed664417c23b [id=8668cef08fbb4364830675233ded3e80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:25.011132 19524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f01fc02d12c8461f96ef0a93412ac641. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:25.014288 19553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f16050752ad44720ba8703ef10bbb359" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:25.016188 19819 tablet_service.cc:1511] Processing CreateTablet for tablet f01fc02d12c8461f96ef0a93412ac641 (DEFAULT_TABLE table=default.Table_1e8d4bb8dcbb45a3b882ed664417c23b [id=8668cef08fbb4364830675233ded3e80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:25.017153 19819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f01fc02d12c8461f96ef0a93412ac641. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:25.017645 19676 tablet_service.cc:1511] Processing CreateTablet for tablet f01fc02d12c8461f96ef0a93412ac641 (DEFAULT_TABLE table=default.Table_1e8d4bb8dcbb45a3b882ed664417c23b [id=8668cef08fbb4364830675233ded3e80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:25.018646 19676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f01fc02d12c8461f96ef0a93412ac641. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:25.020064 19525 tablet_service.cc:1511] Processing CreateTablet for tablet fae82e043ab74e12bc4e14e2b5536a92 (DEFAULT_TABLE table=default.Table_83fc37dc7db944d0951289432d899876 [id=b8837669bcd944948a64e36f132e250e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:25.021051 19525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fae82e043ab74e12bc4e14e2b5536a92. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:25.028178 19629 leader_election.cc:343] T f16050752ad44720ba8703ef10bbb359 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:25.039538 19553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "908df914cfec490cb7058dbe7b501c97" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:25.039551 19938 raft_consensus.cc:359] T f3b46e4fac694084b06b7086bcc596b9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:25.045001 19938 raft_consensus.cc:385] T f3b46e4fac694084b06b7086bcc596b9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:25.046927 19938 raft_consensus.cc:740] T f3b46e4fac694084b06b7086bcc596b9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
W20260925 13:56:25.059677 19773 leader_election.cc:343] T 908df914cfec490cb7058dbe7b501c97 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:25.060111 19773 leader_election.cc:304] T 908df914cfec490cb7058dbe7b501c97 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:25.061244 20478 raft_consensus.cc:2749] T 908df914cfec490cb7058dbe7b501c97 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:25.064036 19838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f16050752ad44720ba8703ef10bbb359" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:25.064061 19804 tablet_service.cc:1511] Processing CreateTablet for tablet fae82e043ab74e12bc4e14e2b5536a92 (DEFAULT_TABLE table=default.Table_83fc37dc7db944d0951289432d899876 [id=b8837669bcd944948a64e36f132e250e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:25.064663 19838 raft_consensus.cc:3060] T f16050752ad44720ba8703ef10bbb359 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:25.064530 19668 tablet_service.cc:1511] Processing CreateTablet for tablet fae82e043ab74e12bc4e14e2b5536a92 (DEFAULT_TABLE table=default.Table_83fc37dc7db944d0951289432d899876 [id=b8837669bcd944948a64e36f132e250e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:25.065227 19804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fae82e043ab74e12bc4e14e2b5536a92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:25.065511 19668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fae82e043ab74e12bc4e14e2b5536a92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:25.073287 19939 tablet_bootstrap.cc:492] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:25.075497 19938 consensus_queue.cc:260] T f3b46e4fac694084b06b7086bcc596b9 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:25.077109 19939 ts_tablet_manager.cc:1403] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.473s user 0.013s sys 0.009s
I20260925 13:56:25.093237 19522 tablet_service.cc:1511] Processing CreateTablet for tablet 67ff370d64d64e9da3a44747e44df9c8 (DEFAULT_TABLE table=default.Table_2b6499b3f48241f58bae5603d9f439c8 [id=435b763f49f04d7d915ea5a21ee582fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:25.091774 19938 ts_tablet_manager.cc:1434] T f3b46e4fac694084b06b7086bcc596b9 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.115s user 0.007s sys 0.002s
I20260925 13:56:25.094326 19522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67ff370d64d64e9da3a44747e44df9c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:25.098445 19938 tablet_bootstrap.cc:492] T 953d41f26f5f4027acfd374c6ec4ee75 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:25.101478 19274 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 72183473a5c94629a58c2cd8e1c56f32
I20260925 13:56:25.103283 19812 tablet_service.cc:1558] Processing DeleteTablet for tablet 72183473a5c94629a58c2cd8e1c56f32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:24 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:25.107002 19667 tablet_service.cc:1558] Processing DeleteTablet for tablet 72183473a5c94629a58c2cd8e1c56f32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:24 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:25.108502 19524 tablet_service.cc:1558] Processing DeleteTablet for tablet 72183473a5c94629a58c2cd8e1c56f32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:24 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:25.111635 20588 tablet.cc:1751] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 7c1b52895a14482fb54a6758f5054f8f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:25.117269 20011 ts_tablet_manager.cc:1929] T f514fa52e7524cd3ba07f3f65de532d4 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:25.117710 19942 tablet_bootstrap.cc:654] T f50ea68e3b3c4459b3f5e5b0df724724 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:25.125566 20011 log.cc:1199] T f514fa52e7524cd3ba07f3f65de532d4 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/f514fa52e7524cd3ba07f3f65de532d4
W20260925 13:56:25.132495 19266 catalog_manager.cc:5043] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): delete failed for tablet 72183473a5c94629a58c2cd8e1c56f32 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 72183473a5c94629a58c2cd8e1c56f32 already in progress: creating tablet
I20260925 13:56:25.134467 19667 tablet_service.cc:1511] Processing CreateTablet for tablet 67ff370d64d64e9da3a44747e44df9c8 (DEFAULT_TABLE table=default.Table_2b6499b3f48241f58bae5603d9f439c8 [id=435b763f49f04d7d915ea5a21ee582fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:25.134469 19812 tablet_service.cc:1511] Processing CreateTablet for tablet 67ff370d64d64e9da3a44747e44df9c8 (DEFAULT_TABLE table=default.Table_2b6499b3f48241f58bae5603d9f439c8 [id=435b763f49f04d7d915ea5a21ee582fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:25.135545 19667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67ff370d64d64e9da3a44747e44df9c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:25.135540 19812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67ff370d64d64e9da3a44747e44df9c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:25.140182 20011 ts_tablet_manager.cc:1950] T f514fa52e7524cd3ba07f3f65de532d4 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:25.141315 20478 raft_consensus.cc:493] T 0fdb45b31f93466c8bed1b848c6cdf15 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:25.141744 20478 raft_consensus.cc:515] T 0fdb45b31f93466c8bed1b848c6cdf15 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:25.142310 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 174) took 2677 ms. Trace:
I20260925 13:56:25.142622 20011 rpcz_store.cc:276] 0925 13:56:22.465344 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:22.465446 (+ 102us) service_pool.cc:224] Handling call
0925 13:56:24.868649 (+2403203us) ts_tablet_manager.cc:1151] Deleting tablet f514fa52e7524cd3ba07f3f65de532d4
0925 13:56:24.868689 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:25.038815 (+170126us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:25.117137 (+ 78322us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:25.142250 (+ 25113us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2402795}
I20260925 13:56:25.143499 20478 leader_election.cc:290] T 0fdb45b31f93466c8bed1b848c6cdf15 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:25.148378 19942 tablet_bootstrap.cc:492] T f50ea68e3b3c4459b3f5e5b0df724724 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:25.149089 19942 ts_tablet_manager.cc:1403] T f50ea68e3b3c4459b3f5e5b0df724724 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.165s user 0.004s sys 0.008s
I20260925 13:56:25.151569 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet f514fa52e7524cd3ba07f3f65de532d4 (table default.Table_ba97a54a19744b058cffd5638ffb97db [id=7e7ee975d4bb457eb3cc4632e12f3cd5]) successfully deleted
I20260925 13:56:25.155869 19938 tablet_bootstrap.cc:654] T 953d41f26f5f4027acfd374c6ec4ee75 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:25.164665 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fdb45b31f93466c8bed1b848c6cdf15" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:25.163168 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 175) took 2694 ms. Trace:
W20260925 13:56:25.167657 19773 leader_election.cc:343] T 0fdb45b31f93466c8bed1b848c6cdf15 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:25.166910 20011 rpcz_store.cc:276] 0925 13:56:22.468880 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:22.468975 (+ 95us) service_pool.cc:224] Handling call
0925 13:56:25.155146 (+2686171us) ts_tablet_manager.cc:1151] Deleting tablet 1832cff5eb3e4da4b9766ae1591df81a
0925 13:56:25.157337 (+ 2191us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:25.162935 (+ 5598us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2673102}
W20260925 13:56:25.171474 19264 catalog_manager.cc:5030] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 1832cff5eb3e4da4b9766ae1591df81a because the tablet was not found. No further retry: Not found: Tablet not found: 1832cff5eb3e4da4b9766ae1591df81a
I20260925 13:56:25.175156 20011 tablet_replica.cc:333] T ec7fe5c834be4a51b627b8b1c3ebe888 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:25.172494 19942 raft_consensus.cc:359] T f50ea68e3b3c4459b3f5e5b0df724724 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:25.177899 19942 raft_consensus.cc:385] T f50ea68e3b3c4459b3f5e5b0df724724 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:25.179567 19942 raft_consensus.cc:740] T f50ea68e3b3c4459b3f5e5b0df724724 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:25.181370 20011 raft_consensus.cc:2243] T ec7fe5c834be4a51b627b8b1c3ebe888 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:25.184290 20011 raft_consensus.cc:2272] T ec7fe5c834be4a51b627b8b1c3ebe888 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:25.183470 19942 consensus_queue.cc:260] T f50ea68e3b3c4459b3f5e5b0df724724 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:25.195075 20011 ts_tablet_manager.cc:1916] T ec7fe5c834be4a51b627b8b1c3ebe888 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:24.935876 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 113) took 6562 ms. Trace:
I20260925 13:56:25.195456 19955 rpcz_store.cc:276] 0925 13:56:18.373741 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:18.373919 (+ 178us) service_pool.cc:224] Handling call
0925 13:56:24.518717 (+6144798us) ts_tablet_manager.cc:1151] Deleting tablet 7ea4fd6817654bff9c0fe8c6e42ed6b9
0925 13:56:24.519382 (+ 665us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:24.647208 (+127826us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b58000d1cd0. stack: 00007f1358052323 00007f1358052220 00007f1357bf6574 0000000000516964 00000000005168f9 00007f135f14faa1 00007f135f1425da 00007f135f1c4977 00007f135f1eeb45 00007f135f62f50c 00007f1360298c3a 00007f13602c9960 00007f13602afe6d 00007f13602afe19 00007f13602afda9 00007f13602afd71
0925 13:56:24.658749 (+ 11541us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:24.924301 (+265552us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:24.935667 (+ 11366us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":232628096,"tablet-delete.queue_time_us":6077433}
I20260925 13:56:25.197026 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet 7ea4fd6817654bff9c0fe8c6e42ed6b9 (table default.Table_d647554488064f3ea60ad9b397cabe30 [id=b889cefa13cb486f9fb18991d8b6a9ba]) successfully deleted
I20260925 13:56:25.191213 19939 raft_consensus.cc:359] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:25.201628 19939 raft_consensus.cc:385] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:25.201925 19939 raft_consensus.cc:740] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:25.202467 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fdb45b31f93466c8bed1b848c6cdf15" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:25.203482 19938 tablet_bootstrap.cc:492] T 953d41f26f5f4027acfd374c6ec4ee75 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
W20260925 13:56:25.203641 19773 leader_election.cc:343] T 0fdb45b31f93466c8bed1b848c6cdf15 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:25.203888 19938 ts_tablet_manager.cc:1403] T 953d41f26f5f4027acfd374c6ec4ee75 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.107s user 0.012s sys 0.002s
I20260925 13:56:25.204005 19773 leader_election.cc:304] T 0fdb45b31f93466c8bed1b848c6cdf15 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:25.205338 20478 raft_consensus.cc:2749] T 0fdb45b31f93466c8bed1b848c6cdf15 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:25.206454 19938 raft_consensus.cc:359] T 953d41f26f5f4027acfd374c6ec4ee75 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:25.207202 19938 raft_consensus.cc:385] T 953d41f26f5f4027acfd374c6ec4ee75 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260925 13:56:25.207297 19629 consensus_peers.cc:597] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 7c1b52895a14482fb54a6758f5054f8f -> Peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Couldn't send request to peer 8f578869bc084a5d8a9a69f143954eb3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:25.207525 19938 raft_consensus.cc:740] T 953d41f26f5f4027acfd374c6ec4ee75 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:25.208244 19955 tablet_replica.cc:333] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:25.208268 19938 consensus_queue.cc:260] T 953d41f26f5f4027acfd374c6ec4ee75 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:25.210418 19938 ts_tablet_manager.cc:1434] T 953d41f26f5f4027acfd374c6ec4ee75 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260925 13:56:25.196965 19942 ts_tablet_manager.cc:1434] T f50ea68e3b3c4459b3f5e5b0df724724 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.047s user 0.008s sys 0.001s
I20260925 13:56:25.216370 19938 tablet_bootstrap.cc:492] T 000aad39953c4688b543c7d882a08ec8 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:25.047150 19692 raft_consensus.cc:2468] T c161ae3edfed4f8488c770ecc6332607 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:25.221802 19942 tablet_bootstrap.cc:492] T eb794cef052b43da831531455e2b5f15 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:25.201341 19275 catalog_manager.cc:5838] T 28219970809145bc9d4e2a3289a9479a P 8f578869bc084a5d8a9a69f143954eb3 reported cstate change: term changed from 1 to 2, leader changed from 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194) to 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193). New cstate: current_term: 2 leader_uuid: "8f578869bc084a5d8a9a69f143954eb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:25.241575 19838 raft_consensus.cc:2468] T f16050752ad44720ba8703ef10bbb359 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
I20260925 13:56:25.245409 19938 tablet_bootstrap.cc:654] T 000aad39953c4688b543c7d882a08ec8 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:25.254421 19631 leader_election.cc:304] T f16050752ad44720ba8703ef10bbb359 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters: 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:25.255576 20407 raft_consensus.cc:2804] T f16050752ad44720ba8703ef10bbb359 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:25.256000 20407 raft_consensus.cc:697] T f16050752ad44720ba8703ef10bbb359 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Becoming Leader. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Running, Role: LEADER
I20260925 13:56:25.256747 20407 consensus_queue.cc:237] T f16050752ad44720ba8703ef10bbb359 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:25.257038 19804 tablet_service.cc:1511] Processing CreateTablet for tablet 89bd21e08943403489e63c7c1895393a (DEFAULT_TABLE table=default.Table_750d58e5a30a4b43ba8d84fb7821273e [id=29cbad4ae5c8445a81291433251cd1dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:25.258042 19804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89bd21e08943403489e63c7c1895393a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:25.258643 19676 tablet_service.cc:1511] Processing CreateTablet for tablet 89bd21e08943403489e63c7c1895393a (DEFAULT_TABLE table=default.Table_750d58e5a30a4b43ba8d84fb7821273e [id=29cbad4ae5c8445a81291433251cd1dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:25.259648 19676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89bd21e08943403489e63c7c1895393a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:25.220454 19488 leader_election.cc:304] T c161ae3edfed4f8488c770ecc6332607 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3; no voters: f790493ba7ad403993b678f2df2b55be
W20260925 13:56:25.271531 19488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.194:38641, user_credentials={real_user=slave}} blocked reactor thread for 51170.8us
I20260925 13:56:25.272814 20455 raft_consensus.cc:2804] T c161ae3edfed4f8488c770ecc6332607 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:25.273247 20455 raft_consensus.cc:697] T c161ae3edfed4f8488c770ecc6332607 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Running, Role: LEADER
I20260925 13:56:25.274061 20455 consensus_queue.cc:237] T c161ae3edfed4f8488c770ecc6332607 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:25.277997 19942 tablet_bootstrap.cc:654] T eb794cef052b43da831531455e2b5f15 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:25.254724 19522 tablet_service.cc:1511] Processing CreateTablet for tablet 89bd21e08943403489e63c7c1895393a (DEFAULT_TABLE table=default.Table_750d58e5a30a4b43ba8d84fb7821273e [id=29cbad4ae5c8445a81291433251cd1dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:25.279938 19522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89bd21e08943403489e63c7c1895393a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:25.287048 19938 tablet_bootstrap.cc:492] T 000aad39953c4688b543c7d882a08ec8 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:25.287595 19938 ts_tablet_manager.cc:1403] T 000aad39953c4688b543c7d882a08ec8 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.071s user 0.011s sys 0.003s
I20260925 13:56:25.289834 19938 raft_consensus.cc:359] T 000aad39953c4688b543c7d882a08ec8 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:25.292500 19938 raft_consensus.cc:385] T 000aad39953c4688b543c7d882a08ec8 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:25.299430 19938 raft_consensus.cc:740] T 000aad39953c4688b543c7d882a08ec8 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:25.301847 19938 consensus_queue.cc:260] T 000aad39953c4688b543c7d882a08ec8 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:25.209087 19955 raft_consensus.cc:2243] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:25.306533 20478 raft_consensus.cc:493] T 1215d875209a40d38cc527e2795d6c75 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:25.311247 20478 raft_consensus.cc:515] T 1215d875209a40d38cc527e2795d6c75 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:25.313383 20478 leader_election.cc:290] T 1215d875209a40d38cc527e2795d6c75 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:25.317203 19938 ts_tablet_manager.cc:1434] T 000aad39953c4688b543c7d882a08ec8 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.029s user 0.004s sys 0.001s
I20260925 13:56:25.318200 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215d875209a40d38cc527e2795d6c75" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:25.319823 19695 raft_consensus.cc:2468] T 1215d875209a40d38cc527e2795d6c75 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 0.
I20260925 13:56:25.322163 19773 leader_election.cc:304] T 1215d875209a40d38cc527e2795d6c75 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:25.322995 20478 raft_consensus.cc:2804] T 1215d875209a40d38cc527e2795d6c75 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:25.323329 20478 raft_consensus.cc:493] T 1215d875209a40d38cc527e2795d6c75 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:25.323699 20478 raft_consensus.cc:3060] T 1215d875209a40d38cc527e2795d6c75 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:25.336341 20478 raft_consensus.cc:515] T 1215d875209a40d38cc527e2795d6c75 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:25.344543 20011 ts_tablet_manager.cc:1929] T ec7fe5c834be4a51b627b8b1c3ebe888 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:25.357482 19955 raft_consensus.cc:2272] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:25.345969 20011 log.cc:1199] T ec7fe5c834be4a51b627b8b1c3ebe888 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/ec7fe5c834be4a51b627b8b1c3ebe888
I20260925 13:56:25.362768 20011 ts_tablet_manager.cc:1950] T ec7fe5c834be4a51b627b8b1c3ebe888 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:25.369397 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 176) took 2862 ms. Trace:
I20260925 13:56:25.202725 19939 consensus_queue.cc:260] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:25.371284 20557 raft_consensus.cc:493] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:25.371791 20557 raft_consensus.cc:515] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:25.375628 20011 rpcz_store.cc:276] 0925 13:56:22.507188 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:22.507296 (+ 108us) service_pool.cc:224] Handling call
0925 13:56:25.173993 (+2666697us) ts_tablet_manager.cc:1151] Deleting tablet ec7fe5c834be4a51b627b8b1c3ebe888
0925 13:56:25.174446 (+ 453us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:25.271834 (+ 97388us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:25.342260 (+ 70426us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:25.369339 (+ 27079us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2629194}
I20260925 13:56:25.384903 20557 leader_election.cc:290] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:25.387122 19553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215d875209a40d38cc527e2795d6c75" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:25.393356 19939 ts_tablet_manager.cc:1434] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.314s user 0.000s sys 0.004s
I20260925 13:56:25.394218 19939 tablet_bootstrap.cc:492] T 5788849b7336496488e5401b4ac91191 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:25.399317 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet ec7fe5c834be4a51b627b8b1c3ebe888 (table default.Table_86e7099f24054da589b5a6f7de9ab131 [id=191bb6f9e3b14882aabe499934d8eb57]) successfully deleted
I20260925 13:56:25.400575 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 178) took 2821 ms. Trace:
I20260925 13:56:25.400938 20011 rpcz_store.cc:276] 0925 13:56:22.579381 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:22.579643 (+ 262us) service_pool.cc:224] Handling call
0925 13:56:25.400145 (+2820502us) ts_tablet_manager.cc:1151] Deleting tablet 908df914cfec490cb7058dbe7b501c97
0925 13:56:25.400198 (+ 53us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:25.400521 (+ 323us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2819994}
W20260925 13:56:25.404677 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 908df914cfec490cb7058dbe7b501c97 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 908df914cfec490cb7058dbe7b501c97 already in progress: creating tablet
I20260925 13:56:25.406589 19955 ts_tablet_manager.cc:1916] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:25.417059 19272 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet e2e44b2ca25c4bbcb91288424b3e18b9
I20260925 13:56:25.378873 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "436b8d46e1d244deb8bb9dc128ee6279" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:25.420608 19676 tablet_service.cc:1558] Processing DeleteTablet for tablet e2e44b2ca25c4bbcb91288424b3e18b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:24 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:25.421232 19525 tablet_service.cc:1558] Processing DeleteTablet for tablet e2e44b2ca25c4bbcb91288424b3e18b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:24 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:25.424263 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 188) took 2148 ms. Trace:
I20260925 13:56:25.424592 20011 rpcz_store.cc:276] 0925 13:56:23.275526 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:23.278532 (+ 3006us) service_pool.cc:224] Handling call
0925 13:56:25.405714 (+2127182us) ts_tablet_manager.cc:1151] Deleting tablet 953d41f26f5f4027acfd374c6ec4ee75
0925 13:56:25.405793 (+ 79us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:25.407576 (+ 1783us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2126625}
W20260925 13:56:25.425900 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 953d41f26f5f4027acfd374c6ec4ee75 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 953d41f26f5f4027acfd374c6ec4ee75 already in progress: creating tablet
I20260925 13:56:25.425699 19278 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_d33ac5b10f974ab592b99e99309c0c8c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:25.428366 19272 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_770e2b27b6d4455787f7b0eff2a7221a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:25.429901 19272 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_770e2b27b6d4455787f7b0eff2a7221a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 13:56:25.427259 19278 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d33ac5b10f974ab592b99e99309c0c8c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:25.431768 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 203) took 1166 ms. Trace:
I20260925 13:56:25.433007 20011 rpcz_store.cc:276] 0925 13:56:24.263707 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:24.264092 (+ 385us) service_pool.cc:224] Handling call
0925 13:56:25.430194 (+1166102us) ts_tablet_manager.cc:1151] Deleting tablet 12984db09f34472ba365fdf63a33b720
0925 13:56:25.430238 (+ 44us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:25.430411 (+ 173us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1165650}
W20260925 13:56:25.436086 19264 catalog_manager.cc:5030] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 12984db09f34472ba365fdf63a33b720 because the tablet was not found. No further retry: Not found: Tablet not found: 12984db09f34472ba365fdf63a33b720
I20260925 13:56:25.436313 19695 raft_consensus.cc:2468] T 436b8d46e1d244deb8bb9dc128ee6279 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:25.437440 20011 tablet_replica.cc:333] T c161ae3edfed4f8488c770ecc6332607 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:25.437419 19488 leader_election.cc:304] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3; no voters:
I20260925 13:56:25.439764 20011 raft_consensus.cc:2243] T c161ae3edfed4f8488c770ecc6332607 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:25.444013 20011 raft_consensus.cc:2272] T c161ae3edfed4f8488c770ecc6332607 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:25.453116 19273 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet c5a9df833271480aa49feb7d9f925459
I20260925 13:56:25.454463 19273 catalog_manager.cc:6934] ReplaceTablet: tablet c5a9df833271480aa49feb7d9f925459 deleted and replaced by tablet 236e308d8694405cb2b874164fa16485
I20260925 13:56:25.456809 19525 tablet_service.cc:1558] Processing DeleteTablet for tablet c5a9df833271480aa49feb7d9f925459 with delete_type TABLET_DATA_DELETED (replaced by tablet 236e308d8694405cb2b874164fa16485) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:25.457978 19676 tablet_service.cc:1558] Processing DeleteTablet for tablet c5a9df833271480aa49feb7d9f925459 with delete_type TABLET_DATA_DELETED (replaced by tablet 236e308d8694405cb2b874164fa16485) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:25.464980 20011 ts_tablet_manager.cc:1916] T c161ae3edfed4f8488c770ecc6332607 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:25.487656 20455 raft_consensus.cc:2804] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:25.488051 20455 raft_consensus.cc:493] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:25.488391 20455 raft_consensus.cc:3060] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:25.490644 19955 ts_tablet_manager.cc:1929] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:25.465010 19938 tablet_bootstrap.cc:492] T e2e44b2ca25c4bbcb91288424b3e18b9 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:25.492799 20478 leader_election.cc:290] T 1215d875209a40d38cc527e2795d6c75 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:25.496485 19938 tablet_bootstrap.cc:654] T e2e44b2ca25c4bbcb91288424b3e18b9 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:25.496500 19525 tablet_service.cc:1511] Processing CreateTablet for tablet 61f141c40ce64fbcb68e01ac15009e3c (DEFAULT_TABLE table=default.Table_86568b68e691429192052565e933af79 [id=3a35752b668747c8bc1485701e8e7282]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:25.498225 19676 tablet_service.cc:1511] Processing CreateTablet for tablet 61f141c40ce64fbcb68e01ac15009e3c (DEFAULT_TABLE table=default.Table_86568b68e691429192052565e933af79 [id=3a35752b668747c8bc1485701e8e7282]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:25.498795 19525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61f141c40ce64fbcb68e01ac15009e3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:25.499167 19676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61f141c40ce64fbcb68e01ac15009e3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:25.473928 20597 raft_consensus.cc:493] T f50ea68e3b3c4459b3f5e5b0df724724 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:25.503582 20597 raft_consensus.cc:515] T f50ea68e3b3c4459b3f5e5b0df724724 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:25.506829 20597 leader_election.cc:290] T f50ea68e3b3c4459b3f5e5b0df724724 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:25.511478 19955 log.cc:1199] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/0de8f2d2766c4ff197b552a1b5ed3d6b
I20260925 13:56:25.514585 19955 ts_tablet_manager.cc:1950] T 0de8f2d2766c4ff197b552a1b5ed3d6b P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:25.506912 19831 raft_consensus.cc:1275] T 28219970809145bc9d4e2a3289a9479a P f790493ba7ad403993b678f2df2b55be [term 2 FOLLOWER]: Refusing update from remote peer 8f578869bc084a5d8a9a69f143954eb3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260925 13:56:25.517758 19773 leader_election.cc:343] T 1215d875209a40d38cc527e2795d6c75 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:25.517903 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 137) took 6071 ms. Trace:
I20260925 13:56:25.517010 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215d875209a40d38cc527e2795d6c75" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:25.518478 19692 raft_consensus.cc:3060] T 1215d875209a40d38cc527e2795d6c75 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:25.518404 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50ea68e3b3c4459b3f5e5b0df724724" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:25.518260 19955 rpcz_store.cc:276] 0925 13:56:19.446048 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:19.455989 (+ 9941us) service_pool.cc:224] Handling call
0925 13:56:25.208084 (+5752095us) ts_tablet_manager.cc:1151] Deleting tablet 0de8f2d2766c4ff197b552a1b5ed3d6b
0925 13:56:25.208139 (+ 55us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:25.437016 (+228877us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:25.490540 (+ 53524us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:25.517854 (+ 27314us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46920,"tablet-delete.queue_time_us":5741301}
I20260925 13:56:25.519064 19695 raft_consensus.cc:2468] T f50ea68e3b3c4459b3f5e5b0df724724 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 0.
I20260925 13:56:25.519618 19804 tablet_service.cc:1558] Processing DeleteTablet for tablet e2e44b2ca25c4bbcb91288424b3e18b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:24 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:25.519899 19812 tablet_service.cc:1558] Processing DeleteTablet for tablet c5a9df833271480aa49feb7d9f925459 with delete_type TABLET_DATA_DELETED (replaced by tablet 236e308d8694405cb2b874164fa16485) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:25.521268 19938 tablet_bootstrap.cc:492] T e2e44b2ca25c4bbcb91288424b3e18b9 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:25.521682 19938 ts_tablet_manager.cc:1403] T e2e44b2ca25c4bbcb91288424b3e18b9 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.057s user 0.009s sys 0.002s
I20260925 13:56:25.521504 19812 tablet_service.cc:1511] Processing CreateTablet for tablet 61f141c40ce64fbcb68e01ac15009e3c (DEFAULT_TABLE table=default.Table_86568b68e691429192052565e933af79 [id=3a35752b668747c8bc1485701e8e7282]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:25.522547 19812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61f141c40ce64fbcb68e01ac15009e3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:25.524196 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet 0de8f2d2766c4ff197b552a1b5ed3d6b (table default.Table_052e4e4ee53e4e408b18cd0ce8063d31 [id=993a34f0c22c4555834fb5cf42da8579]) successfully deleted
I20260925 13:56:25.523955 19938 raft_consensus.cc:359] T e2e44b2ca25c4bbcb91288424b3e18b9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:25.524546 19938 raft_consensus.cc:385] T e2e44b2ca25c4bbcb91288424b3e18b9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:25.530411 19955 tablet_replica.cc:333] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:25.531428 19938 raft_consensus.cc:740] T e2e44b2ca25c4bbcb91288424b3e18b9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:25.532084 19938 consensus_queue.cc:260] T e2e44b2ca25c4bbcb91288424b3e18b9 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:25.533638 19955 raft_consensus.cc:2243] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:25.536150 19942 tablet_bootstrap.cc:492] T eb794cef052b43da831531455e2b5f15 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:25.536821 19942 ts_tablet_manager.cc:1403] T eb794cef052b43da831531455e2b5f15 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.316s user 0.013s sys 0.026s
I20260925 13:56:25.536934 19692 raft_consensus.cc:2468] T 1215d875209a40d38cc527e2795d6c75 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 1.
I20260925 13:56:25.540624 19938 ts_tablet_manager.cc:1434] T e2e44b2ca25c4bbcb91288424b3e18b9 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.019s user 0.004s sys 0.000s
W20260925 13:56:25.540642 19266 catalog_manager.cc:5043] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): delete failed for tablet e2e44b2ca25c4bbcb91288424b3e18b9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e2e44b2ca25c4bbcb91288424b3e18b9 already in progress: creating tablet
W20260925 13:56:25.541288 19266 catalog_manager.cc:5043] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): delete failed for tablet c5a9df833271480aa49feb7d9f925459 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c5a9df833271480aa49feb7d9f925459 already in progress: creating tablet
I20260925 13:56:25.546854 19955 pending_rounds.cc:70] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3: Trying to abort 1 pending ops.
I20260925 13:56:25.547104 19955 pending_rounds.cc:77] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7333251406070804480 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "26644768f516412f8c95d68a76bb99c5" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 2 new_table_name: "default.Table_bd52a86b8ff14bdbb06fcf88a60b1e82" dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" new_extra_config { } }
W20260925 13:56:25.502473 19490 leader_election.cc:343] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 436b8d46e1d244deb8bb9dc128ee6279
I20260925 13:56:25.548028 19938 tablet_bootstrap.cc:492] T 1d641f09429546d683f2884c02152f73 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
W20260925 13:56:25.548133 19955 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:40018: Aborted: Op aborted
I20260925 13:56:25.540902 19773 leader_election.cc:304] T f50ea68e3b3c4459b3f5e5b0df724724 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
W20260925 13:56:25.548499 19490 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 46064us
I20260925 13:56:25.539300 19942 raft_consensus.cc:359] T eb794cef052b43da831531455e2b5f15 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:25.550203 19942 raft_consensus.cc:385] T eb794cef052b43da831531455e2b5f15 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:25.550515 19942 raft_consensus.cc:740] T eb794cef052b43da831531455e2b5f15 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:25.551496 19942 consensus_queue.cc:260] T eb794cef052b43da831531455e2b5f15 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:25.553306 19942 ts_tablet_manager.cc:1434] T eb794cef052b43da831531455e2b5f15 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.016s user 0.004s sys 0.000s
I20260925 13:56:25.553351 19938 tablet_bootstrap.cc:654] T 1d641f09429546d683f2884c02152f73 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:25.554224 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:40018 (request call id 157) took 3990 ms. Trace:
I20260925 13:56:25.555519 20597 raft_consensus.cc:2804] T f50ea68e3b3c4459b3f5e5b0df724724 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:25.555909 20597 raft_consensus.cc:493] T f50ea68e3b3c4459b3f5e5b0df724724 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:25.554587 19955 rpcz_store.cc:276] 0925 13:56:21.558557 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:21.558654 (+ 97us) service_pool.cc:224] Handling call
0925 13:56:25.548542 (+3989888us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:21.559717 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0925 13:56:21.560047 (+ 330us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0925 13:56:21.560075 (+ 28us) alter_schema_op.cc:66] Acquired schema lock
0925 13:56:21.560119 (+ 44us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0925 13:56:21.560213 (+ 94us) alter_schema_op.cc:128] START. Timestamp: 1790344581560255
0925 13:56:21.560232 (+ 19us) op_driver.cc:348] REPLICATION: starting
0925 13:56:21.560710 (+ 478us) log.cc:844] Serialized 173 byte log entry
0925 13:56:25.547971 (+3987261us) op_driver.cc:464] REPLICATION: finished
0925 13:56:25.548009 (+ 38us) op_driver.cc:389] HandleFailure(Aborted: Op aborted)
0925 13:56:25.548031 (+ 22us) alter_schema_op.cc:171] AlterSchemaCommitCallback: op aborted
Metrics: {"child_traces":[["op",{"prepare.queue_time_us":281,"prepare.run_cpu_time_us":3844,"prepare.run_wall_time_us":25989,"raft.queue_time_us":172,"raft.run_cpu_time_us":13,"raft.run_wall_time_us":13,"replication_time_us":3987623,"thread_start_us":24601,"threads_started":1,"wal-append.queue_time_us":23644,"wal-append.run_cpu_time_us":1180,"wal-append.run_wall_time_us":1001179}]]}
I20260925 13:56:25.556283 20597 raft_consensus.cc:3060] T f50ea68e3b3c4459b3f5e5b0df724724 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:25.560025 19773 leader_election.cc:304] T 1215d875209a40d38cc527e2795d6c75 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:25.566581 20407 consensus_queue.cc:1048] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:25.572113 19942 tablet_bootstrap.cc:492] T a8a679a453e0430b866edc707fe7c234 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
W20260925 13:56:25.572144 19264 catalog_manager.cc:5142] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): alter failed for tablet 26644768f516412f8c95d68a76bb99c5 (table default.Table_bd52a86b8ff14bdbb06fcf88a60b1e82 [id=203024d67e444f839391c8670d7c7d3b]): Aborted: Op aborted
I20260925 13:56:25.564007 20597 raft_consensus.cc:515] T f50ea68e3b3c4459b3f5e5b0df724724 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:25.575884 20478 raft_consensus.cc:2804] T 1215d875209a40d38cc527e2795d6c75 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:25.584466 19695 raft_consensus.cc:1275] T 28219970809145bc9d4e2a3289a9479a P 7c1b52895a14482fb54a6758f5054f8f [term 2 FOLLOWER]: Refusing update from remote peer 8f578869bc084a5d8a9a69f143954eb3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260925 13:56:25.585666 19939 tablet_bootstrap.cc:654] T 5788849b7336496488e5401b4ac91191 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:25.586018 20375 consensus_queue.cc:1048] T 28219970809145bc9d4e2a3289a9479a P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260925 13:56:25.587306 20597 leader_election.cc:290] T f50ea68e3b3c4459b3f5e5b0df724724 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:25.587706 19955 raft_consensus.cc:2272] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 13:56:25.588359 20456 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:40018: Illegal state: RaftConsensus is not running: State = Stopped
I20260925 13:56:25.588986 20456 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:40018 (request call id 163) took 3535 ms. Trace:
I20260925 13:56:25.589291 20456 rpcz_store.cc:276] 0925 13:56:22.053119 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:22.053241 (+ 122us) service_pool.cc:224] Handling call
0925 13:56:25.588930 (+3535689us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:22.054238 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0925 13:56:22.054563 (+ 325us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0925 13:56:25.587544 (+3532981us) alter_schema_op.cc:66] Acquired schema lock
0925 13:56:25.587615 (+ 71us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0925 13:56:25.587726 (+ 111us) alter_schema_op.cc:128] START. Timestamp: 1790344585587673
0925 13:56:25.587744 (+ 18us) op_driver.cc:348] REPLICATION: starting
0925 13:56:25.588239 (+ 495us) op_driver.cc:389] HandleFailure(Illegal state: RaftConsensus is not running: State = Stopped)
0925 13:56:25.588272 (+ 33us) alter_schema_op.cc:171] AlterSchemaCommitCallback: op aborted
Metrics: {"child_traces":[["op",{"prepare.queue_time_us":164,"spinlock_wait_cycles":803584}]]}
I20260925 13:56:25.593420 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50ea68e3b3c4459b3f5e5b0df724724" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:25.594725 19692 raft_consensus.cc:3060] T f50ea68e3b3c4459b3f5e5b0df724724 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:25.597167 19938 tablet_bootstrap.cc:492] T 1d641f09429546d683f2884c02152f73 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:25.598259 20011 ts_tablet_manager.cc:1929] T c161ae3edfed4f8488c770ecc6332607 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:25.598421 19938 ts_tablet_manager.cc:1403] T 1d641f09429546d683f2884c02152f73 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.051s user 0.009s sys 0.000s
I20260925 13:56:25.601374 19955 ts_tablet_manager.cc:1916] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:25.603276 20011 log.cc:1199] T c161ae3edfed4f8488c770ecc6332607 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/c161ae3edfed4f8488c770ecc6332607
I20260925 13:56:25.603998 20011 ts_tablet_manager.cc:1950] T c161ae3edfed4f8488c770ecc6332607 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:25.605417 20478 raft_consensus.cc:697] T 1215d875209a40d38cc527e2795d6c75 P f790493ba7ad403993b678f2df2b55be [term 1 LEADER]: Becoming Leader. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Running, Role: LEADER
I20260925 13:56:25.606333 20478 consensus_queue.cc:237] T 1215d875209a40d38cc527e2795d6c75 P f790493ba7ad403993b678f2df2b55be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:25.608495 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 204) took 1344 ms. Trace:
I20260925 13:56:25.609721 20011 rpcz_store.cc:276] 0925 13:56:24.263942 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:24.264603 (+ 661us) service_pool.cc:224] Handling call
0925 13:56:25.437286 (+1172683us) ts_tablet_manager.cc:1151] Deleting tablet c161ae3edfed4f8488c770ecc6332607
0925 13:56:25.437333 (+ 47us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:25.534477 (+ 97144us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:25.598134 (+ 63657us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:25.608441 (+ 10307us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1166920}
I20260925 13:56:25.610484 19938 raft_consensus.cc:359] T 1d641f09429546d683f2884c02152f73 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:25.612031 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet c161ae3edfed4f8488c770ecc6332607 (table default.Table_475efa7896f84e658eb81238ceb58d8f [id=5e975bd1dd6e487e97377bc2896e6a3f]) successfully deleted
I20260925 13:56:25.612676 19938 raft_consensus.cc:385] T 1d641f09429546d683f2884c02152f73 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:25.612581 19273 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_3c6b9bb73ad444eca6106c4aa3289fdc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:25.613003 19938 raft_consensus.cc:740] T 1d641f09429546d683f2884c02152f73 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:25.613616 19938 consensus_queue.cc:260] T 1d641f09429546d683f2884c02152f73 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:25.614228 19273 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3c6b9bb73ad444eca6106c4aa3289fdc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:25.615337 19938 ts_tablet_manager.cc:1434] T 1d641f09429546d683f2884c02152f73 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.016s user 0.000s sys 0.003s
I20260925 13:56:25.616191 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 201) took 1353 ms. Trace:
I20260925 13:56:25.616246 20407 raft_consensus.cc:493] T 9059655f2c3d4e86b3b389b5b9a68891 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:25.616670 20407 raft_consensus.cc:515] T 9059655f2c3d4e86b3b389b5b9a68891 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:25.616547 20011 rpcz_store.cc:276] 0925 13:56:24.263154 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:24.263263 (+ 109us) service_pool.cc:224] Handling call
0925 13:56:25.613555 (+1350292us) ts_tablet_manager.cc:1151] Deleting tablet 137c43fb74ec4cc48d30ed3edac81d98
0925 13:56:25.613886 (+ 331us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:25.616140 (+ 2254us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1341209}
W20260925 13:56:25.618403 19264 catalog_manager.cc:5030] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 137c43fb74ec4cc48d30ed3edac81d98 because the tablet was not found. No further retry: Not found: Tablet not found: 137c43fb74ec4cc48d30ed3edac81d98
I20260925 13:56:25.618781 20407 leader_election.cc:290] T 9059655f2c3d4e86b3b389b5b9a68891 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:25.619827 20011 tablet_replica.cc:333] T a26f5a02311c435a8c2434fd030c80a6 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:25.623271 20011 raft_consensus.cc:2243] T a26f5a02311c435a8c2434fd030c80a6 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:25.624680 20011 raft_consensus.cc:2272] T a26f5a02311c435a8c2434fd030c80a6 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 13:56:25.626222 19264 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 77a401585d3f4c28ac75b9096c9c378f (table default.Table_353d24d106dc42b7b8085d28496b3058 [id=d800a775a0e740799f40625659333f7c], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 77a401585d3f4c28ac75b9096c9c378f has no consensus state
I20260925 13:56:25.623415 19838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9059655f2c3d4e86b3b389b5b9a68891" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:25.628047 19838 raft_consensus.cc:2468] T 9059655f2c3d4e86b3b389b5b9a68891 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:25.628705 19938 tablet_bootstrap.cc:492] T 321c740ba8eb4cbb8974305c5f8702af P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:25.629742 19631 leader_election.cc:304] T 9059655f2c3d4e86b3b389b5b9a68891 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:25.630455 20407 raft_consensus.cc:2804] T 9059655f2c3d4e86b3b389b5b9a68891 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:25.630757 20407 raft_consensus.cc:493] T 9059655f2c3d4e86b3b389b5b9a68891 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:25.631067 20407 raft_consensus.cc:3060] T 9059655f2c3d4e86b3b389b5b9a68891 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:25.632071 20011 ts_tablet_manager.cc:1916] T a26f5a02311c435a8c2434fd030c80a6 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:25.648084 19939 tablet_bootstrap.cc:492] T 5788849b7336496488e5401b4ac91191 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:25.648573 19939 ts_tablet_manager.cc:1403] T 5788849b7336496488e5401b4ac91191 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.255s user 0.009s sys 0.003s
I20260925 13:56:25.650904 20478 raft_consensus.cc:493] T 000aad39953c4688b543c7d882a08ec8 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:25.651319 20478 raft_consensus.cc:515] T 000aad39953c4688b543c7d882a08ec8 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:25.652076 19938 tablet_bootstrap.cc:654] T 321c740ba8eb4cbb8974305c5f8702af P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:25.653241 20478 leader_election.cc:290] T 000aad39953c4688b543c7d882a08ec8 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:25.663676 20011 ts_tablet_manager.cc:1929] T a26f5a02311c435a8c2434fd030c80a6 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:25.667018 19938 tablet_bootstrap.cc:492] T 321c740ba8eb4cbb8974305c5f8702af P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:25.667506 19938 ts_tablet_manager.cc:1403] T 321c740ba8eb4cbb8974305c5f8702af P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.039s user 0.012s sys 0.001s
I20260925 13:56:25.669708 20011 log.cc:1199] T a26f5a02311c435a8c2434fd030c80a6 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/a26f5a02311c435a8c2434fd030c80a6
I20260925 13:56:25.670356 20011 ts_tablet_manager.cc:1950] T a26f5a02311c435a8c2434fd030c80a6 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:25.670509 19938 raft_consensus.cc:359] T 321c740ba8eb4cbb8974305c5f8702af P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:25.671589 19938 raft_consensus.cc:385] T 321c740ba8eb4cbb8974305c5f8702af P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:25.671877 19938 raft_consensus.cc:740] T 321c740ba8eb4cbb8974305c5f8702af P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:25.672956 19939 raft_consensus.cc:359] T 5788849b7336496488e5401b4ac91191 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:25.673604 19939 raft_consensus.cc:385] T 5788849b7336496488e5401b4ac91191 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:25.673846 19691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "000aad39953c4688b543c7d882a08ec8" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:25.674314 19939 raft_consensus.cc:740] T 5788849b7336496488e5401b4ac91191 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
W20260925 13:56:25.675202 19773 leader_election.cc:343] T 000aad39953c4688b543c7d882a08ec8 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:25.678856 19939 consensus_queue.cc:260] T 5788849b7336496488e5401b4ac91191 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:25.680616 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 202) took 1417 ms. Trace:
I20260925 13:56:25.680919 20011 rpcz_store.cc:276] 0925 13:56:24.263444 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:24.263723 (+ 279us) service_pool.cc:224] Handling call
0925 13:56:25.619666 (+1355943us) ts_tablet_manager.cc:1151] Deleting tablet a26f5a02311c435a8c2434fd030c80a6
0925 13:56:25.619722 (+ 56us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:25.651166 (+ 31444us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:25.663559 (+ 12393us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:25.680568 (+ 17009us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1347136}
I20260925 13:56:25.673370 19938 consensus_queue.cc:260] T 321c740ba8eb4cbb8974305c5f8702af P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:25.682050 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet a26f5a02311c435a8c2434fd030c80a6 (table default.Table_65608937f2cd4986b2588a4831750601 [id=7079cc0c725b4578a1b11c7dd3ef9342]) successfully deleted
I20260925 13:56:25.686170 19938 ts_tablet_manager.cc:1434] T 321c740ba8eb4cbb8974305c5f8702af P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.018s user 0.005s sys 0.000s
I20260925 13:56:25.686856 20011 tablet_replica.cc:333] T 436b8d46e1d244deb8bb9dc128ee6279 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:25.685688 19274 catalog_manager.cc:5838] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 7c1b52895a14482fb54a6758f5054f8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194). New cstate: current_term: 1 leader_uuid: "7c1b52895a14482fb54a6758f5054f8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:25.688907 19938 tablet_bootstrap.cc:492] T 5f8a96ae6eec4f15a221f9aad537c094 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:25.687584 20011 raft_consensus.cc:2243] T 436b8d46e1d244deb8bb9dc128ee6279 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:25.688051 19274 catalog_manager.cc:5838] T e3cf9676160c4af08f1f62ae952b4c57 P 7c1b52895a14482fb54a6758f5054f8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194). New cstate: current_term: 1 leader_uuid: "7c1b52895a14482fb54a6758f5054f8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:25.689443 20011 raft_consensus.cc:2272] T 436b8d46e1d244deb8bb9dc128ee6279 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:25.689634 19274 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet e3cf9676160c4af08f1f62ae952b4c57 (table default.Table_4d6d747ea4034b3ab48710e10ff4544f [id=d6e6dd0772bf4901b696e11b10e41a48]). Expected version 1 got 0
I20260925 13:56:25.690428 19274 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet f16050752ad44720ba8703ef10bbb359 (table default.Table_1da8b04f68ab4682a8ec32ad47854930 [id=b954461264164e969a40af810627fcb2]). Expected version 1 got 0
I20260925 13:56:25.691138 20011 ts_tablet_manager.cc:1916] T 436b8d46e1d244deb8bb9dc128ee6279 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:25.690587 19939 ts_tablet_manager.cc:1434] T 5788849b7336496488e5401b4ac91191 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.042s user 0.006s sys 0.000s
I20260925 13:56:25.694183 19938 tablet_bootstrap.cc:654] T 5f8a96ae6eec4f15a221f9aad537c094 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
W20260925 13:56:25.695730 19488 consensus_peers.cc:597] T 2a1f34dba13143b097a38bab3782c404 P 8f578869bc084a5d8a9a69f143954eb3 -> Peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Couldn't send request to peer 7c1b52895a14482fb54a6758f5054f8f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2a1f34dba13143b097a38bab3782c404. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:25.700709 19939 tablet_bootstrap.cc:492] T a26f5a02311c435a8c2434fd030c80a6 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
W20260925 13:56:25.702299 19490 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 147021us
I20260925 13:56:25.703511 20375 consensus_queue.cc:1048] T 28219970809145bc9d4e2a3289a9479a P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 13:56:25.709532 19279 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9059655f2c3d4e86b3b389b5b9a68891 consensus state has no leader
I20260925 13:56:25.710162 19279 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:39288 (request call id 262) took 1253 ms. Trace:
I20260925 13:56:25.710511 19279 rpcz_store.cc:276] 0925 13:56:24.457160 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:24.740566 (+283406us) service_pool.cc:224] Handling call
0925 13:56:24.923120 (+182554us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b5400040454. stack: 00007fdf0a30b323 00007fdf0a30b220 00007fdf09eaf574 0000000000516964 00000000005168f9 00007fdf114095c0 00007fdf114a86fa 00007fdf13219d8c 00007fdf13319be6 00007fdf10ecd634 00007fdf10ecd5c2 00007fdf10ecd4f1 00007fdf10ecd46c 00007fdf10ecc722 00007fdf0f16e09c 00007fdf0f16d4d6
0925 13:56:24.923818 (+ 698us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 13:56:24.924443 (+ 625us) catalog_manager.cc:3764] Apply alter schema
0925 13:56:24.924989 (+ 546us) catalog_manager.cc:3796] Acquired catalog manager lock
0925 13:56:24.925188 (+ 199us) catalog_manager.cc:3954] Updating metadata on disk
0925 13:56:25.412000 (+486812us) catalog_manager.cc:3985] Committing alterations to in-memory state
0925 13:56:25.710093 (+298093us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:25.069106 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:25.072727 (+ 3621us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:25.072752 (+ 25us) write_op.cc:435] Acquired schema lock
0925 13:56:25.072764 (+ 12us) tablet.cc:662] Decoding operations
0925 13:56:25.073016 (+ 252us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:25.073060 (+ 44us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:25.073087 (+ 27us) tablet.cc:685] Acquiring locks for 1 operations
0925 13:56:25.077770 (+ 4683us) tablet.cc:701] Row locks acquired
0925 13:56:25.077785 (+ 15us) write_op.cc:260] PREPARE: finished
0925 13:56:25.077895 (+ 110us) write_op.cc:270] Start()
0925 13:56:25.077971 (+ 76us) write_op.cc:276] Timestamp: P: 1790344585077868 usec, L: 0
0925 13:56:25.077985 (+ 14us) op_driver.cc:348] REPLICATION: starting
0925 13:56:25.078457 (+ 472us) log.cc:844] Serialized 405 byte log entry
0925 13:56:25.404871 (+326414us) op_driver.cc:464] REPLICATION: finished
0925 13:56:25.407576 (+ 2705us) write_op.cc:301] APPLY: starting
0925 13:56:25.407636 (+ 60us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:25.407734 (+ 98us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:25.407825 (+ 91us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:25.407979 (+ 154us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:25.408107 (+ 128us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 13:56:25.408151 (+ 44us) write_op.cc:312] APPLY: finished
0925 13:56:25.408312 (+ 161us) log.cc:844] Serialized 23 byte log entry
0925 13:56:25.408473 (+ 161us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:25.408532 (+ 59us) write_op.cc:454] Released schema lock
0925 13:56:25.408564 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0925 13:56:25.408761 (+ 197us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:25.408786 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":417227520,"child_traces":[["op",{"apply.queue_time_us":2561,"apply.run_cpu_time_us":1165,"apply.run_wall_time_us":1166,"num_ops":1,"prepare.queue_time_us":142890,"prepare.run_cpu_time_us":1890,"prepare.run_wall_time_us":54754,"replication_time_us":326800}]]}
I20260925 13:56:25.722138 19280 catalog_manager.cc:6391] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: default.Table_05aaf80be87b4d8489b392a7b7a382ea [id=54685374ee92481fbbf6543c920b782d] alter complete (version 1)
I20260925 13:56:25.723271 19271 catalog_manager.cc:6391] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: default.Table_d65fe24e58a949bca116a094fbb025c5 [id=b673ac6542c5403fbf0fad16c4917dfd] alter complete (version 2)
W20260925 13:56:25.725371 19271 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.195:57799 (request call id 34) took 2397 ms (client timeout 1000 ms). Trace:
I20260925 13:56:25.727667 19955 ts_tablet_manager.cc:1929] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:56:25.728157 19955 log.cc:1199] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/26644768f516412f8c95d68a76bb99c5
I20260925 13:56:25.732318 19524 tablet_service.cc:1558] Processing DeleteTablet for tablet 0de8f2d2766c4ff197b552a1b5ed3d6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:18 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:25.733160 19525 tablet_service.cc:1558] Processing DeleteTablet for tablet c2d333f892f84a258f8a148a90ce3b06 with delete_type TABLET_DATA_DELETED (replaced by tablet 908df914cfec490cb7058dbe7b501c97) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:25.733779 19519 tablet_service.cc:1558] Processing DeleteTablet for tablet 7ea4fd6817654bff9c0fe8c6e42ed6b9 with delete_type TABLET_DATA_DELETED (replaced by tablet 321c740ba8eb4cbb8974305c5f8702af) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:25.734381 19529 tablet_service.cc:1558] Processing DeleteTablet for tablet 1da6eadb1648479889027fa911f4676b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:17 UTC) from {username='slave'} at 127.0.0.1:40018
W20260925 13:56:25.735414 19276 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.195:57799 (request call id 35) took 1390 ms (client timeout 1000 ms). Trace:
I20260925 13:56:25.735584 19939 tablet_bootstrap.cc:654] T a26f5a02311c435a8c2434fd030c80a6 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
W20260925 13:56:25.735720 19276 rpcz_store.cc:269] 0925 13:56:24.344609 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:24.344742 (+ 133us) service_pool.cc:224] Handling call
0925 13:56:25.735327 (+1390585us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":6}
W20260925 13:56:25.736282 19277 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.193:37907 (request call id 32) took 2342 ms (client timeout 1000 ms). Trace:
W20260925 13:56:25.736562 19277 rpcz_store.cc:269] 0925 13:56:23.393455 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:23.393552 (+ 97us) service_pool.cc:224] Handling call
0925 13:56:25.736225 (+2342673us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":25}
I20260925 13:56:25.737489 19804 tablet_service.cc:1558] Processing DeleteTablet for tablet 908df914cfec490cb7058dbe7b501c97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:22 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:25.738286 20589 tablet_replica.cc:333] T 908df914cfec490cb7058dbe7b501c97 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:25.739374 20589 raft_consensus.cc:2243] T 908df914cfec490cb7058dbe7b501c97 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:25.739982 20589 raft_consensus.cc:2272] T 908df914cfec490cb7058dbe7b501c97 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:25.740615 20011 ts_tablet_manager.cc:1929] T 436b8d46e1d244deb8bb9dc128ee6279 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:25.740945 20011 log.cc:1199] T 436b8d46e1d244deb8bb9dc128ee6279 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/436b8d46e1d244deb8bb9dc128ee6279
I20260925 13:56:25.741570 20011 ts_tablet_manager.cc:1950] T 436b8d46e1d244deb8bb9dc128ee6279 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:25.741922 20589 ts_tablet_manager.cc:1916] T 908df914cfec490cb7058dbe7b501c97 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:25.743428 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 200) took 1482 ms. Trace:
I20260925 13:56:25.743927 19955 ts_tablet_manager.cc:1950] T 26644768f516412f8c95d68a76bb99c5 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:25.743712 20011 rpcz_store.cc:276] 0925 13:56:24.260660 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:24.260759 (+ 99us) service_pool.cc:224] Handling call
0925 13:56:25.686694 (+1425935us) ts_tablet_manager.cc:1151] Deleting tablet 436b8d46e1d244deb8bb9dc128ee6279
0925 13:56:25.686743 (+ 49us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:25.704699 (+ 17956us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:25.723169 (+ 18470us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:25.743338 (+ 20169us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1414010}
I20260925 13:56:25.744910 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 436b8d46e1d244deb8bb9dc128ee6279 (table default.Table_079723ff92144ba293733e54ab25d782 [id=4b0bfb9726854123864f8fe743a16a33]) successfully deleted
I20260925 13:56:25.749060 20478 raft_consensus.cc:493] T 321c740ba8eb4cbb8974305c5f8702af P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:25.749493 20478 raft_consensus.cc:515] T 321c740ba8eb4cbb8974305c5f8702af P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:25.749907 20011 tablet_replica.cc:333] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:25.750582 20011 raft_consensus.cc:2243] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:25.751041 20011 raft_consensus.cc:2272] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:25.751238 20478 leader_election.cc:290] T 321c740ba8eb4cbb8974305c5f8702af P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:25.753037 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "321c740ba8eb4cbb8974305c5f8702af" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:25.755695 19938 tablet_bootstrap.cc:492] T 5f8a96ae6eec4f15a221f9aad537c094 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
W20260925 13:56:25.755991 19773 leader_election.cc:343] T 321c740ba8eb4cbb8974305c5f8702af P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:25.756222 20011 ts_tablet_manager.cc:1916] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:25.756413 19529 tablet_service.cc:1558] Processing DeleteTablet for tablet f514fa52e7524cd3ba07f3f65de532d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:16 UTC) from {username='slave'} at 127.0.0.1:40018
W20260925 13:56:25.725700 19271 rpcz_store.cc:269] 0925 13:56:23.327707 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:23.327826 (+ 119us) service_pool.cc:224] Handling call
0925 13:56:25.725306 (+2397480us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:23.447007 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:23.447340 (+ 333us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:23.447384 (+ 44us) write_op.cc:435] Acquired schema lock
0925 13:56:23.447399 (+ 15us) tablet.cc:662] Decoding operations
0925 13:56:23.447622 (+ 223us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:23.447666 (+ 44us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:23.447687 (+ 21us) tablet.cc:685] Acquiring locks for 1 operations
0925 13:56:23.447820 (+ 133us) tablet.cc:701] Row locks acquired
0925 13:56:23.447833 (+ 13us) write_op.cc:260] PREPARE: finished
0925 13:56:23.447931 (+ 98us) write_op.cc:270] Start()
0925 13:56:23.448005 (+ 74us) write_op.cc:276] Timestamp: P: 1790344583447983 usec, L: 0
0925 13:56:23.448019 (+ 14us) op_driver.cc:348] REPLICATION: starting
0925 13:56:23.448432 (+ 413us) log.cc:844] Serialized 356 byte log entry
0925 13:56:23.843205 (+394773us) op_driver.cc:464] REPLICATION: finished
0925 13:56:23.845227 (+ 2022us) write_op.cc:301] APPLY: starting
0925 13:56:23.845274 (+ 47us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:23.845359 (+ 85us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:23.845456 (+ 97us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:23.845610 (+ 154us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:23.845734 (+ 124us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 13:56:23.845775 (+ 41us) write_op.cc:312] APPLY: finished
0925 13:56:23.845925 (+ 150us) log.cc:844] Serialized 23 byte log entry
0925 13:56:23.846054 (+ 129us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:23.846105 (+ 51us) write_op.cc:454] Released schema lock
0925 13:56:23.846133 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0925 13:56:23.846352 (+ 219us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:23.846382 (+ 30us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":1914,"apply.run_cpu_time_us":1088,"apply.run_wall_time_us":1087,"num_ops":1,"prepare.queue_time_us":110,"prepare.run_cpu_time_us":1610,"prepare.run_wall_time_us":1610,"replication_time_us":395117}]]}
I20260925 13:56:25.756415 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 139) took 6309 ms. Trace:
I20260925 13:56:25.758715 19955 rpcz_store.cc:276] 0925 13:56:19.446710 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:19.456559 (+ 9849us) service_pool.cc:224] Handling call
0925 13:56:25.530255 (+6073696us) ts_tablet_manager.cc:1151] Deleting tablet 26644768f516412f8c95d68a76bb99c5
0925 13:56:25.530305 (+ 50us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:25.645023 (+114718us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:25.727508 (+ 82485us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:25.756355 (+ 28847us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":725,"tablet-delete.queue_time_us":6073273}
I20260925 13:56:25.760380 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet 26644768f516412f8c95d68a76bb99c5 (table default.Table_bd52a86b8ff14bdbb06fcf88a60b1e82 [id=203024d67e444f839391c8670d7c7d3b]) successfully deleted
I20260925 13:56:25.762310 19955 tablet_replica.cc:333] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:25.763432 19955 raft_consensus.cc:2243] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:25.764401 19955 raft_consensus.cc:2272] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:25.768602 19939 tablet_bootstrap.cc:492] T a26f5a02311c435a8c2434fd030c80a6 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:25.769079 19939 ts_tablet_manager.cc:1403] T a26f5a02311c435a8c2434fd030c80a6 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.069s user 0.005s sys 0.013s
I20260925 13:56:25.770836 20478 raft_consensus.cc:493] T 953d41f26f5f4027acfd374c6ec4ee75 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:25.771486 19939 raft_consensus.cc:359] T a26f5a02311c435a8c2434fd030c80a6 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:25.771422 20478 raft_consensus.cc:515] T 953d41f26f5f4027acfd374c6ec4ee75 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:25.772197 19939 raft_consensus.cc:385] T a26f5a02311c435a8c2434fd030c80a6 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:25.772477 19939 raft_consensus.cc:740] T a26f5a02311c435a8c2434fd030c80a6 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:25.773183 19939 consensus_queue.cc:260] T a26f5a02311c435a8c2434fd030c80a6 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:25.773667 20478 leader_election.cc:290] T 953d41f26f5f4027acfd374c6ec4ee75 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:25.776779 19939 ts_tablet_manager.cc:1434] T a26f5a02311c435a8c2434fd030c80a6 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20260925 13:56:25.777025 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "953d41f26f5f4027acfd374c6ec4ee75" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:25.777684 19939 tablet_bootstrap.cc:492] T e3cf9676160c4af08f1f62ae952b4c57 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
W20260925 13:56:25.779172 19773 leader_election.cc:343] T 953d41f26f5f4027acfd374c6ec4ee75 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:25.782315 20478 raft_consensus.cc:493] T 1d641f09429546d683f2884c02152f73 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:25.783820 20338 raft_consensus.cc:493] T 12984db09f34472ba365fdf63a33b720 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:25.784224 20338 raft_consensus.cc:515] T 12984db09f34472ba365fdf63a33b720 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:25.784428 19553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1215d875209a40d38cc527e2795d6c75" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:25.785311 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "000aad39953c4688b543c7d882a08ec8" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:25.785723 19550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "321c740ba8eb4cbb8974305c5f8702af" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:25.785440 19551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50ea68e3b3c4459b3f5e5b0df724724" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:25.785059 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f50ea68e3b3c4459b3f5e5b0df724724" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:25.786125 20338 leader_election.cc:290] T 12984db09f34472ba365fdf63a33b720 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:25.786235 19549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "953d41f26f5f4027acfd374c6ec4ee75" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
W20260925 13:56:25.790324 19488 leader_election.cc:343] T 12984db09f34472ba365fdf63a33b720 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Not found: Tablet not found: 12984db09f34472ba365fdf63a33b720
I20260925 13:56:25.782955 20478 raft_consensus.cc:515] T 1d641f09429546d683f2884c02152f73 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:25.791626 19692 raft_consensus.cc:2468] T f50ea68e3b3c4459b3f5e5b0df724724 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 1.
I20260925 13:56:25.792537 20478 leader_election.cc:290] T 1d641f09429546d683f2884c02152f73 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:25.793874 19773 leader_election.cc:304] T f50ea68e3b3c4459b3f5e5b0df724724 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:25.794191 20455 raft_consensus.cc:515] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:25.794773 20478 raft_consensus.cc:2804] T f50ea68e3b3c4459b3f5e5b0df724724 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:25.795261 20478 raft_consensus.cc:697] T f50ea68e3b3c4459b3f5e5b0df724724 P f790493ba7ad403993b678f2df2b55be [term 1 LEADER]: Becoming Leader. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Running, Role: LEADER
I20260925 13:56:25.796077 20478 consensus_queue.cc:237] T f50ea68e3b3c4459b3f5e5b0df724724 P f790493ba7ad403993b678f2df2b55be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:25.806918 19942 tablet_bootstrap.cc:654] T a8a679a453e0430b866edc707fe7c234 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:25.756100 19938 ts_tablet_manager.cc:1403] T 5f8a96ae6eec4f15a221f9aad537c094 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.069s user 0.016s sys 0.036s
I20260925 13:56:25.807976 19276 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_92f5e6ebf342495b8333c8da779d6998" } new_table_name: "default.Table_be48ddb268274ca685a6b7995ec4f0ee" modify_external_catalogs: true
I20260925 13:56:25.787487 19547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9059655f2c3d4e86b3b389b5b9a68891" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:25.787330 19515 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:40014 (request call id 29) took 1326 ms. Trace:
I20260925 13:56:25.812479 19271 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_db5fac75a1364fbda83732d71eabb1e2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:25.812911 19277 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_b42a6e2de4e74cb7a6f01dba22f3b94c" } modify_external_catalogs: true
I20260925 13:56:25.812762 19515 rpcz_store.cc:276] 0925 13:56:24.461152 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:24.461245 (+ 93us) service_pool.cc:224] Handling call
0925 13:56:24.919673 (+458428us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b54000839d4. stack: 00007f1358052323 00007f1358052220 00007f1357bf6574 0000000000516964 00000000005168f9 00007f135f14faa1 00007f135f1e06b8 00007f136025198d 00007f135e9a1104 00007f135e9a1092 00007f135e9a0fc1 00007f135e9a0f3c 00007f135e9a01f2 00007f135ceb509c 00007f135ceb44d6 00007f135ceb3e7f
0925 13:56:25.787147 (+867474us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38568192}
I20260925 13:56:25.813530 19277 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_b42a6e2de4e74cb7a6f01dba22f3b94c" } modify_external_catalogs: true
I20260925 13:56:25.814287 19939 tablet_bootstrap.cc:654] T e3cf9676160c4af08f1f62ae952b4c57 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
W20260925 13:56:25.814251 19271 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_db5fac75a1364fbda83732d71eabb1e2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:25.819625 20604 raft_consensus.cc:493] T f3b46e4fac694084b06b7086bcc596b9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:25.820061 20604 raft_consensus.cc:515] T f3b46e4fac694084b06b7086bcc596b9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:25.821987 20604 leader_election.cc:290] T f3b46e4fac694084b06b7086bcc596b9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:25.828780 19938 raft_consensus.cc:359] T 5f8a96ae6eec4f15a221f9aad537c094 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:25.829547 19938 raft_consensus.cc:385] T 5f8a96ae6eec4f15a221f9aad537c094 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:25.829836 19938 raft_consensus.cc:740] T 5f8a96ae6eec4f15a221f9aad537c094 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:25.833798 19939 tablet_bootstrap.cc:492] T e3cf9676160c4af08f1f62ae952b4c57 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:25.834244 19939 ts_tablet_manager.cc:1403] T e3cf9676160c4af08f1f62ae952b4c57 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.057s user 0.009s sys 0.000s
I20260925 13:56:25.835500 20604 raft_consensus.cc:493] T a8a679a453e0430b866edc707fe7c234 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:25.835760 19955 ts_tablet_manager.cc:1916] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:25.835927 20604 raft_consensus.cc:515] T a8a679a453e0430b866edc707fe7c234 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:25.837002 19939 raft_consensus.cc:359] T e3cf9676160c4af08f1f62ae952b4c57 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:25.837579 19939 raft_consensus.cc:385] T e3cf9676160c4af08f1f62ae952b4c57 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:25.837764 20604 leader_election.cc:290] T a8a679a453e0430b866edc707fe7c234 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:25.837807 19939 raft_consensus.cc:740] T e3cf9676160c4af08f1f62ae952b4c57 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:25.838372 19939 consensus_queue.cc:260] T e3cf9676160c4af08f1f62ae952b4c57 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:25.839735 19939 ts_tablet_manager.cc:1434] T e3cf9676160c4af08f1f62ae952b4c57 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260925 13:56:25.840215 20375 raft_consensus.cc:493] T a26f5a02311c435a8c2434fd030c80a6 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:25.840950 19939 tablet_bootstrap.cc:492] T f16050752ad44720ba8703ef10bbb359 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:25.840684 20375 raft_consensus.cc:515] T a26f5a02311c435a8c2434fd030c80a6 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:25.842984 20375 leader_election.cc:290] T a26f5a02311c435a8c2434fd030c80a6 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
W20260925 13:56:25.845379 19488 leader_election.cc:343] T a26f5a02311c435a8c2434fd030c80a6 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Not found: Tablet not found: a26f5a02311c435a8c2434fd030c80a6
I20260925 13:56:25.846848 19939 tablet_bootstrap.cc:654] T f16050752ad44720ba8703ef10bbb359 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:25.855197 19938 consensus_queue.cc:260] T 5f8a96ae6eec4f15a221f9aad537c094 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:25.857333 19939 tablet_bootstrap.cc:492] T f16050752ad44720ba8703ef10bbb359 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:25.857762 19939 ts_tablet_manager.cc:1403] T f16050752ad44720ba8703ef10bbb359 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.017s user 0.016s sys 0.000s
I20260925 13:56:25.860026 19939 raft_consensus.cc:359] T f16050752ad44720ba8703ef10bbb359 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:25.860812 19939 raft_consensus.cc:385] T f16050752ad44720ba8703ef10bbb359 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:25.861126 19939 raft_consensus.cc:740] T f16050752ad44720ba8703ef10bbb359 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:25.863260 19938 ts_tablet_manager.cc:1434] T 5f8a96ae6eec4f15a221f9aad537c094 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.055s user 0.005s sys 0.000s
I20260925 13:56:25.861838 19939 consensus_queue.cc:260] T f16050752ad44720ba8703ef10bbb359 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:25.868037 19939 ts_tablet_manager.cc:1434] T f16050752ad44720ba8703ef10bbb359 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20260925 13:56:25.871001 19938 tablet_bootstrap.cc:492] T 82972fa4868d4c59b3b9bb1240300938 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:25.882447 19939 tablet_bootstrap.cc:492] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:25.887250 20589 ts_tablet_manager.cc:1929] T 908df914cfec490cb7058dbe7b501c97 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:25.887740 20589 log.cc:1199] T 908df914cfec490cb7058dbe7b501c97 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/908df914cfec490cb7058dbe7b501c97
I20260925 13:56:25.888501 20589 ts_tablet_manager.cc:1950] T 908df914cfec490cb7058dbe7b501c97 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:25.891322 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 908df914cfec490cb7058dbe7b501c97 (table default.Table_c90b2c8024944acb9ffe76b506fff5a2 [id=d06e00bf26bd4c47a9e27f7eb35835df]) successfully deleted
I20260925 13:56:25.894065 19938 tablet_bootstrap.cc:654] T 82972fa4868d4c59b3b9bb1240300938 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:25.896085 19939 tablet_bootstrap.cc:654] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:25.912217 19955 ts_tablet_manager.cc:1929] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:56:25.912622 19955 log.cc:1199] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/1513c0048bd244028d3dfa8d118bc14e
I20260925 13:56:25.913338 19955 ts_tablet_manager.cc:1950] T 1513c0048bd244028d3dfa8d118bc14e P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:25.914270 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 140) took 6467 ms. Trace:
I20260925 13:56:25.914565 19955 rpcz_store.cc:276] 0925 13:56:19.447057 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:19.457006 (+ 9949us) service_pool.cc:224] Handling call
0925 13:56:25.762099 (+6305093us) ts_tablet_manager.cc:1151] Deleting tablet 1513c0048bd244028d3dfa8d118bc14e
0925 13:56:25.762179 (+ 80us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:25.867335 (+105156us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:25.902353 (+ 35018us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:25.914218 (+ 11865us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6304557}
I20260925 13:56:25.915771 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet 1513c0048bd244028d3dfa8d118bc14e (table default.Table_74b6076b22674a31bc58fef71cc9bfb1 [id=d81cf07e5f9a49d4bbf5aceda3da5a25]) successfully deleted
I20260925 13:56:25.937266 20478 raft_consensus.cc:493] T 5f8a96ae6eec4f15a221f9aad537c094 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:25.937516 20455 leader_election.cc:290] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:25.937916 20478 raft_consensus.cc:515] T 5f8a96ae6eec4f15a221f9aad537c094 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:25.938776 19488 leader_election.cc:343] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Not found: Tablet not found: 436b8d46e1d244deb8bb9dc128ee6279
I20260925 13:56:25.940433 20478 leader_election.cc:290] T 5f8a96ae6eec4f15a221f9aad537c094 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:25.946791 19955 tablet_replica.cc:333] T 2a1f34dba13143b097a38bab3782c404 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:25.947867 19955 raft_consensus.cc:2243] T 2a1f34dba13143b097a38bab3782c404 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:25.949059 19955 raft_consensus.cc:2272] T 2a1f34dba13143b097a38bab3782c404 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:25.960107 19272 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:39288 (request call id 267) took 1010 ms. Trace:
I20260925 13:56:25.960464 19272 rpcz_store.cc:276] 0925 13:56:24.949737 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:25.418968 (+469231us) service_pool.cc:224] Handling call
0925 13:56:25.428262 (+ 9294us) spinlock_profiling.cc:243] Waited 8.49 ms on lock 0x7b5400040454. stack: 00007fdf0a30b323 00007fdf0a30b220 00007fdf09eaf574 0000000000516964 00000000005168f9 00007fdf114095c0 00007fdf114a86fa 00007fdf13219d8c 00007fdf13319566 00007fdf10ec4034 00007fdf10ec3fc2 00007fdf10ec3ef1 00007fdf10ec3e6c 00007fdf10ec3122 00007fdf0f16e09c 00007fdf0f16d4d6
0925 13:56:25.430241 (+ 1979us) catalog_manager.cc:2140] Acquired catalog manager lock
0925 13:56:25.435183 (+ 4942us) catalog_manager.cc:2186] Created new table and tablet info
0925 13:56:25.959510 (+524327us) catalog_manager.cc:2246] Wrote table and tablets to system table
0925 13:56:25.959797 (+ 287us) catalog_manager.cc:2280] Inserted table and tablets into CatalogManager maps
0925 13:56:25.960048 (+ 251us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:25.823140 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:25.823545 (+ 405us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:25.823569 (+ 24us) write_op.cc:435] Acquired schema lock
0925 13:56:25.823582 (+ 13us) tablet.cc:662] Decoding operations
0925 13:56:25.823949 (+ 367us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:25.823984 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:25.824005 (+ 21us) tablet.cc:685] Acquiring locks for 2 operations
0925 13:56:25.824186 (+ 181us) tablet.cc:701] Row locks acquired
0925 13:56:25.824199 (+ 13us) write_op.cc:260] PREPARE: finished
0925 13:56:25.824304 (+ 105us) write_op.cc:270] Start()
0925 13:56:25.824384 (+ 80us) write_op.cc:276] Timestamp: P: 1790344585824267 usec, L: 0
0925 13:56:25.824399 (+ 15us) op_driver.cc:348] REPLICATION: starting
0925 13:56:25.938371 (+113972us) log.cc:844] Serialized 433 byte log entry
0925 13:56:25.956907 (+ 18536us) op_driver.cc:464] REPLICATION: finished
0925 13:56:25.957365 (+ 458us) write_op.cc:301] APPLY: starting
0925 13:56:25.957434 (+ 69us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:25.957569 (+ 135us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:25.957694 (+ 125us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:25.958369 (+ 675us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:25.958554 (+ 185us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 13:56:25.958606 (+ 52us) write_op.cc:312] APPLY: finished
0925 13:56:25.958809 (+ 203us) log.cc:844] Serialized 29 byte log entry
0925 13:56:25.959154 (+ 345us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:25.959247 (+ 93us) write_op.cc:454] Released schema lock
0925 13:56:25.959292 (+ 45us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":19527936,"child_traces":[["op",{"apply.queue_time_us":265,"apply.run_cpu_time_us":2167,"apply.run_wall_time_us":2167,"num_ops":2,"prepare.queue_time_us":386875,"replication_time_us":132400}]]}
I20260925 13:56:25.965142 19272 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_e814702f1f874529a98bd1631d3b5c84" } modify_external_catalogs: true
I20260925 13:56:25.966049 19272 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_e814702f1f874529a98bd1631d3b5c84" } modify_external_catalogs: true
I20260925 13:56:25.951268 19955 ts_tablet_manager.cc:1916] T 2a1f34dba13143b097a38bab3782c404 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:25.643039 20407 raft_consensus.cc:515] T 9059655f2c3d4e86b3b389b5b9a68891 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:25.825613 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b46e4fac694084b06b7086bcc596b9" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:25.839605 19691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8a679a453e0430b866edc707fe7c234" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:25.857067 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d641f09429546d683f2884c02152f73" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:25.901790 20571 consensus_queue.cc:1048] T e3cf9676160c4af08f1f62ae952b4c57 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 13:56:25.942422 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f8a96ae6eec4f15a221f9aad537c094" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:25.987320 19773 leader_election.cc:343] T f3b46e4fac694084b06b7086bcc596b9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:25.987968 19773 leader_election.cc:343] T a8a679a453e0430b866edc707fe7c234 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:25.988453 19773 leader_election.cc:343] T 1d641f09429546d683f2884c02152f73 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:25.988520 20407 leader_election.cc:290] T 9059655f2c3d4e86b3b389b5b9a68891 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
W20260925 13:56:25.989379 19773 leader_election.cc:343] T 5f8a96ae6eec4f15a221f9aad537c094 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:25.992941 19488 consensus_peers.cc:597] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3 -> Peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Couldn't send request to peer 7c1b52895a14482fb54a6758f5054f8f. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:25.993284 20375 raft_consensus.cc:493] T 137c43fb74ec4cc48d30ed3edac81d98 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c1b52895a14482fb54a6758f5054f8f)
I20260925 13:56:25.993808 20375 raft_consensus.cc:515] T 137c43fb74ec4cc48d30ed3edac81d98 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:25.996171 20375 leader_election.cc:290] T 137c43fb74ec4cc48d30ed3edac81d98 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
W20260925 13:56:26.000335 19490 leader_election.cc:343] T 12984db09f34472ba365fdf63a33b720 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 12984db09f34472ba365fdf63a33b720
I20260925 13:56:26.000766 19490 leader_election.cc:304] T 12984db09f34472ba365fdf63a33b720 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3; no voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be
W20260925 13:56:26.001564 19488 leader_election.cc:343] T 137c43fb74ec4cc48d30ed3edac81d98 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Not found: Tablet not found: 137c43fb74ec4cc48d30ed3edac81d98
I20260925 13:56:26.002022 20375 raft_consensus.cc:2749] T 12984db09f34472ba365fdf63a33b720 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:26.008028 19938 tablet_bootstrap.cc:492] T 82972fa4868d4c59b3b9bb1240300938 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:26.012430 19938 ts_tablet_manager.cc:1403] T 82972fa4868d4c59b3b9bb1240300938 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.148s user 0.018s sys 0.012s
W20260925 13:56:26.017391 19773 leader_election.cc:343] T 000aad39953c4688b543c7d882a08ec8 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:26.017750 19773 leader_election.cc:304] T 000aad39953c4688b543c7d882a08ec8 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
W20260925 13:56:26.018602 19773 leader_election.cc:343] T 1215d875209a40d38cc527e2795d6c75 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:26.019280 19773 leader_election.cc:343] T 321c740ba8eb4cbb8974305c5f8702af P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:26.019606 19773 leader_election.cc:304] T 321c740ba8eb4cbb8974305c5f8702af P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:26.020332 20604 raft_consensus.cc:2749] T 000aad39953c4688b543c7d882a08ec8 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:26.020828 20604 raft_consensus.cc:2749] T 321c740ba8eb4cbb8974305c5f8702af P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:56:26.021812 19773 leader_election.cc:343] T f50ea68e3b3c4459b3f5e5b0df724724 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:26.022611 19773 leader_election.cc:343] T f50ea68e3b3c4459b3f5e5b0df724724 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:26.023447 19773 leader_election.cc:343] T 953d41f26f5f4027acfd374c6ec4ee75 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:26.023813 19773 leader_election.cc:304] T 953d41f26f5f4027acfd374c6ec4ee75 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:26.024451 20478 raft_consensus.cc:2749] T 953d41f26f5f4027acfd374c6ec4ee75 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:26.028352 19938 raft_consensus.cc:359] T 82972fa4868d4c59b3b9bb1240300938 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:26.031040 19938 raft_consensus.cc:385] T 82972fa4868d4c59b3b9bb1240300938 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:26.031431 19938 raft_consensus.cc:740] T 82972fa4868d4c59b3b9bb1240300938 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:26.031900 19838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9059655f2c3d4e86b3b389b5b9a68891" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:26.032393 19838 raft_consensus.cc:3060] T 9059655f2c3d4e86b3b389b5b9a68891 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:26.032457 19942 tablet_bootstrap.cc:492] T a8a679a453e0430b866edc707fe7c234 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:26.032310 19938 consensus_queue.cc:260] T 82972fa4868d4c59b3b9bb1240300938 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:26.033012 19942 ts_tablet_manager.cc:1403] T a8a679a453e0430b866edc707fe7c234 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.461s user 0.009s sys 0.002s
I20260925 13:56:26.034391 19938 ts_tablet_manager.cc:1434] T 82972fa4868d4c59b3b9bb1240300938 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.021s user 0.003s sys 0.001s
I20260925 13:56:26.035912 19938 tablet_bootstrap.cc:492] T 089dc4dfeb5243788dc03061c8743a81 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:26.037271 19939 tablet_bootstrap.cc:492] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:26.037726 19939 ts_tablet_manager.cc:1403] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.155s user 0.027s sys 0.031s
I20260925 13:56:26.041002 19838 raft_consensus.cc:2468] T 9059655f2c3d4e86b3b389b5b9a68891 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
I20260925 13:56:26.042773 19938 tablet_bootstrap.cc:654] T 089dc4dfeb5243788dc03061c8743a81 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:26.046702 20011 ts_tablet_manager.cc:1929] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:56:26.047088 20011 log.cc:1199] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/8dc5ea164d6749b0926c63d6bda9dcc5
I20260925 13:56:26.048225 20011 ts_tablet_manager.cc:1950] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:26.050477 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 205) took 1786 ms. Trace:
I20260925 13:56:26.050793 20011 rpcz_store.cc:276] 0925 13:56:24.264208 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:24.269782 (+ 5574us) service_pool.cc:224] Handling call
0925 13:56:25.749748 (+1479966us) ts_tablet_manager.cc:1151] Deleting tablet 8dc5ea164d6749b0926c63d6bda9dcc5
0925 13:56:25.749797 (+ 49us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:25.768501 (+ 18704us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:26.046582 (+278081us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:26.050421 (+ 3839us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1439728}
I20260925 13:56:26.052398 19631 leader_election.cc:304] T 9059655f2c3d4e86b3b389b5b9a68891 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:26.053386 20571 raft_consensus.cc:2804] T 9059655f2c3d4e86b3b389b5b9a68891 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:26.053881 20571 raft_consensus.cc:697] T 9059655f2c3d4e86b3b389b5b9a68891 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Becoming Leader. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Running, Role: LEADER
W20260925 13:56:26.054008 19629 leader_election.cc:343] T 9059655f2c3d4e86b3b389b5b9a68891 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:26.054685 20571 consensus_queue.cc:237] T 9059655f2c3d4e86b3b389b5b9a68891 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:26.063934 19547 raft_consensus.cc:3060] T e3cf9676160c4af08f1f62ae952b4c57 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:26.067745 19939 raft_consensus.cc:359] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:26.066202 19942 raft_consensus.cc:359] T a8a679a453e0430b866edc707fe7c234 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:26.069736 19942 raft_consensus.cc:385] T a8a679a453e0430b866edc707fe7c234 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:26.071285 19942 raft_consensus.cc:740] T a8a679a453e0430b866edc707fe7c234 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:26.068857 19549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9059655f2c3d4e86b3b389b5b9a68891" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:26.079841 19939 raft_consensus.cc:385] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:26.069950 19551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d641f09429546d683f2884c02152f73" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:26.080116 19939 raft_consensus.cc:740] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:26.070568 19550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b46e4fac694084b06b7086bcc596b9" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:26.071923 19553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8a679a453e0430b866edc707fe7c234" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:26.072927 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f8a96ae6eec4f15a221f9aad537c094" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:26.081014 19942 consensus_queue.cc:260] T a8a679a453e0430b866edc707fe7c234 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:26.083105 19942 ts_tablet_manager.cc:1434] T a8a679a453e0430b866edc707fe7c234 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.050s user 0.004s sys 0.000s
W20260925 13:56:26.087881 19490 leader_election.cc:343] T a26f5a02311c435a8c2434fd030c80a6 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: a26f5a02311c435a8c2434fd030c80a6
I20260925 13:56:26.089260 19490 leader_election.cc:304] T a26f5a02311c435a8c2434fd030c80a6 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3; no voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:26.090310 20455 raft_consensus.cc:2749] T a26f5a02311c435a8c2434fd030c80a6 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:26.088541 19939 consensus_queue.cc:260] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:26.090781 20604 raft_consensus.cc:493] T e2e44b2ca25c4bbcb91288424b3e18b9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:26.091207 20604 raft_consensus.cc:515] T e2e44b2ca25c4bbcb91288424b3e18b9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:26.093075 20604 leader_election.cc:290] T e2e44b2ca25c4bbcb91288424b3e18b9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:26.094641 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2e44b2ca25c4bbcb91288424b3e18b9" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:26.100687 19773 leader_election.cc:343] T e2e44b2ca25c4bbcb91288424b3e18b9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:26.101833 19942 tablet_bootstrap.cc:492] T 3aaf29a2310a41d0971eded7ef51fdc9 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:26.108419 20604 raft_consensus.cc:493] T 82972fa4868d4c59b3b9bb1240300938 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:26.108903 20604 raft_consensus.cc:515] T 82972fa4868d4c59b3b9bb1240300938 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:26.110688 20604 leader_election.cc:290] T 82972fa4868d4c59b3b9bb1240300938 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:26.111482 19939 ts_tablet_manager.cc:1434] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.073s user 0.007s sys 0.000s
I20260925 13:56:26.114037 19938 tablet_bootstrap.cc:492] T 089dc4dfeb5243788dc03061c8743a81 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:26.114722 19939 tablet_bootstrap.cc:492] T ec7fe5c834be4a51b627b8b1c3ebe888 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:26.116432 19938 ts_tablet_manager.cc:1403] T 089dc4dfeb5243788dc03061c8743a81 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.081s user 0.006s sys 0.005s
I20260925 13:56:26.117412 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82972fa4868d4c59b3b9bb1240300938" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:26.127243 19955 ts_tablet_manager.cc:1929] T 2a1f34dba13143b097a38bab3782c404 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260925 13:56:26.133229 19773 leader_election.cc:343] T 82972fa4868d4c59b3b9bb1240300938 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:26.134907 19938 raft_consensus.cc:359] T 089dc4dfeb5243788dc03061c8743a81 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:26.139997 19938 raft_consensus.cc:385] T 089dc4dfeb5243788dc03061c8743a81 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:26.140303 19938 raft_consensus.cc:740] T 089dc4dfeb5243788dc03061c8743a81 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:26.140992 19955 log.cc:1199] T 2a1f34dba13143b097a38bab3782c404 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/2a1f34dba13143b097a38bab3782c404
I20260925 13:56:26.141438 19938 consensus_queue.cc:260] T 089dc4dfeb5243788dc03061c8743a81 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:26.142838 19955 ts_tablet_manager.cc:1950] T 2a1f34dba13143b097a38bab3782c404 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:26.145828 19938 ts_tablet_manager.cc:1434] T 089dc4dfeb5243788dc03061c8743a81 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.026s user 0.005s sys 0.000s
I20260925 13:56:26.146670 19938 tablet_bootstrap.cc:492] T 31049bccd170436893512a7e9ef435cc P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:26.147066 19939 tablet_bootstrap.cc:654] T ec7fe5c834be4a51b627b8b1c3ebe888 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:26.152194 19942 tablet_bootstrap.cc:654] T 3aaf29a2310a41d0971eded7ef51fdc9 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:26.159435 19938 tablet_bootstrap.cc:654] T 31049bccd170436893512a7e9ef435cc P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:26.161242 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 8dc5ea164d6749b0926c63d6bda9dcc5 (table default.Table_e0f7b807e8884431898298a1c635c784 [id=e49e015b173c48c1b6ef8112f60b33e4]) successfully deleted
W20260925 13:56:26.162269 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 72183473a5c94629a58c2cd8e1c56f32 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 72183473a5c94629a58c2cd8e1c56f32 already in progress: creating tablet
W20260925 13:56:26.163044 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet e2e44b2ca25c4bbcb91288424b3e18b9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e2e44b2ca25c4bbcb91288424b3e18b9 already in progress: creating tablet
W20260925 13:56:26.163968 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet c5a9df833271480aa49feb7d9f925459 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c5a9df833271480aa49feb7d9f925459 already in progress: creating tablet
W20260925 13:56:26.103174 19490 leader_election.cc:343] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 436b8d46e1d244deb8bb9dc128ee6279
I20260925 13:56:26.171300 19490 leader_election.cc:304] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3; no voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:26.162379 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 135) took 6732 ms. Trace:
I20260925 13:56:26.172034 19955 rpcz_store.cc:276] 0925 13:56:19.429909 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:19.451730 (+ 21821us) service_pool.cc:224] Handling call
0925 13:56:25.946553 (+6494823us) ts_tablet_manager.cc:1151] Deleting tablet 2a1f34dba13143b097a38bab3782c404
0925 13:56:25.946643 (+ 90us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:26.005140 (+ 58497us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:26.127123 (+121983us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:26.162317 (+ 35194us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6488271}
I20260925 13:56:26.173116 19955 tablet_replica.cc:333] T a26f5a02311c435a8c2434fd030c80a6 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:26.173954 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet 2a1f34dba13143b097a38bab3782c404 (table default.Table_3cf69a468c014c279a928211b6ff89a2 [id=fe0f78d1c6e44f589aede852f77f88fc]) successfully deleted
I20260925 13:56:26.173476 20455 raft_consensus.cc:2749] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260925 13:56:26.175560 19955 raft_consensus.cc:2243] T a26f5a02311c435a8c2434fd030c80a6 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260925 13:56:26.174026 19490 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 70888.5us
W20260925 13:56:26.177512 19490 leader_election.cc:343] T 137c43fb74ec4cc48d30ed3edac81d98 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 137c43fb74ec4cc48d30ed3edac81d98
I20260925 13:56:26.178925 19490 leader_election.cc:304] T 137c43fb74ec4cc48d30ed3edac81d98 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3; no voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:26.180536 19938 tablet_bootstrap.cc:492] T 31049bccd170436893512a7e9ef435cc P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:26.180950 19938 ts_tablet_manager.cc:1403] T 31049bccd170436893512a7e9ef435cc P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.034s user 0.010s sys 0.001s
I20260925 13:56:26.183310 19273 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:39288 (request call id 268) took 1069 ms. Trace:
I20260925 13:56:26.183413 19938 raft_consensus.cc:359] T 31049bccd170436893512a7e9ef435cc P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:26.184087 19938 raft_consensus.cc:385] T 31049bccd170436893512a7e9ef435cc P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:26.184335 19938 raft_consensus.cc:740] T 31049bccd170436893512a7e9ef435cc P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:26.185039 19938 consensus_queue.cc:260] T 31049bccd170436893512a7e9ef435cc P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:26.183928 19273 rpcz_store.cc:276] 0925 13:56:25.113299 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:25.455094 (+341795us) service_pool.cc:224] Handling call
0925 13:56:25.612490 (+157396us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b5400040454. stack: 00007fdf0a30b323 00007fdf0a30b220 00007fdf09eaf574 0000000000516964 00000000005168f9 00007fdf114095c0 00007fdf114a86fa 00007fdf13219d8c 00007fdf13319566 00007fdf10ec4034 00007fdf10ec3fc2 00007fdf10ec3ef1 00007fdf10ec3e6c 00007fdf10ec3122 00007fdf0f16e09c 00007fdf0f16d4d6
0925 13:56:25.623956 (+ 11466us) catalog_manager.cc:2140] Acquired catalog manager lock
0925 13:56:25.626086 (+ 2130us) catalog_manager.cc:2186] Created new table and tablet info
0925 13:56:26.181714 (+555628us) catalog_manager.cc:2246] Wrote table and tablets to system table
0925 13:56:26.182312 (+ 598us) catalog_manager.cc:2280] Inserted table and tablets into CatalogManager maps
0925 13:56:26.182978 (+ 666us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:25.971329 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:25.971807 (+ 478us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:25.971829 (+ 22us) write_op.cc:435] Acquired schema lock
0925 13:56:25.971842 (+ 13us) tablet.cc:662] Decoding operations
0925 13:56:25.972106 (+ 264us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:25.972156 (+ 50us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:25.972178 (+ 22us) tablet.cc:685] Acquiring locks for 2 operations
0925 13:56:25.972466 (+ 288us) tablet.cc:701] Row locks acquired
0925 13:56:25.972483 (+ 17us) write_op.cc:260] PREPARE: finished
0925 13:56:25.972657 (+ 174us) write_op.cc:270] Start()
0925 13:56:25.972742 (+ 85us) write_op.cc:276] Timestamp: P: 1790344585972560 usec, L: 0
0925 13:56:25.972857 (+ 115us) op_driver.cc:348] REPLICATION: starting
0925 13:56:25.973327 (+ 470us) log.cc:844] Serialized 433 byte log entry
0925 13:56:26.041853 (+ 68526us) op_driver.cc:464] REPLICATION: finished
0925 13:56:26.042378 (+ 525us) write_op.cc:301] APPLY: starting
0925 13:56:26.042438 (+ 60us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:26.042555 (+ 117us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:26.042665 (+ 110us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:26.043091 (+ 426us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:26.043219 (+ 128us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 13:56:26.043264 (+ 45us) write_op.cc:312] APPLY: finished
0925 13:56:26.043472 (+ 208us) log.cc:844] Serialized 29 byte log entry
0925 13:56:26.043609 (+ 137us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:26.043681 (+ 72us) write_op.cc:454] Released schema lock
0925 13:56:26.043725 (+ 44us) write_op.cc:341] FINISH: Updating metrics
0925 13:56:26.043943 (+ 218us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:26.043967 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":360228736,"child_traces":[["op",{"apply.queue_time_us":365,"apply.run_cpu_time_us":1534,"apply.run_wall_time_us":1535,"num_ops":2,"prepare.queue_time_us":335319,"prepare.run_cpu_time_us":2030,"prepare.run_wall_time_us":3232,"replication_time_us":68892}]]}
I20260925 13:56:26.179375 19955 raft_consensus.cc:2272] T a26f5a02311c435a8c2434fd030c80a6 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:26.187278 19939 tablet_bootstrap.cc:492] T ec7fe5c834be4a51b627b8b1c3ebe888 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:26.196130 19939 ts_tablet_manager.cc:1403] T ec7fe5c834be4a51b627b8b1c3ebe888 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.082s user 0.011s sys 0.002s
I20260925 13:56:26.198210 19938 ts_tablet_manager.cc:1434] T 31049bccd170436893512a7e9ef435cc P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.017s user 0.004s sys 0.000s
I20260925 13:56:26.203461 20455 raft_consensus.cc:2749] T 137c43fb74ec4cc48d30ed3edac81d98 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:56:26.205629 19939 raft_consensus.cc:359] T ec7fe5c834be4a51b627b8b1c3ebe888 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:26.206360 19939 raft_consensus.cc:385] T ec7fe5c834be4a51b627b8b1c3ebe888 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:26.206663 19939 raft_consensus.cc:740] T ec7fe5c834be4a51b627b8b1c3ebe888 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:26.207429 19939 consensus_queue.cc:260] T ec7fe5c834be4a51b627b8b1c3ebe888 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:26.214129 19939 ts_tablet_manager.cc:1434] T ec7fe5c834be4a51b627b8b1c3ebe888 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.018s user 0.006s sys 0.000s
I20260925 13:56:26.215065 20455 raft_consensus.cc:493] T f16050752ad44720ba8703ef10bbb359 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260925 13:56:26.217530 19629 leader_election.cc:343] T 9059655f2c3d4e86b3b389b5b9a68891 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:26.215528 20455 raft_consensus.cc:515] T f16050752ad44720ba8703ef10bbb359 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:26.219897 19938 tablet_bootstrap.cc:492] T eff33a56361946edb556ac611d2299eb P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:26.224190 20571 consensus_queue.cc:1048] T e3cf9676160c4af08f1f62ae952b4c57 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 13:56:26.225092 19938 tablet_bootstrap.cc:654] T eff33a56361946edb556ac611d2299eb P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:26.234115 19939 tablet_bootstrap.cc:492] T 3aaf29a2310a41d0971eded7ef51fdc9 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:26.236014 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f16050752ad44720ba8703ef10bbb359" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:26.236696 19955 ts_tablet_manager.cc:1916] T a26f5a02311c435a8c2434fd030c80a6 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:26.250945 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2e44b2ca25c4bbcb91288424b3e18b9" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:26.242304 20455 leader_election.cc:290] T f16050752ad44720ba8703ef10bbb359 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:26.251952 19553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82972fa4868d4c59b3b9bb1240300938" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:26.253981 19838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f16050752ad44720ba8703ef10bbb359" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:26.254665 19838 raft_consensus.cc:2393] T f16050752ad44720ba8703ef10bbb359 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f578869bc084a5d8a9a69f143954eb3 in current term 1: Already voted for candidate 7c1b52895a14482fb54a6758f5054f8f in this term.
W20260925 13:56:26.255208 19276 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a0109e4115174d8eb8174a1a2236150f consensus state has no leader
I20260925 13:56:26.255782 19276 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:39288 (request call id 269) took 1092 ms. Trace:
I20260925 13:56:26.257809 19955 ts_tablet_manager.cc:1929] T a26f5a02311c435a8c2434fd030c80a6 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:26.258131 19955 log.cc:1199] T a26f5a02311c435a8c2434fd030c80a6 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/a26f5a02311c435a8c2434fd030c80a6
I20260925 13:56:26.258693 19955 ts_tablet_manager.cc:1950] T a26f5a02311c435a8c2434fd030c80a6 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:26.266170 20571 raft_consensus.cc:493] T eb794cef052b43da831531455e2b5f15 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:26.266637 20571 raft_consensus.cc:515] T eb794cef052b43da831531455e2b5f15 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:26.278482 19271 master_service.cc:960] ReplaceTablet: received request to replace tablet 65ad0197267d47158751bc44c539b555 from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:26.281080 20571 leader_election.cc:290] T eb794cef052b43da831531455e2b5f15 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
W20260925 13:56:26.282512 19773 leader_election.cc:343] T 1d641f09429546d683f2884c02152f73 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:26.282959 19773 leader_election.cc:304] T 1d641f09429546d683f2884c02152f73 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:26.283835 20604 raft_consensus.cc:2749] T 1d641f09429546d683f2884c02152f73 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:56:26.284149 19773 leader_election.cc:343] T f3b46e4fac694084b06b7086bcc596b9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:26.284580 19773 leader_election.cc:304] T f3b46e4fac694084b06b7086bcc596b9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:26.285131 20604 raft_consensus.cc:2749] T f3b46e4fac694084b06b7086bcc596b9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:26.256188 19276 rpcz_store.cc:276] 0925 13:56:25.163567 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:25.736451 (+572884us) service_pool.cc:224] Handling call
0925 13:56:25.807885 (+ 71434us) spinlock_profiling.cc:243] Waited 70.7 ms on lock 0x7b5400040454. stack: 00007fdf0a30b323 00007fdf0a30b220 00007fdf09eaf574 0000000000516964 00000000005168f9 00007fdf114095c0 00007fdf114a86fa 00007fdf13219d8c 00007fdf13319be6 00007fdf10ecd634 00007fdf10ecd5c2 00007fdf10ecd4f1 00007fdf10ecd46c 00007fdf10ecc722 00007fdf0f16e09c 00007fdf0f16d4d6
0925 13:56:25.808746 (+ 861us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 13:56:25.809430 (+ 684us) catalog_manager.cc:3764] Apply alter schema
0925 13:56:25.809987 (+ 557us) catalog_manager.cc:3796] Acquired catalog manager lock
0925 13:56:25.810286 (+ 299us) catalog_manager.cc:3954] Updating metadata on disk
0925 13:56:26.249683 (+439397us) catalog_manager.cc:3985] Committing alterations to in-memory state
0925 13:56:26.255687 (+ 6004us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:26.047687 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:26.048090 (+ 403us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:26.048120 (+ 30us) write_op.cc:435] Acquired schema lock
0925 13:56:26.048133 (+ 13us) tablet.cc:662] Decoding operations
0925 13:56:26.048401 (+ 268us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:26.048443 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:26.048467 (+ 24us) tablet.cc:685] Acquiring locks for 1 operations
0925 13:56:26.048611 (+ 144us) tablet.cc:701] Row locks acquired
0925 13:56:26.048624 (+ 13us) write_op.cc:260] PREPARE: finished
0925 13:56:26.048738 (+ 114us) write_op.cc:270] Start()
0925 13:56:26.048822 (+ 84us) write_op.cc:276] Timestamp: P: 1790344586048700 usec, L: 0
0925 13:56:26.048856 (+ 34us) op_driver.cc:348] REPLICATION: starting
0925 13:56:26.049398 (+ 542us) log.cc:844] Serialized 405 byte log entry
0925 13:56:26.247455 (+198057us) op_driver.cc:464] REPLICATION: finished
0925 13:56:26.248291 (+ 836us) write_op.cc:301] APPLY: starting
0925 13:56:26.248377 (+ 86us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:26.248478 (+ 101us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:26.248576 (+ 98us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:26.248756 (+ 180us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:26.248932 (+ 176us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 13:56:26.248973 (+ 41us) write_op.cc:312] APPLY: finished
0925 13:56:26.249149 (+ 176us) log.cc:844] Serialized 23 byte log entry
0925 13:56:26.249299 (+ 150us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:26.249353 (+ 54us) write_op.cc:454] Released schema lock
0925 13:56:26.249381 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0925 13:56:26.250593 (+ 1212us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:26.250721 (+ 128us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":162643712,"child_traces":[["op",{"apply.queue_time_us":659,"apply.run_cpu_time_us":1287,"apply.run_wall_time_us":1931,"num_ops":1,"prepare.queue_time_us":230572,"prepare.run_cpu_time_us":1956,"prepare.run_wall_time_us":3112,"raft.queue_time_us":64,"raft.run_cpu_time_us":13,"raft.run_wall_time_us":14,"replication_time_us":198398}]]}
I20260925 13:56:26.288120 20571 consensus_queue.cc:1048] T f16050752ad44720ba8703ef10bbb359 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 13:56:26.296830 19773 leader_election.cc:343] T a8a679a453e0430b866edc707fe7c234 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:26.297055 19938 tablet_bootstrap.cc:492] T eff33a56361946edb556ac611d2299eb P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:26.297305 19773 leader_election.cc:304] T a8a679a453e0430b866edc707fe7c234 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:26.297578 19938 ts_tablet_manager.cc:1403] T eff33a56361946edb556ac611d2299eb P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.078s user 0.011s sys 0.014s
I20260925 13:56:26.298071 20604 raft_consensus.cc:2749] T a8a679a453e0430b866edc707fe7c234 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:26.276674 20375 raft_consensus.cc:493] T 5788849b7336496488e5401b4ac91191 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260925 13:56:26.299048 19773 leader_election.cc:343] T 5f8a96ae6eec4f15a221f9aad537c094 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:26.299332 20375 raft_consensus.cc:515] T 5788849b7336496488e5401b4ac91191 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:26.299736 19773 leader_election.cc:304] T 5f8a96ae6eec4f15a221f9aad537c094 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:26.300460 19938 raft_consensus.cc:359] T eff33a56361946edb556ac611d2299eb P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:26.301045 19938 raft_consensus.cc:385] T eff33a56361946edb556ac611d2299eb P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:26.301254 19938 raft_consensus.cc:740] T eff33a56361946edb556ac611d2299eb P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:26.301445 20375 leader_election.cc:290] T 5788849b7336496488e5401b4ac91191 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:26.301774 19938 consensus_queue.cc:260] T eff33a56361946edb556ac611d2299eb P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
W20260925 13:56:26.305087 19488 leader_election.cc:343] T 5788849b7336496488e5401b4ac91191 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Not found: Tablet not found: 5788849b7336496488e5401b4ac91191
I20260925 13:56:26.307461 19942 tablet_bootstrap.cc:492] T 3aaf29a2310a41d0971eded7ef51fdc9 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:26.268806 20523 raft_consensus.cc:493] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:26.308244 19942 ts_tablet_manager.cc:1403] T 3aaf29a2310a41d0971eded7ef51fdc9 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.207s user 0.009s sys 0.011s
I20260925 13:56:26.295702 19490 leader_election.cc:304] T f16050752ad44720ba8703ef10bbb359 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3; no voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:26.308625 20523 raft_consensus.cc:515] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:26.309082 20375 raft_consensus.cc:3060] T f16050752ad44720ba8703ef10bbb359 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:26.315325 20478 raft_consensus.cc:2749] T 5f8a96ae6eec4f15a221f9aad537c094 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:26.324682 19831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb794cef052b43da831531455e2b5f15" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:26.325304 19831 raft_consensus.cc:2468] T eb794cef052b43da831531455e2b5f15 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:26.341683 19276 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_1f476c49446b4488821bbbd7d02b71cd" } new_table_name: "default.Table_49155edad5db40549fa76e9195967efc" modify_external_catalogs: true
I20260925 13:56:26.343216 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a86ea84d1d49d1b0950b4ce3a3d770" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:26.347100 20523 leader_election.cc:290] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:26.348011 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 145) took 5936 ms. Trace:
I20260925 13:56:26.348044 19631 leader_election.cc:304] T eb794cef052b43da831531455e2b5f15 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:26.312158 19942 raft_consensus.cc:359] T 3aaf29a2310a41d0971eded7ef51fdc9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:26.348732 19955 rpcz_store.cc:276] 0925 13:56:20.411310 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:20.411439 (+ 129us) service_pool.cc:224] Handling call
0925 13:56:26.172932 (+5761493us) ts_tablet_manager.cc:1151] Deleting tablet a26f5a02311c435a8c2434fd030c80a6
0925 13:56:26.172994 (+ 62us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:26.251261 (+ 78267us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:26.257691 (+ 6430us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:26.347945 (+ 90254us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8483,"tablet-delete.queue_time_us":5725621}
I20260925 13:56:26.349411 19942 raft_consensus.cc:385] T 3aaf29a2310a41d0971eded7ef51fdc9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:26.349670 19942 raft_consensus.cc:740] T 3aaf29a2310a41d0971eded7ef51fdc9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:26.350302 20571 raft_consensus.cc:2804] T eb794cef052b43da831531455e2b5f15 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:26.350625 20571 raft_consensus.cc:493] T eb794cef052b43da831531455e2b5f15 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:26.350924 20571 raft_consensus.cc:3060] T eb794cef052b43da831531455e2b5f15 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:26.350994 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet a26f5a02311c435a8c2434fd030c80a6 (table default.Table_65608937f2cd4986b2588a4831750601 [id=7079cc0c725b4578a1b11c7dd3ef9342]) successfully deleted
I20260925 13:56:26.352080 19955 tablet_replica.cc:333] T e6e59d6f4f6e4de1be1cf450620cfa59 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:26.352222 19942 consensus_queue.cc:260] T 3aaf29a2310a41d0971eded7ef51fdc9 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:26.352811 19955 raft_consensus.cc:2243] T e6e59d6f4f6e4de1be1cf450620cfa59 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:26.356633 19955 raft_consensus.cc:2272] T e6e59d6f4f6e4de1be1cf450620cfa59 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:26.359366 19942 ts_tablet_manager.cc:1434] T 3aaf29a2310a41d0971eded7ef51fdc9 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.049s user 0.005s sys 0.000s
I20260925 13:56:26.362253 19942 tablet_bootstrap.cc:492] T 5a9b3dfaa7474611a1824d63eb906b43 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:26.367142 19955 ts_tablet_manager.cc:1916] T e6e59d6f4f6e4de1be1cf450620cfa59 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:26.368156 19942 tablet_bootstrap.cc:654] T 5a9b3dfaa7474611a1824d63eb906b43 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:26.380888 19271 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 65ad0197267d47158751bc44c539b555
I20260925 13:56:26.382416 19271 catalog_manager.cc:6934] ReplaceTablet: tablet 65ad0197267d47158751bc44c539b555 deleted and replaced by tablet f806a82ee59c4da79e2f30647adc3c60
I20260925 13:56:26.383059 19271 master_service.cc:960] ReplaceTablet: received request to replace tablet 953d41f26f5f4027acfd374c6ec4ee75 from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:26.385280 19522 tablet_service.cc:1558] Processing DeleteTablet for tablet 65ad0197267d47158751bc44c539b555 with delete_type TABLET_DATA_DELETED (replaced by tablet f806a82ee59c4da79e2f30647adc3c60) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:26.394830 19271 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_d5b57f5cf2574edcbe5bd91f38508763"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:26.395776 19488 consensus_peers.cc:597] T c161ae3edfed4f8488c770ecc6332607 P 8f578869bc084a5d8a9a69f143954eb3 -> Peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Couldn't send request to peer 7c1b52895a14482fb54a6758f5054f8f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c161ae3edfed4f8488c770ecc6332607. This is attempt 1: this message will repeat every 5th retry.
W20260925 13:56:26.396852 19271 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d5b57f5cf2574edcbe5bd91f38508763 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:26.400446 19812 tablet_service.cc:1558] Processing DeleteTablet for tablet 65ad0197267d47158751bc44c539b555 with delete_type TABLET_DATA_DELETED (replaced by tablet f806a82ee59c4da79e2f30647adc3c60) from {username='slave'} at 127.0.0.1:51442
W20260925 13:56:26.405287 19266 catalog_manager.cc:5043] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): delete failed for tablet 65ad0197267d47158751bc44c539b555 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 65ad0197267d47158751bc44c539b555 already in progress: creating tablet
I20260925 13:56:26.408142 20604 raft_consensus.cc:493] T 31049bccd170436893512a7e9ef435cc P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:26.408582 20604 raft_consensus.cc:515] T 31049bccd170436893512a7e9ef435cc P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:26.410531 20604 leader_election.cc:290] T 31049bccd170436893512a7e9ef435cc P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:26.388670 19667 tablet_service.cc:1558] Processing DeleteTablet for tablet 65ad0197267d47158751bc44c539b555 with delete_type TABLET_DATA_DELETED (replaced by tablet f806a82ee59c4da79e2f30647adc3c60) from {username='slave'} at 127.0.0.1:45948
W20260925 13:56:26.414781 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 65ad0197267d47158751bc44c539b555 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 65ad0197267d47158751bc44c539b555 already in progress: creating tablet
I20260925 13:56:26.419030 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31049bccd170436893512a7e9ef435cc" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:26.421830 19773 leader_election.cc:343] T 31049bccd170436893512a7e9ef435cc P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:26.423885 19276 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 69934b9a49354c92b5b13e182247ef77 consensus state has no leader
I20260925 13:56:26.426466 19276 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_d1ac7e953d034f28babf9808d47e7fbf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:26.419340 19955 ts_tablet_manager.cc:1929] T e6e59d6f4f6e4de1be1cf450620cfa59 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:26.432827 19955 log.cc:1199] T e6e59d6f4f6e4de1be1cf450620cfa59 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/e6e59d6f4f6e4de1be1cf450620cfa59
I20260925 13:56:26.433882 19955 ts_tablet_manager.cc:1950] T e6e59d6f4f6e4de1be1cf450620cfa59 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:26.437163 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 146) took 6025 ms. Trace:
I20260925 13:56:26.440361 20571 raft_consensus.cc:515] T eb794cef052b43da831531455e2b5f15 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:26.444192 19955 rpcz_store.cc:276] 0925 13:56:20.412114 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:20.412215 (+ 101us) service_pool.cc:224] Handling call
0925 13:56:26.351916 (+5939701us) ts_tablet_manager.cc:1151] Deleting tablet e6e59d6f4f6e4de1be1cf450620cfa59
0925 13:56:26.351967 (+ 51us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:26.397500 (+ 45533us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:26.419220 (+ 21720us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:26.437091 (+ 17871us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5904334}
I20260925 13:56:26.446277 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet e6e59d6f4f6e4de1be1cf450620cfa59 (table default.Table_546097946d054781b0acae5186c5d43d [id=912de4c02216461f8a69b880207d8652]) successfully deleted
I20260925 13:56:26.447511 19955 tablet_replica.cc:333] T bbfc4f99a5964e2ba6146e390e8dba44 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:26.452025 19955 raft_consensus.cc:2243] T bbfc4f99a5964e2ba6146e390e8dba44 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:26.453722 19955 raft_consensus.cc:2272] T bbfc4f99a5964e2ba6146e390e8dba44 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:26.456453 19955 ts_tablet_manager.cc:1916] T bbfc4f99a5964e2ba6146e390e8dba44 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:26.457768 20375 raft_consensus.cc:2749] T f16050752ad44720ba8703ef10bbb359 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:26.467801 20604 consensus_queue.cc:1048] T 1215d875209a40d38cc527e2795d6c75 P f790493ba7ad403993b678f2df2b55be [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
W20260925 13:56:26.501499 19276 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d1ac7e953d034f28babf9808d47e7fbf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:26.514782 19955 ts_tablet_manager.cc:1929] T bbfc4f99a5964e2ba6146e390e8dba44 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:26.515146 19955 log.cc:1199] T bbfc4f99a5964e2ba6146e390e8dba44 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/bbfc4f99a5964e2ba6146e390e8dba44
I20260925 13:56:26.528602 19938 ts_tablet_manager.cc:1434] T eff33a56361946edb556ac611d2299eb P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.231s user 0.023s sys 0.034s
I20260925 13:56:26.529505 19955 ts_tablet_manager.cc:1950] T bbfc4f99a5964e2ba6146e390e8dba44 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:26.298837 19939 tablet_bootstrap.cc:654] T 3aaf29a2310a41d0971eded7ef51fdc9 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:26.464766 19942 tablet_bootstrap.cc:492] T 5a9b3dfaa7474611a1824d63eb906b43 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:26.539825 19942 ts_tablet_manager.cc:1403] T 5a9b3dfaa7474611a1824d63eb906b43 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.178s user 0.008s sys 0.011s
I20260925 13:56:26.542418 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 144) took 6131 ms. Trace:
W20260925 13:56:26.542899 19490 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 247300us
I20260925 13:56:26.542742 19955 rpcz_store.cc:276] 0925 13:56:20.411048 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:20.443644 (+ 32596us) service_pool.cc:224] Handling call
0925 13:56:26.447329 (+6003685us) ts_tablet_manager.cc:1151] Deleting tablet bbfc4f99a5964e2ba6146e390e8dba44
0925 13:56:26.447410 (+ 81us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:26.487260 (+ 39850us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:26.514665 (+ 27405us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:26.542350 (+ 27685us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5890005}
I20260925 13:56:26.542865 19942 raft_consensus.cc:359] T 5a9b3dfaa7474611a1824d63eb906b43 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:26.543967 19942 raft_consensus.cc:385] T 5a9b3dfaa7474611a1824d63eb906b43 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:26.544267 19942 raft_consensus.cc:740] T 5a9b3dfaa7474611a1824d63eb906b43 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:26.544319 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet bbfc4f99a5964e2ba6146e390e8dba44 (table default.Table_5d692242213f4df6863089b93b4e9636 [id=6d79107dc2d941058c5ecbe9a4410843]) successfully deleted
W20260925 13:56:26.545161 19773 leader_election.cc:343] T e2e44b2ca25c4bbcb91288424b3e18b9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:26.545056 19942 consensus_queue.cc:260] T 5a9b3dfaa7474611a1824d63eb906b43 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:26.545594 19773 leader_election.cc:304] T e2e44b2ca25c4bbcb91288424b3e18b9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:26.547129 19955 tablet_replica.cc:333] T 12984db09f34472ba365fdf63a33b720 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:26.548475 19955 raft_consensus.cc:2243] T 12984db09f34472ba365fdf63a33b720 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:26.549451 20642 raft_consensus.cc:2749] T e2e44b2ca25c4bbcb91288424b3e18b9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:26.556460 19942 ts_tablet_manager.cc:1434] T 5a9b3dfaa7474611a1824d63eb906b43 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.016s user 0.005s sys 0.000s
I20260925 13:56:26.557624 19942 tablet_bootstrap.cc:492] T 908df914cfec490cb7058dbe7b501c97 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:26.562484 19955 raft_consensus.cc:2272] T 12984db09f34472ba365fdf63a33b720 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:26.568405 19838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb794cef052b43da831531455e2b5f15" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:26.568970 19838 raft_consensus.cc:3060] T eb794cef052b43da831531455e2b5f15 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:26.570977 19955 ts_tablet_manager.cc:1916] T 12984db09f34472ba365fdf63a33b720 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:26.574312 19938 tablet_bootstrap.cc:492] T e9dbcd85dc8647c68c120ba3fcfe1790 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:26.584883 19938 tablet_bootstrap.cc:654] T e9dbcd85dc8647c68c120ba3fcfe1790 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:26.599443 20571 leader_election.cc:290] T eb794cef052b43da831531455e2b5f15 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:26.601744 19272 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 18a6d1c4eb7948149e709a380da50d93
I20260925 13:56:26.603518 19522 tablet_service.cc:1558] Processing DeleteTablet for tablet 18a6d1c4eb7948149e709a380da50d93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:25 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:26.605021 19667 tablet_service.cc:1558] Processing DeleteTablet for tablet 18a6d1c4eb7948149e709a380da50d93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:25 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:26.605674 20011 tablet_replica.cc:333] T 18a6d1c4eb7948149e709a380da50d93 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:26.606376 20011 raft_consensus.cc:2243] T 18a6d1c4eb7948149e709a380da50d93 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:26.606887 20011 raft_consensus.cc:2272] T 18a6d1c4eb7948149e709a380da50d93 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:26.608762 20011 ts_tablet_manager.cc:1916] T 18a6d1c4eb7948149e709a380da50d93 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:26.610137 19942 tablet_bootstrap.cc:654] T 908df914cfec490cb7058dbe7b501c97 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:26.614565 19812 tablet_service.cc:1558] Processing DeleteTablet for tablet 18a6d1c4eb7948149e709a380da50d93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:25 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:26.615389 20644 tablet_replica.cc:333] T 18a6d1c4eb7948149e709a380da50d93 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:26.616240 20644 raft_consensus.cc:2243] T 18a6d1c4eb7948149e709a380da50d93 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:26.616791 19938 tablet_bootstrap.cc:492] T e9dbcd85dc8647c68c120ba3fcfe1790 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:26.617174 19938 ts_tablet_manager.cc:1403] T e9dbcd85dc8647c68c120ba3fcfe1790 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.043s user 0.008s sys 0.000s
W20260925 13:56:26.619446 19773 leader_election.cc:343] T 82972fa4868d4c59b3b9bb1240300938 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:26.619891 19773 leader_election.cc:304] T 82972fa4868d4c59b3b9bb1240300938 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:26.627705 20644 raft_consensus.cc:2272] T 18a6d1c4eb7948149e709a380da50d93 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:26.629958 20644 ts_tablet_manager.cc:1916] T 18a6d1c4eb7948149e709a380da50d93 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:26.631435 20604 raft_consensus.cc:2749] T 82972fa4868d4c59b3b9bb1240300938 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:26.631505 19938 raft_consensus.cc:359] T e9dbcd85dc8647c68c120ba3fcfe1790 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:26.615343 19939 tablet_bootstrap.cc:492] T 3aaf29a2310a41d0971eded7ef51fdc9 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:26.632331 19938 raft_consensus.cc:385] T e9dbcd85dc8647c68c120ba3fcfe1790 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:26.632484 19939 ts_tablet_manager.cc:1403] T 3aaf29a2310a41d0971eded7ef51fdc9 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.399s user 0.007s sys 0.003s
I20260925 13:56:26.632620 19938 raft_consensus.cc:740] T e9dbcd85dc8647c68c120ba3fcfe1790 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:26.633396 19938 consensus_queue.cc:260] T e9dbcd85dc8647c68c120ba3fcfe1790 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:26.635787 19938 ts_tablet_manager.cc:1434] T e9dbcd85dc8647c68c120ba3fcfe1790 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.018s user 0.006s sys 0.001s
I20260925 13:56:26.656404 19938 tablet_bootstrap.cc:492] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:26.661574 19938 tablet_bootstrap.cc:654] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:26.673842 20011 ts_tablet_manager.cc:1929] T 18a6d1c4eb7948149e709a380da50d93 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:56:26.674819 19550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31049bccd170436893512a7e9ef435cc" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:26.675557 20011 log.cc:1199] T 18a6d1c4eb7948149e709a380da50d93 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/18a6d1c4eb7948149e709a380da50d93
I20260925 13:56:26.676450 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb794cef052b43da831531455e2b5f15" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:26.677070 19942 tablet_bootstrap.cc:492] T 908df914cfec490cb7058dbe7b501c97 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:26.677461 19942 ts_tablet_manager.cc:1403] T 908df914cfec490cb7058dbe7b501c97 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.120s user 0.008s sys 0.001s
I20260925 13:56:26.679826 19942 raft_consensus.cc:359] T 908df914cfec490cb7058dbe7b501c97 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:26.647794 19836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70a86ea84d1d49d1b0950b4ce3a3d770" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:26.680419 19942 raft_consensus.cc:385] T 908df914cfec490cb7058dbe7b501c97 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:26.680668 19942 raft_consensus.cc:740] T 908df914cfec490cb7058dbe7b501c97 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:26.676316 20011 ts_tablet_manager.cc:1950] T 18a6d1c4eb7948149e709a380da50d93 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
W20260925 13:56:26.683470 19267 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 69934b9a49354c92b5b13e182247ef77 (table default.Table_49155edad5db40549fa76e9195967efc [id=50c3843b240b47e1bfd1bfd07e3d4875], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 69934b9a49354c92b5b13e182247ef77 consensus state has no leader
I20260925 13:56:26.687435 20644 ts_tablet_manager.cc:1929] T 18a6d1c4eb7948149e709a380da50d93 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:56:26.687637 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 18a6d1c4eb7948149e709a380da50d93 (table default.Table_e814702f1f874529a98bd1631d3b5c84 [id=6c0ec9465714408da93ff6e47827eb48]) successfully deleted
I20260925 13:56:26.695748 19939 raft_consensus.cc:359] T 3aaf29a2310a41d0971eded7ef51fdc9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:26.695695 19942 consensus_queue.cc:260] T 908df914cfec490cb7058dbe7b501c97 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:26.696419 19939 raft_consensus.cc:385] T 3aaf29a2310a41d0971eded7ef51fdc9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:26.696687 19939 raft_consensus.cc:740] T 3aaf29a2310a41d0971eded7ef51fdc9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:26.697944 19271 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_29a7a24743f647108b2fa4629b620df5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:26.698304 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb794cef052b43da831531455e2b5f15" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
W20260925 13:56:26.699662 19629 leader_election.cc:343] T eb794cef052b43da831531455e2b5f15 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:26.701961 19955 ts_tablet_manager.cc:1929] T 12984db09f34472ba365fdf63a33b720 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:26.702338 19955 log.cc:1199] T 12984db09f34472ba365fdf63a33b720 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/12984db09f34472ba365fdf63a33b720
I20260925 13:56:26.703155 19955 ts_tablet_manager.cc:1950] T 12984db09f34472ba365fdf63a33b720 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:26.703970 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 151) took 5933 ms. Trace:
I20260925 13:56:26.704260 19955 rpcz_store.cc:276] 0925 13:56:20.770894 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:20.795988 (+ 25094us) service_pool.cc:224] Handling call
0925 13:56:26.546950 (+5750962us) ts_tablet_manager.cc:1151] Deleting tablet 12984db09f34472ba365fdf63a33b720
0925 13:56:26.547016 (+ 66us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:26.645513 (+ 98497us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:26.701842 (+ 56329us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:26.703922 (+ 2080us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5750299}
I20260925 13:56:26.705914 19276 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_746abdd6da964b3592615357002e76bd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:26.708284 19942 ts_tablet_manager.cc:1434] T 908df914cfec490cb7058dbe7b501c97 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.031s user 0.008s sys 0.001s
I20260925 13:56:26.713665 19942 tablet_bootstrap.cc:492] T 2711e7a6973546b1822d93f91095dae2 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:26.714995 20644 log.cc:1199] T 18a6d1c4eb7948149e709a380da50d93 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/18a6d1c4eb7948149e709a380da50d93
I20260925 13:56:26.716810 20644 ts_tablet_manager.cc:1950] T 18a6d1c4eb7948149e709a380da50d93 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:26.697554 19838 raft_consensus.cc:2468] T eb794cef052b43da831531455e2b5f15 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
W20260925 13:56:26.699527 19773 leader_election.cc:343] T 31049bccd170436893512a7e9ef435cc P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:26.717764 19773 leader_election.cc:304] T 31049bccd170436893512a7e9ef435cc P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
W20260925 13:56:26.718956 19773 consensus_peers.cc:597] T 1215d875209a40d38cc527e2795d6c75 P f790493ba7ad403993b678f2df2b55be -> Peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Couldn't send request to peer 8f578869bc084a5d8a9a69f143954eb3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:26.702143 19938 tablet_bootstrap.cc:492] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:26.719807 19938 ts_tablet_manager.cc:1403] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.064s user 0.004s sys 0.005s
I20260925 13:56:26.722486 19938 raft_consensus.cc:359] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:26.708926 19939 consensus_queue.cc:260] T 3aaf29a2310a41d0971eded7ef51fdc9 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:26.723858 20604 raft_consensus.cc:2749] T 31049bccd170436893512a7e9ef435cc P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:26.712581 20642 raft_consensus.cc:493] T eff33a56361946edb556ac611d2299eb P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:26.724334 20571 consensus_queue.cc:1048] T f16050752ad44720ba8703ef10bbb359 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:26.724545 20642 raft_consensus.cc:515] T eff33a56361946edb556ac611d2299eb P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:26.726434 20642 leader_election.cc:290] T eff33a56361946edb556ac611d2299eb P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:26.732880 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff33a56361946edb556ac611d2299eb" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:26.734404 19773 leader_election.cc:343] T eff33a56361946edb556ac611d2299eb P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:26.735227 19631 leader_election.cc:304] T eb794cef052b43da831531455e2b5f15 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:26.723129 19938 raft_consensus.cc:385] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:26.735908 19938 raft_consensus.cc:740] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:26.736034 19955 tablet_replica.cc:333] T 137c43fb74ec4cc48d30ed3edac81d98 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:26.736570 20571 raft_consensus.cc:2804] T eb794cef052b43da831531455e2b5f15 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:26.736749 19955 raft_consensus.cc:2243] T 137c43fb74ec4cc48d30ed3edac81d98 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:26.737030 20571 raft_consensus.cc:697] T eb794cef052b43da831531455e2b5f15 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Becoming Leader. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Running, Role: LEADER
I20260925 13:56:26.737798 20571 consensus_queue.cc:237] T eb794cef052b43da831531455e2b5f15 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:26.738041 19490 leader_election.cc:343] T 5788849b7336496488e5401b4ac91191 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 5788849b7336496488e5401b4ac91191
I20260925 13:56:26.705600 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet 12984db09f34472ba365fdf63a33b720 (table default.Table_10868b99c6ec43b2a38a3a4a5bdcf960 [id=46b6ad1a631c4c08a05ef5a1b5f62ab9]) successfully deleted
I20260925 13:56:26.743932 20375 raft_consensus.cc:493] T 3aaf29a2310a41d0971eded7ef51fdc9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:26.744328 20375 raft_consensus.cc:515] T 3aaf29a2310a41d0971eded7ef51fdc9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:26.745545 19276 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_746abdd6da964b3592615357002e76bd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:26.746222 20375 leader_election.cc:290] T 3aaf29a2310a41d0971eded7ef51fdc9 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:26.746861 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 18a6d1c4eb7948149e709a380da50d93 (table default.Table_e814702f1f874529a98bd1631d3b5c84 [id=6c0ec9465714408da93ff6e47827eb48]) successfully deleted
I20260925 13:56:26.747305 19938 consensus_queue.cc:260] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:26.750051 19938 ts_tablet_manager.cc:1434] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.030s user 0.005s sys 0.001s
I20260925 13:56:26.750855 19938 tablet_bootstrap.cc:492] T c5a9df833271480aa49feb7d9f925459 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:26.751391 19955 raft_consensus.cc:2272] T 137c43fb74ec4cc48d30ed3edac81d98 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:26.753348 19955 ts_tablet_manager.cc:1916] T 137c43fb74ec4cc48d30ed3edac81d98 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:26.754990 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaf29a2310a41d0971eded7ef51fdc9" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:26.755591 19692 raft_consensus.cc:2468] T 3aaf29a2310a41d0971eded7ef51fdc9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:26.756845 19488 leader_election.cc:304] T 3aaf29a2310a41d0971eded7ef51fdc9 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3; no voters:
I20260925 13:56:26.758291 20523 raft_consensus.cc:2804] T 3aaf29a2310a41d0971eded7ef51fdc9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:26.758611 20523 raft_consensus.cc:493] T 3aaf29a2310a41d0971eded7ef51fdc9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:26.758903 20523 raft_consensus.cc:3060] T 3aaf29a2310a41d0971eded7ef51fdc9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
W20260925 13:56:26.767531 19271 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_29a7a24743f647108b2fa4629b620df5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:26.768615 19942 tablet_bootstrap.cc:654] T 2711e7a6973546b1822d93f91095dae2 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:26.774427 19938 tablet_bootstrap.cc:654] T c5a9df833271480aa49feb7d9f925459 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:26.780791 20523 raft_consensus.cc:515] T 3aaf29a2310a41d0971eded7ef51fdc9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:26.784238 19271 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_cbf20322808c49ef86dff2c576d5654c" } new_table_name: "default.Table_8a9f077513e94c6d985f3c86954c3ccd" modify_external_catalogs: true
I20260925 13:56:26.794378 19276 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_954c5af93adb4e8586f870597a63bb74" } new_table_name: "default.Table_95a1130b07a34c82a6a6d7bb1c8245c8" modify_external_catalogs: true
I20260925 13:56:26.741904 19490 leader_election.cc:304] T 5788849b7336496488e5401b4ac91191 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3; no voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:26.800344 20375 raft_consensus.cc:2749] T 5788849b7336496488e5401b4ac91191 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:56:26.801472 19490 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 63925.2us
I20260925 13:56:26.803926 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaf29a2310a41d0971eded7ef51fdc9" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:26.804471 19692 raft_consensus.cc:3060] T 3aaf29a2310a41d0971eded7ef51fdc9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
W20260925 13:56:26.804536 19271 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9c5ef5b050624127990737acab3a8e9f consensus state has no leader
I20260925 13:56:26.808995 19942 tablet_bootstrap.cc:492] T 2711e7a6973546b1822d93f91095dae2 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:26.809387 19942 ts_tablet_manager.cc:1403] T 2711e7a6973546b1822d93f91095dae2 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.096s user 0.003s sys 0.005s
I20260925 13:56:26.811713 19271 master_service.cc:960] ReplaceTablet: received request to replace tablet eb794cef052b43da831531455e2b5f15 from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:26.813066 19942 raft_consensus.cc:359] T 2711e7a6973546b1822d93f91095dae2 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:26.813648 19942 raft_consensus.cc:385] T 2711e7a6973546b1822d93f91095dae2 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:26.813890 19942 raft_consensus.cc:740] T 2711e7a6973546b1822d93f91095dae2 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
W20260925 13:56:26.814640 19490 consensus_peers.cc:597] T 18a6d1c4eb7948149e709a380da50d93 P 8f578869bc084a5d8a9a69f143954eb3 -> Peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Couldn't send request to peer f790493ba7ad403993b678f2df2b55be. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:26.814447 19942 consensus_queue.cc:260] T 2711e7a6973546b1822d93f91095dae2 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:26.815541 19490 consensus_peers.cc:597] T c161ae3edfed4f8488c770ecc6332607 P 8f578869bc084a5d8a9a69f143954eb3 -> Peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Couldn't send request to peer f790493ba7ad403993b678f2df2b55be. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c161ae3edfed4f8488c770ecc6332607. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:26.816094 19490 leader_election.cc:304] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3; no voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:26.816754 20375 raft_consensus.cc:2749] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:26.786512 19939 ts_tablet_manager.cc:1434] T 3aaf29a2310a41d0971eded7ef51fdc9 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.154s user 0.008s sys 0.000s
I20260925 13:56:26.819159 19939 tablet_bootstrap.cc:492] T eb794cef052b43da831531455e2b5f15 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:26.823002 19955 ts_tablet_manager.cc:1929] T 137c43fb74ec4cc48d30ed3edac81d98 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:26.824542 19692 raft_consensus.cc:2468] T 3aaf29a2310a41d0971eded7ef51fdc9 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:26.825743 19488 leader_election.cc:304] T 3aaf29a2310a41d0971eded7ef51fdc9 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3; no voters:
I20260925 13:56:26.826527 20375 raft_consensus.cc:2804] T 3aaf29a2310a41d0971eded7ef51fdc9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:26.826958 20375 raft_consensus.cc:697] T 3aaf29a2310a41d0971eded7ef51fdc9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Running, Role: LEADER
I20260925 13:56:26.827713 20375 consensus_queue.cc:237] T 3aaf29a2310a41d0971eded7ef51fdc9 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:26.832177 19488 consensus_peers.cc:597] T 18a6d1c4eb7948149e709a380da50d93 P 8f578869bc084a5d8a9a69f143954eb3 -> Peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Couldn't send request to peer 7c1b52895a14482fb54a6758f5054f8f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 18a6d1c4eb7948149e709a380da50d93. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:26.833428 19942 ts_tablet_manager.cc:1434] T 2711e7a6973546b1822d93f91095dae2 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.024s user 0.006s sys 0.001s
I20260925 13:56:26.835945 20523 leader_election.cc:290] T 3aaf29a2310a41d0971eded7ef51fdc9 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
W20260925 13:56:26.846192 19629 leader_election.cc:343] T eb794cef052b43da831531455e2b5f15 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:26.846578 19838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaf29a2310a41d0971eded7ef51fdc9" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:26.847390 19838 raft_consensus.cc:2468] T 3aaf29a2310a41d0971eded7ef51fdc9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:26.853370 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff33a56361946edb556ac611d2299eb" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:26.854112 19838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aaf29a2310a41d0971eded7ef51fdc9" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:26.854609 19942 tablet_bootstrap.cc:492] T 0fdb45b31f93466c8bed1b848c6cdf15 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:26.854646 19838 raft_consensus.cc:3060] T 3aaf29a2310a41d0971eded7ef51fdc9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
W20260925 13:56:26.858057 19276 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3aaf29a2310a41d0971eded7ef51fdc9 consensus state has no leader
I20260925 13:56:26.858963 19276 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_04d053f331f243ce8e98beb7c00d4159" } modify_external_catalogs: true
I20260925 13:56:26.859505 19276 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_04d053f331f243ce8e98beb7c00d4159" } modify_external_catalogs: true
I20260925 13:56:26.866611 19955 log.cc:1199] T 137c43fb74ec4cc48d30ed3edac81d98 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/137c43fb74ec4cc48d30ed3edac81d98
I20260925 13:56:26.867480 19955 ts_tablet_manager.cc:1950] T 137c43fb74ec4cc48d30ed3edac81d98 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:26.869357 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 152) took 5968 ms. Trace:
I20260925 13:56:26.869683 19955 rpcz_store.cc:276] 0925 13:56:20.900749 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:20.931149 (+ 30400us) service_pool.cc:224] Handling call
0925 13:56:26.735876 (+5804727us) ts_tablet_manager.cc:1151] Deleting tablet 137c43fb74ec4cc48d30ed3edac81d98
0925 13:56:26.735932 (+ 56us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:26.800460 (+ 64528us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:26.822896 (+ 22436us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:26.869299 (+ 46403us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5797241}
I20260925 13:56:26.871016 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 153) took 5846 ms. Trace:
I20260925 13:56:26.871980 19939 tablet_bootstrap.cc:654] T eb794cef052b43da831531455e2b5f15 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:26.878135 19838 raft_consensus.cc:2468] T 3aaf29a2310a41d0971eded7ef51fdc9 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:26.888104 19942 tablet_bootstrap.cc:654] T 0fdb45b31f93466c8bed1b848c6cdf15 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:26.871320 19955 rpcz_store.cc:276] 0925 13:56:21.024120 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:21.024544 (+ 424us) service_pool.cc:224] Handling call
0925 13:56:26.870619 (+5846075us) ts_tablet_manager.cc:1151] Deleting tablet 5a9b3dfaa7474611a1824d63eb906b43
0925 13:56:26.870681 (+ 62us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:26.870966 (+ 285us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5794575}
I20260925 13:56:26.892232 19955 tablet_replica.cc:333] T 5788849b7336496488e5401b4ac91191 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:26.893117 19955 raft_consensus.cc:2243] T 5788849b7336496488e5401b4ac91191 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:26.893612 19955 raft_consensus.cc:2272] T 5788849b7336496488e5401b4ac91191 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:26.895900 19276 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet f5a0bd0e205f4889be93739dfbd2a816
I20260925 13:56:26.897565 19667 tablet_service.cc:1558] Processing DeleteTablet for tablet f5a0bd0e205f4889be93739dfbd2a816 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:26 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:26.898389 20011 tablet_replica.cc:333] T f5a0bd0e205f4889be93739dfbd2a816 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:26.899292 20011 raft_consensus.cc:2243] T f5a0bd0e205f4889be93739dfbd2a816 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:26.899935 20011 raft_consensus.cc:2272] T f5a0bd0e205f4889be93739dfbd2a816 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:26.901268 19955 ts_tablet_manager.cc:1916] T 5788849b7336496488e5401b4ac91191 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:26.902010 20011 ts_tablet_manager.cc:1916] T f5a0bd0e205f4889be93739dfbd2a816 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:26.908803 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet 137c43fb74ec4cc48d30ed3edac81d98 (table default.Table_5321051e9f6e4cd28b9b77619c908dc9 [id=9dac142084c948f9a50cfc9acf24d461]) successfully deleted
W20260925 13:56:26.885402 19773 leader_election.cc:343] T eff33a56361946edb556ac611d2299eb P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:26.933744 19773 leader_election.cc:304] T eff33a56361946edb556ac611d2299eb P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
W20260925 13:56:26.934057 19264 catalog_manager.cc:5043] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): delete failed for tablet 5a9b3dfaa7474611a1824d63eb906b43 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5a9b3dfaa7474611a1824d63eb906b43 already in progress: creating tablet
W20260925 13:56:26.934571 19773 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.193:38269, user_credentials={real_user=slave}} blocked reactor thread for 49195.8us
I20260925 13:56:26.935506 19812 tablet_service.cc:1558] Processing DeleteTablet for tablet f5a0bd0e205f4889be93739dfbd2a816 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:26 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:26.935453 20642 raft_consensus.cc:2749] T eff33a56361946edb556ac611d2299eb P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:26.936233 20644 tablet_replica.cc:333] T f5a0bd0e205f4889be93739dfbd2a816 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:26.937032 20644 raft_consensus.cc:2243] T f5a0bd0e205f4889be93739dfbd2a816 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:26.937522 20644 raft_consensus.cc:2272] T f5a0bd0e205f4889be93739dfbd2a816 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:26.939369 20644 ts_tablet_manager.cc:1916] T f5a0bd0e205f4889be93739dfbd2a816 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:26.939893 20571 raft_consensus.cc:493] T a8a679a453e0430b866edc707fe7c234 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:26.940340 20571 raft_consensus.cc:515] T a8a679a453e0430b866edc707fe7c234 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:26.942202 20571 leader_election.cc:290] T a8a679a453e0430b866edc707fe7c234 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:26.943608 19276 master_service.cc:960] ReplaceTablet: received request to replace tablet f01fc02d12c8461f96ef0a93412ac641 from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:26.955183 19838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8a679a453e0430b866edc707fe7c234" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:26.957391 19838 raft_consensus.cc:2468] T a8a679a453e0430b866edc707fe7c234 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:26.961299 19631 leader_election.cc:304] T a8a679a453e0430b866edc707fe7c234 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:26.962001 20571 raft_consensus.cc:2804] T a8a679a453e0430b866edc707fe7c234 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:26.962309 20571 raft_consensus.cc:493] T a8a679a453e0430b866edc707fe7c234 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:26.962579 20571 raft_consensus.cc:3060] T a8a679a453e0430b866edc707fe7c234 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:26.966859 19522 tablet_service.cc:1558] Processing DeleteTablet for tablet f5a0bd0e205f4889be93739dfbd2a816 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:26 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:26.969319 20571 raft_consensus.cc:515] T a8a679a453e0430b866edc707fe7c234 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:26.972258 19836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8a679a453e0430b866edc707fe7c234" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:26.972817 19836 raft_consensus.cc:3060] T a8a679a453e0430b866edc707fe7c234 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:26.977650 19938 tablet_bootstrap.cc:492] T c5a9df833271480aa49feb7d9f925459 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:26.978044 19938 ts_tablet_manager.cc:1403] T c5a9df833271480aa49feb7d9f925459 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.227s user 0.022s sys 0.020s
I20260925 13:56:26.979367 19836 raft_consensus.cc:2468] T a8a679a453e0430b866edc707fe7c234 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
I20260925 13:56:26.980441 19631 leader_election.cc:304] T a8a679a453e0430b866edc707fe7c234 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:26.980420 19938 raft_consensus.cc:359] T c5a9df833271480aa49feb7d9f925459 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:26.981145 20625 raft_consensus.cc:2804] T a8a679a453e0430b866edc707fe7c234 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:26.981592 20625 raft_consensus.cc:697] T a8a679a453e0430b866edc707fe7c234 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Becoming Leader. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Running, Role: LEADER
I20260925 13:56:26.982283 20625 consensus_queue.cc:237] T a8a679a453e0430b866edc707fe7c234 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:26.988091 19942 tablet_bootstrap.cc:492] T 0fdb45b31f93466c8bed1b848c6cdf15 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:26.988476 19942 ts_tablet_manager.cc:1403] T 0fdb45b31f93466c8bed1b848c6cdf15 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.134s user 0.019s sys 0.007s
I20260925 13:56:26.990749 19942 raft_consensus.cc:359] T 0fdb45b31f93466c8bed1b848c6cdf15 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:26.991325 19942 raft_consensus.cc:385] T 0fdb45b31f93466c8bed1b848c6cdf15 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:26.991600 19942 raft_consensus.cc:740] T 0fdb45b31f93466c8bed1b848c6cdf15 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:26.991550 20642 raft_consensus.cc:493] T e9dbcd85dc8647c68c120ba3fcfe1790 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:26.991976 20642 raft_consensus.cc:515] T e9dbcd85dc8647c68c120ba3fcfe1790 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:26.992216 19942 consensus_queue.cc:260] T 0fdb45b31f93466c8bed1b848c6cdf15 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:26.993741 20642 leader_election.cc:290] T e9dbcd85dc8647c68c120ba3fcfe1790 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:26.995518 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9dbcd85dc8647c68c120ba3fcfe1790" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:26.997838 19773 leader_election.cc:343] T e9dbcd85dc8647c68c120ba3fcfe1790 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:26.999401 19942 ts_tablet_manager.cc:1434] T 0fdb45b31f93466c8bed1b848c6cdf15 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.009s user 0.002s sys 0.004s
I20260925 13:56:27.000712 19942 tablet_bootstrap.cc:492] T 0e8aa12e95dc434a98a426499593d9a5 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:27.002300 19938 raft_consensus.cc:385] T c5a9df833271480aa49feb7d9f925459 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:27.002627 19938 raft_consensus.cc:740] T c5a9df833271480aa49feb7d9f925459 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:27.008558 19938 consensus_queue.cc:260] T c5a9df833271480aa49feb7d9f925459 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:27.014228 19938 ts_tablet_manager.cc:1434] T c5a9df833271480aa49feb7d9f925459 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.036s user 0.006s sys 0.001s
I20260925 13:56:27.019454 20571 leader_election.cc:290] T a8a679a453e0430b866edc707fe7c234 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:27.019845 20642 consensus_queue.cc:1048] T f50ea68e3b3c4459b3f5e5b0df724724 P f790493ba7ad403993b678f2df2b55be [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:27.023541 20011 ts_tablet_manager.cc:1929] T f5a0bd0e205f4889be93739dfbd2a816 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:27.021396 19942 tablet_bootstrap.cc:654] T 0e8aa12e95dc434a98a426499593d9a5 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:27.023902 20011 log.cc:1199] T f5a0bd0e205f4889be93739dfbd2a816 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/f5a0bd0e205f4889be93739dfbd2a816
I20260925 13:56:27.024598 20011 ts_tablet_manager.cc:1950] T f5a0bd0e205f4889be93739dfbd2a816 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:27.027386 20642 raft_consensus.cc:493] T 089dc4dfeb5243788dc03061c8743a81 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:27.027849 20642 raft_consensus.cc:515] T 089dc4dfeb5243788dc03061c8743a81 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:27.026024 19939 tablet_bootstrap.cc:492] T eb794cef052b43da831531455e2b5f15 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:27.029793 19939 ts_tablet_manager.cc:1403] T eb794cef052b43da831531455e2b5f15 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.211s user 0.008s sys 0.002s
I20260925 13:56:27.030390 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "089dc4dfeb5243788dc03061c8743a81" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:27.031654 19773 leader_election.cc:343] T 089dc4dfeb5243788dc03061c8743a81 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:27.032795 20642 leader_election.cc:290] T 089dc4dfeb5243788dc03061c8743a81 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
W20260925 13:56:27.034416 19488 consensus_peers.cc:597] T f5a0bd0e205f4889be93739dfbd2a816 P 8f578869bc084a5d8a9a69f143954eb3 -> Peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Couldn't send request to peer 7c1b52895a14482fb54a6758f5054f8f. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: SHUTDOWN. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:27.037225 19551 raft_consensus.cc:3060] T 70a86ea84d1d49d1b0950b4ce3a3d770 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:27.037251 20661 raft_consensus.cc:493] T ec7fe5c834be4a51b627b8b1c3ebe888 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:27.037792 20661 raft_consensus.cc:515] T ec7fe5c834be4a51b627b8b1c3ebe888 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:27.041131 20661 leader_election.cc:290] T ec7fe5c834be4a51b627b8b1c3ebe888 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:27.047938 20644 ts_tablet_manager.cc:1929] T f5a0bd0e205f4889be93739dfbd2a816 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:27.048398 20644 log.cc:1199] T f5a0bd0e205f4889be93739dfbd2a816 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/f5a0bd0e205f4889be93739dfbd2a816
I20260925 13:56:27.049191 20644 ts_tablet_manager.cc:1950] T f5a0bd0e205f4889be93739dfbd2a816 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:27.060129 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8a679a453e0430b866edc707fe7c234" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:27.059260 19938 tablet_bootstrap.cc:492] T b3172d99afee4a4c878e71b645037917 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:27.064363 19942 tablet_bootstrap.cc:492] T 0e8aa12e95dc434a98a426499593d9a5 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:27.064770 19942 ts_tablet_manager.cc:1403] T 0e8aa12e95dc434a98a426499593d9a5 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.064s user 0.009s sys 0.002s
I20260925 13:56:27.068253 19551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8a679a453e0430b866edc707fe7c234" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:27.068039 19942 raft_consensus.cc:359] T 0e8aa12e95dc434a98a426499593d9a5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:27.068734 19942 raft_consensus.cc:385] T 0e8aa12e95dc434a98a426499593d9a5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:27.069037 19942 raft_consensus.cc:740] T 0e8aa12e95dc434a98a426499593d9a5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:27.069607 19942 consensus_queue.cc:260] T 0e8aa12e95dc434a98a426499593d9a5 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:27.072103 19942 ts_tablet_manager.cc:1434] T 0e8aa12e95dc434a98a426499593d9a5 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260925 13:56:27.075850 19629 leader_election.cc:343] T a8a679a453e0430b866edc707fe7c234 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:27.077893 19488 leader_election.cc:343] T ec7fe5c834be4a51b627b8b1c3ebe888 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Not found: Tablet not found: ec7fe5c834be4a51b627b8b1c3ebe888
I20260925 13:56:27.085412 19942 tablet_bootstrap.cc:492] T 82972fa4868d4c59b3b9bb1240300938 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:27.098337 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet f5a0bd0e205f4889be93739dfbd2a816 (table default.Table_04d053f331f243ce8e98beb7c00d4159 [id=745c9e0d4f2c4ce58c3a201c34f1f1e3]) successfully deleted
I20260925 13:56:27.100251 19939 raft_consensus.cc:359] T eb794cef052b43da831531455e2b5f15 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:27.101143 19939 raft_consensus.cc:385] T eb794cef052b43da831531455e2b5f15 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:27.101473 19939 raft_consensus.cc:740] T eb794cef052b43da831531455e2b5f15 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:27.058568 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet f5a0bd0e205f4889be93739dfbd2a816 (table default.Table_04d053f331f243ce8e98beb7c00d4159 [id=745c9e0d4f2c4ce58c3a201c34f1f1e3]) successfully deleted
W20260925 13:56:27.118242 19266 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 59698.1us
W20260925 13:56:27.118896 19266 consensus_peers.cc:597] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 -> Peer fc3953a2ba984a658630c423ec9b9728 (127.18.182.252:36181): Couldn't send request to peer fc3953a2ba984a658630c423ec9b9728. Status: Network error: Client connection negotiation failed: client connection to 127.18.182.252:36181: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260925 13:56:27.124027 19629 leader_election.cc:343] T a8a679a453e0430b866edc707fe7c234 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:27.125011 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9dbcd85dc8647c68c120ba3fcfe1790" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:27.125272 19547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "089dc4dfeb5243788dc03061c8743a81" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
W20260925 13:56:27.127494 19773 consensus_peers.cc:597] T f50ea68e3b3c4459b3f5e5b0df724724 P f790493ba7ad403993b678f2df2b55be -> Peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Couldn't send request to peer 8f578869bc084a5d8a9a69f143954eb3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260925 13:56:27.127851 19490 consensus_peers.cc:597] T f5a0bd0e205f4889be93739dfbd2a816 P 8f578869bc084a5d8a9a69f143954eb3 -> Peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Couldn't send request to peer f790493ba7ad403993b678f2df2b55be. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f5a0bd0e205f4889be93739dfbd2a816. This is attempt 1: this message will repeat every 5th retry.
W20260925 13:56:27.128414 19490 leader_election.cc:343] T ec7fe5c834be4a51b627b8b1c3ebe888 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: ec7fe5c834be4a51b627b8b1c3ebe888
I20260925 13:56:27.128813 19490 leader_election.cc:304] T ec7fe5c834be4a51b627b8b1c3ebe888 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3; no voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:27.129577 20661 raft_consensus.cc:2749] T ec7fe5c834be4a51b627b8b1c3ebe888 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:27.130409 19942 tablet_bootstrap.cc:654] T 82972fa4868d4c59b3b9bb1240300938 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:27.155701 20571 raft_consensus.cc:493] T 908df914cfec490cb7058dbe7b501c97 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:27.156343 20571 raft_consensus.cc:515] T 908df914cfec490cb7058dbe7b501c97 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:27.160079 20571 leader_election.cc:290] T 908df914cfec490cb7058dbe7b501c97 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:17.511933 20083 server_base.cc:1061] running on GCE node
W20260925 13:56:17.525566 20271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:56:17.495462 20278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:56:27.162470 19631 leader_election.cc:343] T 908df914cfec490cb7058dbe7b501c97 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 908df914cfec490cb7058dbe7b501c97
I20260925 13:56:27.163475 20083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:56:27.167896 19939 consensus_queue.cc:260] T eb794cef052b43da831531455e2b5f15 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:27.170389 19939 ts_tablet_manager.cc:1434] T eb794cef052b43da831531455e2b5f15 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.140s user 0.006s sys 0.001s
I20260925 13:56:27.181066 19942 tablet_bootstrap.cc:492] T 82972fa4868d4c59b3b9bb1240300938 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:27.181964 19942 ts_tablet_manager.cc:1403] T 82972fa4868d4c59b3b9bb1240300938 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.099s user 0.009s sys 0.003s
I20260925 13:56:27.184370 19942 raft_consensus.cc:359] T 82972fa4868d4c59b3b9bb1240300938 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:27.184957 19942 raft_consensus.cc:385] T 82972fa4868d4c59b3b9bb1240300938 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:27.185199 19942 raft_consensus.cc:740] T 82972fa4868d4c59b3b9bb1240300938 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
W20260925 13:56:27.185966 19490 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 57591.8us
W20260925 13:56:27.187027 19773 leader_election.cc:343] T e9dbcd85dc8647c68c120ba3fcfe1790 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:27.187520 19773 leader_election.cc:304] T e9dbcd85dc8647c68c120ba3fcfe1790 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
W20260925 13:56:27.188974 19773 leader_election.cc:343] T 089dc4dfeb5243788dc03061c8743a81 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:27.189325 19773 leader_election.cc:304] T 089dc4dfeb5243788dc03061c8743a81 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:27.191232 20478 raft_consensus.cc:2749] T e9dbcd85dc8647c68c120ba3fcfe1790 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:27.171284 19939 tablet_bootstrap.cc:492] T 2711e7a6973546b1822d93f91095dae2 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:27.191823 20478 raft_consensus.cc:2749] T 089dc4dfeb5243788dc03061c8743a81 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:27.197016 19551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "908df914cfec490cb7058dbe7b501c97" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:27.199741 20571 consensus_queue.cc:1048] T 9059655f2c3d4e86b3b389b5b9a68891 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:27.200770 20083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:56:27.205343 19938 tablet_bootstrap.cc:654] T b3172d99afee4a4c878e71b645037917 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:27.209429 20083 hybrid_clock.cc:648] HybridClock initialized: now 1790344587209324 us; error 92 us; skew 500 ppm
I20260925 13:56:27.210520 19955 ts_tablet_manager.cc:1929] T 5788849b7336496488e5401b4ac91191 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:27.210656 20083 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 13:56:27.210999 19955 log.cc:1199] T 5788849b7336496488e5401b4ac91191 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/5788849b7336496488e5401b4ac91191
W20260925 13:56:27.216637 19629 leader_election.cc:343] T 908df914cfec490cb7058dbe7b501c97 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:27.217075 19629 leader_election.cc:304] T 908df914cfec490cb7058dbe7b501c97 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:27.220340 20571 raft_consensus.cc:2749] T 908df914cfec490cb7058dbe7b501c97 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:27.226629 19938 tablet_bootstrap.cc:492] T b3172d99afee4a4c878e71b645037917 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:27.229486 19938 ts_tablet_manager.cc:1403] T b3172d99afee4a4c878e71b645037917 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.170s user 0.015s sys 0.000s
I20260925 13:56:27.229976 19942 consensus_queue.cc:260] T 82972fa4868d4c59b3b9bb1240300938 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:27.232501 19942 ts_tablet_manager.cc:1434] T 82972fa4868d4c59b3b9bb1240300938 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.050s user 0.005s sys 0.000s
I20260925 13:56:27.233361 19942 tablet_bootstrap.cc:492] T 953d41f26f5f4027acfd374c6ec4ee75 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:27.235561 20571 raft_consensus.cc:493] T 5a9b3dfaa7474611a1824d63eb906b43 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:27.235970 20571 raft_consensus.cc:515] T 5a9b3dfaa7474611a1824d63eb906b43 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:27.238135 20571 leader_election.cc:290] T 5a9b3dfaa7474611a1824d63eb906b43 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:27.238095 19938 raft_consensus.cc:359] T b3172d99afee4a4c878e71b645037917 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:27.238695 19938 raft_consensus.cc:385] T b3172d99afee4a4c878e71b645037917 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:27.238942 19938 raft_consensus.cc:740] T b3172d99afee4a4c878e71b645037917 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:27.239326 19955 ts_tablet_manager.cc:1950] T 5788849b7336496488e5401b4ac91191 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:27.239529 19938 consensus_queue.cc:260] T b3172d99afee4a4c878e71b645037917 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:27.241189 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 154) took 6177 ms. Trace:
W20260925 13:56:27.241570 19631 leader_election.cc:343] T 5a9b3dfaa7474611a1824d63eb906b43 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 5a9b3dfaa7474611a1824d63eb906b43
I20260925 13:56:27.241472 19955 rpcz_store.cc:276] 0925 13:56:21.063671 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:21.064208 (+ 537us) service_pool.cc:224] Handling call
0925 13:56:26.892032 (+5827824us) ts_tablet_manager.cc:1151] Deleting tablet 5788849b7336496488e5401b4ac91191
0925 13:56:26.892101 (+ 69us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:27.113334 (+221233us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:27.210382 (+ 97048us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:27.241122 (+ 30740us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5815763}
I20260925 13:56:27.242136 19938 ts_tablet_manager.cc:1434] T b3172d99afee4a4c878e71b645037917 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.012s user 0.005s sys 0.000s
I20260925 13:56:27.235272 19939 tablet_bootstrap.cc:654] T 2711e7a6973546b1822d93f91095dae2 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:27.242892 19938 tablet_bootstrap.cc:492] T 78d8c01c88d24deeae21d647e7f205ba P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:27.242856 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet 5788849b7336496488e5401b4ac91191 (table default.Table_16c91b8d55be449899b5e3a1884d81b4 [id=72cd050d1582435e8a7e8c101df3e117]) successfully deleted
I20260925 13:56:27.244346 19955 tablet_replica.cc:333] T c1f15b9de572479aae6c2e644da3392a P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:27.244525 20083 webserver.cc:533] Webserver started at http://127.18.182.252:42511/ using document root <none> and password file <none>
I20260925 13:56:27.246043 20083 fs_manager.cc:362] Metadata directory not provided
I20260925 13:56:27.246563 20083 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
W20260925 13:56:27.261493 19629 consensus_peers.cc:597] T 9059655f2c3d4e86b3b389b5b9a68891 P 7c1b52895a14482fb54a6758f5054f8f -> Peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Couldn't send request to peer 8f578869bc084a5d8a9a69f143954eb3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:27.274237 19938 tablet_bootstrap.cc:654] T 78d8c01c88d24deeae21d647e7f205ba P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:27.279968 20604 raft_consensus.cc:493] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:27.280431 20604 raft_consensus.cc:515] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:27.281327 20571 raft_consensus.cc:493] T 0e8aa12e95dc434a98a426499593d9a5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:27.281777 20571 raft_consensus.cc:515] T 0e8aa12e95dc434a98a426499593d9a5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:27.282353 20604 leader_election.cc:290] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:27.283705 20571 leader_election.cc:290] T 0e8aa12e95dc434a98a426499593d9a5 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:27.284500 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8aa12e95dc434a98a426499593d9a5" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:27.285064 19692 raft_consensus.cc:2468] T 0e8aa12e95dc434a98a426499593d9a5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 0.
I20260925 13:56:27.285162 19836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8aa12e95dc434a98a426499593d9a5" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:27.285706 19836 raft_consensus.cc:2468] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:27.286923 19773 leader_election.cc:304] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:27.287746 20604 raft_consensus.cc:2804] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:27.288062 20604 raft_consensus.cc:493] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:27.288381 20604 raft_consensus.cc:3060] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:27.297436 19631 leader_election.cc:304] T 0e8aa12e95dc434a98a426499593d9a5 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:27.299268 20571 raft_consensus.cc:2804] T 0e8aa12e95dc434a98a426499593d9a5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:27.299610 20571 raft_consensus.cc:493] T 0e8aa12e95dc434a98a426499593d9a5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:27.299933 20571 raft_consensus.cc:3060] T 0e8aa12e95dc434a98a426499593d9a5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:27.301476 19938 tablet_bootstrap.cc:492] T 78d8c01c88d24deeae21d647e7f205ba P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:27.301853 19938 ts_tablet_manager.cc:1403] T 78d8c01c88d24deeae21d647e7f205ba P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.059s user 0.006s sys 0.005s
I20260925 13:56:27.304175 19938 raft_consensus.cc:359] T 78d8c01c88d24deeae21d647e7f205ba P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:27.304771 19938 raft_consensus.cc:385] T 78d8c01c88d24deeae21d647e7f205ba P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:27.314838 20083 fs_manager.cc:714] Time spent opening directory manager: real 0.057s user 0.004s sys 0.004s
I20260925 13:56:27.316913 19942 tablet_bootstrap.cc:654] T 953d41f26f5f4027acfd374c6ec4ee75 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:27.305032 19938 raft_consensus.cc:740] T 78d8c01c88d24deeae21d647e7f205ba P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:27.319279 19938 consensus_queue.cc:260] T 78d8c01c88d24deeae21d647e7f205ba P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:27.319653 20604 raft_consensus.cc:515] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:27.327713 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8aa12e95dc434a98a426499593d9a5" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:27.331238 19938 ts_tablet_manager.cc:1434] T 78d8c01c88d24deeae21d647e7f205ba P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.029s user 0.005s sys 0.000s
I20260925 13:56:27.332218 19938 tablet_bootstrap.cc:492] T b23948807b2c4df9bf9a4da5f767e3b5 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:27.336421 19276 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet f01fc02d12c8461f96ef0a93412ac641
I20260925 13:56:27.337203 19938 tablet_bootstrap.cc:654] T b23948807b2c4df9bf9a4da5f767e3b5 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:27.337744 19276 catalog_manager.cc:6934] ReplaceTablet: tablet f01fc02d12c8461f96ef0a93412ac641 deleted and replaced by tablet 765b022051e84f5487b389c3ff29d0e5
I20260925 13:56:27.340620 19812 tablet_service.cc:1558] Processing DeleteTablet for tablet f01fc02d12c8461f96ef0a93412ac641 with delete_type TABLET_DATA_DELETED (replaced by tablet 765b022051e84f5487b389c3ff29d0e5) from {username='slave'} at 127.0.0.1:51442
W20260925 13:56:27.342365 19266 catalog_manager.cc:5043] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): delete failed for tablet f01fc02d12c8461f96ef0a93412ac641 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f01fc02d12c8461f96ef0a93412ac641 already in progress: creating tablet
I20260925 13:56:27.342998 20604 leader_election.cc:290] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:27.344584 19667 tablet_service.cc:1558] Processing DeleteTablet for tablet f01fc02d12c8461f96ef0a93412ac641 with delete_type TABLET_DATA_DELETED (replaced by tablet 765b022051e84f5487b389c3ff29d0e5) from {username='slave'} at 127.0.0.1:45948
W20260925 13:56:27.346226 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet f01fc02d12c8461f96ef0a93412ac641 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f01fc02d12c8461f96ef0a93412ac641 already in progress: creating tablet
I20260925 13:56:27.356137 19271 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet eb794cef052b43da831531455e2b5f15
I20260925 13:56:27.357492 19271 catalog_manager.cc:6934] ReplaceTablet: tablet eb794cef052b43da831531455e2b5f15 deleted and replaced by tablet 3d754e17930549189d65becb98afe2a6
I20260925 13:56:27.358103 19667 tablet_service.cc:1558] Processing DeleteTablet for tablet eb794cef052b43da831531455e2b5f15 with delete_type TABLET_DATA_DELETED (replaced by tablet 3d754e17930549189d65becb98afe2a6) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:27.358865 20011 tablet_replica.cc:333] T eb794cef052b43da831531455e2b5f15 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:27.359628 20011 raft_consensus.cc:2243] T eb794cef052b43da831531455e2b5f15 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:27.360272 20011 pending_rounds.cc:70] T eb794cef052b43da831531455e2b5f15 P 7c1b52895a14482fb54a6758f5054f8f: Trying to abort 1 pending ops.
I20260925 13:56:27.360471 20011 pending_rounds.cc:77] T eb794cef052b43da831531455e2b5f15 P 7c1b52895a14482fb54a6758f5054f8f: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251427288309760 op_type: NO_OP noop_request { }
I20260925 13:56:27.360795 20011 raft_consensus.cc:2889] T eb794cef052b43da831531455e2b5f15 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:56:27.360728 19812 tablet_service.cc:1558] Processing DeleteTablet for tablet eb794cef052b43da831531455e2b5f15 with delete_type TABLET_DATA_DELETED (replaced by tablet 3d754e17930549189d65becb98afe2a6) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:27.361088 20011 raft_consensus.cc:2272] T eb794cef052b43da831531455e2b5f15 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:27.361761 20644 tablet_replica.cc:333] T eb794cef052b43da831531455e2b5f15 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:27.362618 20644 raft_consensus.cc:2243] T eb794cef052b43da831531455e2b5f15 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:27.363059 20644 raft_consensus.cc:2272] T eb794cef052b43da831531455e2b5f15 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:27.365483 20644 ts_tablet_manager.cc:1916] T eb794cef052b43da831531455e2b5f15 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:27.367666 19273 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet f50ea68e3b3c4459b3f5e5b0df724724 (table default.Table_01f0109a149148dfaf1974cee0c3e42b [id=f1353f53d28748c8a8a4a2c6259581bf]). Expected version 3 got 0
I20260925 13:56:27.368847 19667 tablet_service.cc:1511] Processing CreateTablet for tablet 7d55b56fa80143248f4ccc14898381b6 (DEFAULT_TABLE table=default.Table_6800d8124538458699f660740653e710 [id=6fd743f0cf5a430fb7b37f6d8d12189c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:27.368510 19273 catalog_manager.cc:5838] T f16050752ad44720ba8703ef10bbb359 P 7c1b52895a14482fb54a6758f5054f8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194). New cstate: current_term: 1 leader_uuid: "7c1b52895a14482fb54a6758f5054f8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:27.370091 19667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d55b56fa80143248f4ccc14898381b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:27.369799 19273 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet f16050752ad44720ba8703ef10bbb359 (table default.Table_1da8b04f68ab4682a8ec32ad47854930 [id=b954461264164e969a40af810627fcb2]). Expected version 1 got 0
I20260925 13:56:27.369935 19812 tablet_service.cc:1511] Processing CreateTablet for tablet 7d55b56fa80143248f4ccc14898381b6 (DEFAULT_TABLE table=default.Table_6800d8124538458699f660740653e710 [id=6fd743f0cf5a430fb7b37f6d8d12189c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:27.370971 19812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d55b56fa80143248f4ccc14898381b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:27.370747 19273 catalog_manager.cc:5838] T 9059655f2c3d4e86b3b389b5b9a68891 P 7c1b52895a14482fb54a6758f5054f8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194). New cstate: current_term: 1 leader_uuid: "7c1b52895a14482fb54a6758f5054f8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:27.374953 19273 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 9059655f2c3d4e86b3b389b5b9a68891 (table default.Table_1b1f36048d2a4a3c86e513e46bcf5471 [id=0c13bfd171f2464b887f15d085439fa3]). Expected version 1 got 0
I20260925 13:56:27.375931 19273 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet e3cf9676160c4af08f1f62ae952b4c57 (table default.Table_4d6d747ea4034b3ab48710e10ff4544f [id=d6e6dd0772bf4901b696e11b10e41a48]). Expected version 1 got 0
I20260925 13:56:27.376739 20011 ts_tablet_manager.cc:1916] T eb794cef052b43da831531455e2b5f15 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:27.249715 19551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9b3dfaa7474611a1824d63eb906b43" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:27.320516 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8aa12e95dc434a98a426499593d9a5" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:27.324394 19547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8aa12e95dc434a98a426499593d9a5" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:27.341420 19522 tablet_service.cc:1558] Processing DeleteTablet for tablet f01fc02d12c8461f96ef0a93412ac641 with delete_type TABLET_DATA_DELETED (replaced by tablet 765b022051e84f5487b389c3ff29d0e5) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:27.354074 19549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8aa12e95dc434a98a426499593d9a5" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:27.357915 19525 tablet_service.cc:1558] Processing DeleteTablet for tablet eb794cef052b43da831531455e2b5f15 with delete_type TABLET_DATA_DELETED (replaced by tablet 3d754e17930549189d65becb98afe2a6) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:27.366973 19524 tablet_service.cc:1511] Processing CreateTablet for tablet 7d55b56fa80143248f4ccc14898381b6 (DEFAULT_TABLE table=default.Table_6800d8124538458699f660740653e710 [id=6fd743f0cf5a430fb7b37f6d8d12189c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:27.392131 20685 raft_consensus.cc:493] T 2711e7a6973546b1822d93f91095dae2 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:27.392305 19524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d55b56fa80143248f4ccc14898381b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:27.392532 20685 raft_consensus.cc:515] T 2711e7a6973546b1822d93f91095dae2 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:27.398088 19631 leader_election.cc:343] T 2711e7a6973546b1822d93f91095dae2 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 2711e7a6973546b1822d93f91095dae2
W20260925 13:56:27.403295 19629 leader_election.cc:343] T 0e8aa12e95dc434a98a426499593d9a5 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:27.404809 19773 leader_election.cc:343] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:27.406031 19629 leader_election.cc:343] T 5a9b3dfaa7474611a1824d63eb906b43 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:27.406369 19629 leader_election.cc:304] T 5a9b3dfaa7474611a1824d63eb906b43 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:27.407310 20625 raft_consensus.cc:2749] T 5a9b3dfaa7474611a1824d63eb906b43 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:27.408986 20686 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:56:27.410552 20083 fs_manager.cc:730] Time spent opening block manager: real 0.092s user 0.014s sys 0.011s
I20260925 13:56:27.411041 20083 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-2/data,/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-2/wal
uuid: "fc3953a2ba984a658630c423ec9b9728"
format_stamp: "Formatted at 2026-09-25 13:55:59 on dist-test-slave-nlqf"
I20260925 13:56:27.415004 20083 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260925 13:56:27.415793 19773 leader_election.cc:343] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:27.416724 19279 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 9059655f2c3d4e86b3b389b5b9a68891 (table default.Table_1b1f36048d2a4a3c86e513e46bcf5471 [id=0c13bfd171f2464b887f15d085439fa3]). Expected version 1 got 0
I20260925 13:56:27.417080 19549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2711e7a6973546b1822d93f91095dae2" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:27.417976 19279 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet f16050752ad44720ba8703ef10bbb359 (table default.Table_1da8b04f68ab4682a8ec32ad47854930 [id=b954461264164e969a40af810627fcb2]). Expected version 1 got 0
I20260925 13:56:27.417891 20685 leader_election.cc:290] T 2711e7a6973546b1822d93f91095dae2 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:27.418325 19955 raft_consensus.cc:2243] T c1f15b9de572479aae6c2e644da3392a P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Raft consensus shutting down.
W20260925 13:56:27.419049 19629 leader_election.cc:343] T 2711e7a6973546b1822d93f91095dae2 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:27.419190 19955 raft_consensus.cc:2272] T c1f15b9de572479aae6c2e644da3392a P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:27.419167 19279 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 000aad39953c4688b543c7d882a08ec8 (table default.Table_3a3e48a0641a43a1a783c6ba6374a708 [id=e64fe44a237f48859a288a87a565ee5e]). Expected version 1 got 0
I20260925 13:56:27.419443 19629 leader_election.cc:304] T 2711e7a6973546b1822d93f91095dae2 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:27.420739 20685 raft_consensus.cc:2749] T 2711e7a6973546b1822d93f91095dae2 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:27.448966 20571 raft_consensus.cc:515] T 0e8aa12e95dc434a98a426499593d9a5 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:27.456257 19525 tablet_service.cc:1511] Processing CreateTablet for tablet 80be2ffb278e4d649e5e2e4c8e2328f3 (DEFAULT_TABLE table=default.Table_7e877880587848638cd8f4b2a05e0723 [id=9ac029abcdf74f83a490e110c34a8675]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:27.457127 19276 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet e3cf9676160c4af08f1f62ae952b4c57 (table default.Table_4d6d747ea4034b3ab48710e10ff4544f [id=d6e6dd0772bf4901b696e11b10e41a48]). Expected version 1 got 0
I20260925 13:56:27.457338 19525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80be2ffb278e4d649e5e2e4c8e2328f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:27.457962 19276 catalog_manager.cc:5838] T 3aaf29a2310a41d0971eded7ef51fdc9 P 8f578869bc084a5d8a9a69f143954eb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "8f578869bc084a5d8a9a69f143954eb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:27.459069 19276 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet 3aaf29a2310a41d0971eded7ef51fdc9 (table default.Table_95a1130b07a34c82a6a6d7bb1c8245c8 [id=f4beec04dea942f98007b1f4cf33c9ec]). Expected version 1 got 0
I20260925 13:56:27.452137 19676 tablet_service.cc:1511] Processing CreateTablet for tablet 80be2ffb278e4d649e5e2e4c8e2328f3 (DEFAULT_TABLE table=default.Table_7e877880587848638cd8f4b2a05e0723 [id=9ac029abcdf74f83a490e110c34a8675]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:27.471065 19676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80be2ffb278e4d649e5e2e4c8e2328f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:27.487792 19692 raft_consensus.cc:2393] T 0e8aa12e95dc434a98a426499593d9a5 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f790493ba7ad403993b678f2df2b55be in current term 1: Already voted for candidate 7c1b52895a14482fb54a6758f5054f8f in this term.
I20260925 13:56:27.487747 19276 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet f16050752ad44720ba8703ef10bbb359 (table default.Table_1da8b04f68ab4682a8ec32ad47854930 [id=b954461264164e969a40af810627fcb2]). Expected version 1 got 0
I20260925 13:56:27.469409 20011 ts_tablet_manager.cc:1929] T eb794cef052b43da831531455e2b5f15 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:27.489555 20011 log.cc:1199] T eb794cef052b43da831531455e2b5f15 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/eb794cef052b43da831531455e2b5f15
I20260925 13:56:27.489645 19773 leader_election.cc:304] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:27.490653 20604 raft_consensus.cc:2749] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260925 13:56:27.491025 20011 ts_tablet_manager.cc:1950] T eb794cef052b43da831531455e2b5f15 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:27.490660 20644 ts_tablet_manager.cc:1929] T eb794cef052b43da831531455e2b5f15 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:27.493008 20571 leader_election.cc:290] T 0e8aa12e95dc434a98a426499593d9a5 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:27.493208 19549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8aa12e95dc434a98a426499593d9a5" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:27.493963 19838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e8aa12e95dc434a98a426499593d9a5" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:27.494580 19838 raft_consensus.cc:2393] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7c1b52895a14482fb54a6758f5054f8f in current term 1: Already voted for candidate f790493ba7ad403993b678f2df2b55be in this term.
I20260925 13:56:27.494926 19939 tablet_bootstrap.cc:492] T 2711e7a6973546b1822d93f91095dae2 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
W20260925 13:56:27.495450 19629 leader_election.cc:343] T 0e8aa12e95dc434a98a426499593d9a5 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:27.495908 19631 leader_election.cc:304] T 0e8aa12e95dc434a98a426499593d9a5 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:27.496778 20644 log.cc:1199] T eb794cef052b43da831531455e2b5f15 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/eb794cef052b43da831531455e2b5f15
I20260925 13:56:27.496069 19939 ts_tablet_manager.cc:1403] T 2711e7a6973546b1822d93f91095dae2 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.325s user 0.010s sys 0.000s
I20260925 13:56:27.497440 20644 ts_tablet_manager.cc:1950] T eb794cef052b43da831531455e2b5f15 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:27.499825 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet eb794cef052b43da831531455e2b5f15 (table default.Table_c389f408e051410dac7a521879c22729 [id=d2bc135460ab4f2fad4740a281bfc6e2]) successfully deleted
I20260925 13:56:27.501628 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet eb794cef052b43da831531455e2b5f15 (table default.Table_c389f408e051410dac7a521879c22729 [id=d2bc135460ab4f2fad4740a281bfc6e2]) successfully deleted
I20260925 13:56:27.502446 19938 tablet_bootstrap.cc:492] T b23948807b2c4df9bf9a4da5f767e3b5 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:27.502862 19938 ts_tablet_manager.cc:1403] T b23948807b2c4df9bf9a4da5f767e3b5 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.171s user 0.015s sys 0.034s
I20260925 13:56:27.505947 19938 raft_consensus.cc:359] T b23948807b2c4df9bf9a4da5f767e3b5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:27.506572 19938 raft_consensus.cc:385] T b23948807b2c4df9bf9a4da5f767e3b5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:27.508508 19938 raft_consensus.cc:740] T b23948807b2c4df9bf9a4da5f767e3b5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:27.509156 19938 consensus_queue.cc:260] T b23948807b2c4df9bf9a4da5f767e3b5 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:27.510123 19939 raft_consensus.cc:359] T 2711e7a6973546b1822d93f91095dae2 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:27.510739 19939 raft_consensus.cc:385] T 2711e7a6973546b1822d93f91095dae2 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:27.510985 19939 raft_consensus.cc:740] T 2711e7a6973546b1822d93f91095dae2 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:27.514444 20571 raft_consensus.cc:2749] T 0e8aa12e95dc434a98a426499593d9a5 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260925 13:56:27.516152 19955 ts_tablet_manager.cc:1916] T c1f15b9de572479aae6c2e644da3392a P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:27.527036 19939 consensus_queue.cc:260] T 2711e7a6973546b1822d93f91095dae2 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:27.538683 19939 ts_tablet_manager.cc:1434] T 2711e7a6973546b1822d93f91095dae2 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.041s user 0.004s sys 0.000s
I20260925 13:56:27.541534 19812 tablet_service.cc:1511] Processing CreateTablet for tablet 80be2ffb278e4d649e5e2e4c8e2328f3 (DEFAULT_TABLE table=default.Table_7e877880587848638cd8f4b2a05e0723 [id=9ac029abcdf74f83a490e110c34a8675]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:27.542573 19812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80be2ffb278e4d649e5e2e4c8e2328f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:27.544551 19271 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet e3cf9676160c4af08f1f62ae952b4c57 (table default.Table_4d6d747ea4034b3ab48710e10ff4544f [id=d6e6dd0772bf4901b696e11b10e41a48]). Expected version 1 got 0
I20260925 13:56:27.545318 19271 catalog_manager.cc:5838] T 1215d875209a40d38cc527e2795d6c75 P 7c1b52895a14482fb54a6758f5054f8f reported cstate change: term changed from 0 to 1, leader changed from <none> to f790493ba7ad403993b678f2df2b55be (127.18.182.195). New cstate: current_term: 1 leader_uuid: "f790493ba7ad403993b678f2df2b55be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } }
I20260925 13:56:27.546581 19271 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 3aaf29a2310a41d0971eded7ef51fdc9 (table default.Table_95a1130b07a34c82a6a6d7bb1c8245c8 [id=f4beec04dea942f98007b1f4cf33c9ec]). Expected version 1 got 0
I20260925 13:56:27.547385 19271 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet f16050752ad44720ba8703ef10bbb359 (table default.Table_1da8b04f68ab4682a8ec32ad47854930 [id=b954461264164e969a40af810627fcb2]). Expected version 1 got 0
I20260925 13:56:27.548602 19271 catalog_manager.cc:5838] T f50ea68e3b3c4459b3f5e5b0df724724 P 7c1b52895a14482fb54a6758f5054f8f reported cstate change: term changed from 0 to 1, leader changed from <none> to f790493ba7ad403993b678f2df2b55be (127.18.182.195). New cstate: current_term: 1 leader_uuid: "f790493ba7ad403993b678f2df2b55be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } }
I20260925 13:56:27.553207 19271 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet f50ea68e3b3c4459b3f5e5b0df724724 (table default.Table_01f0109a149148dfaf1974cee0c3e42b [id=f1353f53d28748c8a8a4a2c6259581bf]). Expected version 3 got 0
I20260925 13:56:27.553869 19271 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 9059655f2c3d4e86b3b389b5b9a68891 (table default.Table_1b1f36048d2a4a3c86e513e46bcf5471 [id=0c13bfd171f2464b887f15d085439fa3]). Expected version 1 got 0
I20260925 13:56:27.554771 19271 catalog_manager.cc:5838] T a8a679a453e0430b866edc707fe7c234 P 7c1b52895a14482fb54a6758f5054f8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194). New cstate: current_term: 1 leader_uuid: "7c1b52895a14482fb54a6758f5054f8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:27.561374 19819 tablet_service.cc:1511] Processing CreateTablet for tablet 120825852c314caf95e67c1cbb95154d (DEFAULT_TABLE table=default.Table_b8c44d75312d4cf48bedf12f8f6a948f [id=3880006011944a55966280e067b54a6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:27.562351 19819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 120825852c314caf95e67c1cbb95154d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:27.567592 20604 raft_consensus.cc:493] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:27.567991 20604 raft_consensus.cc:515] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:27.569697 20604 leader_election.cc:290] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:27.572224 19804 tablet_service.cc:1511] Processing CreateTablet for tablet ef5220aead1f43e98cbaef791d749c60 (DEFAULT_TABLE table=default.Table_f20a7a770a834948b4fb54d6d19ae070 [id=2200d43a3c7547958fc856b9af4d9852]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:27.573251 19804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef5220aead1f43e98cbaef791d749c60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:27.581627 19818 tablet_service.cc:1511] Processing CreateTablet for tablet 8b858870f1254e72ae2af17c8192ffa0 (DEFAULT_TABLE table=default.Table_1de14a8309044a888188fc1c228700fe [id=2071747026054f07846f214fc67275d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:27.582485 19818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b858870f1254e72ae2af17c8192ffa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:27.596675 19272 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 5f8a96ae6eec4f15a221f9aad537c094 (table default.Table_3a06919b658e46f893318b4e1ec8b52f [id=e6f2a872536b4e55b071231020903581]). Expected version 1 got 0
I20260925 13:56:27.597466 19272 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 000aad39953c4688b543c7d882a08ec8 (table default.Table_3a3e48a0641a43a1a783c6ba6374a708 [id=e64fe44a237f48859a288a87a565ee5e]). Expected version 1 got 0
I20260925 13:56:27.598058 19272 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet f16050752ad44720ba8703ef10bbb359 (table default.Table_1da8b04f68ab4682a8ec32ad47854930 [id=b954461264164e969a40af810627fcb2]). Expected version 1 got 0
I20260925 13:56:27.599382 19272 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet e3cf9676160c4af08f1f62ae952b4c57 (table default.Table_4d6d747ea4034b3ab48710e10ff4544f [id=d6e6dd0772bf4901b696e11b10e41a48]). Expected version 1 got 0
I20260925 13:56:27.600219 19272 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet f50ea68e3b3c4459b3f5e5b0df724724 (table default.Table_01f0109a149148dfaf1974cee0c3e42b [id=f1353f53d28748c8a8a4a2c6259581bf]). Expected version 3 got 0
I20260925 13:56:27.600903 19272 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 9059655f2c3d4e86b3b389b5b9a68891 (table default.Table_1b1f36048d2a4a3c86e513e46bcf5471 [id=0c13bfd171f2464b887f15d085439fa3]). Expected version 1 got 0
I20260925 13:56:27.601419 19272 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 82972fa4868d4c59b3b9bb1240300938 (table default.Table_d3d2076fbb6241a68f04e83462a34182 [id=4232cecee9104120ac2521eb250bbd0d]). Expected version 1 got 0
I20260925 13:56:27.609930 19667 tablet_service.cc:1511] Processing CreateTablet for tablet 120825852c314caf95e67c1cbb95154d (DEFAULT_TABLE table=default.Table_b8c44d75312d4cf48bedf12f8f6a948f [id=3880006011944a55966280e067b54a6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:27.613637 19805 tablet_service.cc:1511] Processing CreateTablet for tablet 243b9d20890c4ef79e39956239e48c3c (DEFAULT_TABLE table=default.Table_660b4f2919ba4fd4916ff027d5e01ac9 [id=d611df71f13e49989d0711220d9db00c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:27.613974 19277 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 0e8aa12e95dc434a98a426499593d9a5
I20260925 13:56:27.614684 19805 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 243b9d20890c4ef79e39956239e48c3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:27.615494 19277 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:39288 (request call id 270) took 2307 ms. Trace:
I20260925 13:56:27.615317 19275 catalog_manager.cc:6391] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: default.Table_d0dadd9b31b143fd8dd5adcb7f752c5f [id=beee0ac537e44859b0f7c76d6808b0e0] alter complete (version 1)
I20260925 13:56:27.620843 19801 tablet_service.cc:1511] Processing CreateTablet for tablet 77a401585d3f4c28ac75b9096c9c378f (DEFAULT_TABLE table=default.Table_353d24d106dc42b7b8085d28496b3058 [id=d800a775a0e740799f40625659333f7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:27.621698 19817 tablet_service.cc:1558] Processing DeleteTablet for tablet 0e8aa12e95dc434a98a426499593d9a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:25 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:27.621912 19801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77a401585d3f4c28ac75b9096c9c378f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:27.615840 19277 rpcz_store.cc:276] 0925 13:56:25.308579 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:25.737792 (+429213us) service_pool.cc:224] Handling call
0925 13:56:25.812821 (+ 75029us) spinlock_profiling.cc:243] Waited 73.5 ms on lock 0x7b5400040454. stack: 00007fdf0a30b323 00007fdf0a30b220 00007fdf09eaf574 0000000000516964 00000000005168f9 00007fdf114095c0 00007fdf114a86fa 00007fdf13219d8c 00007fdf13319876 00007fdf10ec8b34 00007fdf10ec8ac2 00007fdf10ec89f1 00007fdf10ec896c 00007fdf10ec7c22 00007fdf0f16e09c 00007fdf0f16d4d6
0925 13:56:25.813900 (+ 1079us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 13:56:25.815122 (+ 1222us) catalog_manager.cc:2854] Modifying in-memory table state
0925 13:56:25.815231 (+ 109us) catalog_manager.cc:2863] Locking tablets
0925 13:56:27.264745 (+1449514us) catalog_manager.cc:2877] Removing table and tablets from system table
0925 13:56:27.288217 (+ 23472us) catalog_manager.cc:2895] Removing table from by-name map
0925 13:56:27.288425 (+ 208us) catalog_manager.cc:2911] Committing in-memory state
0925 13:56:27.613795 (+325370us) catalog_manager.cc:2915] Aborting table tasks
0925 13:56:27.615429 (+ 1634us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:27.266691 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:27.267067 (+ 376us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:27.267088 (+ 21us) write_op.cc:435] Acquired schema lock
0925 13:56:27.267100 (+ 12us) tablet.cc:662] Decoding operations
0925 13:56:27.267430 (+ 330us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:27.267470 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:27.267492 (+ 22us) tablet.cc:685] Acquiring locks for 2 operations
0925 13:56:27.267750 (+ 258us) tablet.cc:701] Row locks acquired
0925 13:56:27.267763 (+ 13us) write_op.cc:260] PREPARE: finished
0925 13:56:27.267868 (+ 105us) write_op.cc:270] Start()
0925 13:56:27.267945 (+ 77us) write_op.cc:276] Timestamp: P: 1790344587267833 usec, L: 0
0925 13:56:27.267960 (+ 15us) op_driver.cc:348] REPLICATION: starting
0925 13:56:27.268412 (+ 452us) log.cc:844] Serialized 734 byte log entry
0925 13:56:27.276337 (+ 7925us) op_driver.cc:464] REPLICATION: finished
0925 13:56:27.286294 (+ 9957us) write_op.cc:301] APPLY: starting
0925 13:56:27.286356 (+ 62us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:27.286468 (+ 112us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:27.286641 (+ 173us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:27.286889 (+ 248us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:27.287031 (+ 142us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 13:56:27.287077 (+ 46us) write_op.cc:312] APPLY: finished
0925 13:56:27.287424 (+ 347us) log.cc:844] Serialized 29 byte log entry
0925 13:56:27.287571 (+ 147us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:27.287642 (+ 71us) write_op.cc:454] Released schema lock
0925 13:56:27.287680 (+ 38us) write_op.cc:341] FINISH: Updating metrics
0925 13:56:27.287903 (+ 223us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:27.287927 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":168938368,"child_traces":[["op",{"apply.queue_time_us":9785,"apply.run_cpu_time_us":1449,"apply.run_wall_time_us":1583,"num_ops":2,"prepare.queue_time_us":232,"prepare.run_cpu_time_us":2023,"prepare.run_wall_time_us":2023,"raft.queue_time_us":473,"raft.run_cpu_time_us":676,"raft.run_wall_time_us":675,"replication_time_us":8145}]]}
I20260925 13:56:27.623629 20644 tablet_replica.cc:333] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:27.624089 19813 tablet_service.cc:1511] Processing CreateTablet for tablet d0caac86b259433e9c490aa71ac9d329 (DEFAULT_TABLE table=default.Table_e0f7b807e8884431898298a1c635c784 [id=e49e015b173c48c1b6ef8112f60b33e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:27.624058 19277 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_732ec1f43dec4e6b8cd73691cdd91358"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:27.624436 20644 raft_consensus.cc:2243] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:27.624889 20644 raft_consensus.cc:2272] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:27.625103 19813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0caac86b259433e9c490aa71ac9d329. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:27.626631 20644 ts_tablet_manager.cc:1916] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 13:56:27.630039 19279 catalog_manager.cc:6038] Failed to send AlterTable RPC for tablet 000aad39953c4688b543c7d882a08ec8 (table default.Table_3a3e48a0641a43a1a783c6ba6374a708 [id=e64fe44a237f48859a288a87a565ee5e], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 000aad39953c4688b543c7d882a08ec8 consensus state has no leader
W20260925 13:56:27.630383 19277 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_732ec1f43dec4e6b8cd73691cdd91358 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 13:56:27.630892 19279 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.195:57799 (request call id 36) took 2140 ms (client timeout 1000 ms). Trace:
I20260925 13:56:27.636873 19803 tablet_service.cc:1558] Processing DeleteTablet for tablet 953d41f26f5f4027acfd374c6ec4ee75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:22 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:27.637511 19817 tablet_service.cc:1558] Processing DeleteTablet for tablet 908df914cfec490cb7058dbe7b501c97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:22 UTC) from {username='slave'} at 127.0.0.1:51442
W20260925 13:56:27.640651 19280 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.194:34125 (request call id 39) took 3615 ms (client timeout 1000 ms). Trace:
I20260925 13:56:27.610999 19667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 120825852c314caf95e67c1cbb95154d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:27.640950 19280 rpcz_store.cc:269] 0925 13:56:24.008268 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:24.008405 (+ 137us) service_pool.cc:224] Handling call
0925 13:56:27.623222 (+3614817us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:24.952006 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:24.952458 (+ 452us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:24.952491 (+ 33us) write_op.cc:435] Acquired schema lock
0925 13:56:24.952510 (+ 19us) tablet.cc:662] Decoding operations
0925 13:56:24.952794 (+ 284us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:24.952838 (+ 44us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:24.952870 (+ 32us) tablet.cc:685] Acquiring locks for 1 operations
0925 13:56:24.957347 (+ 4477us) tablet.cc:701] Row locks acquired
0925 13:56:24.957359 (+ 12us) write_op.cc:260] PREPARE: finished
0925 13:56:24.957521 (+ 162us) write_op.cc:270] Start()
0925 13:56:24.957599 (+ 78us) write_op.cc:276] Timestamp: P: 1790344584957496 usec, L: 0
0925 13:56:24.957614 (+ 15us) op_driver.cc:348] REPLICATION: starting
0925 13:56:24.958040 (+ 426us) log.cc:844] Serialized 356 byte log entry
0925 13:56:24.995695 (+ 37655us) op_driver.cc:464] REPLICATION: finished
0925 13:56:24.996093 (+ 398us) write_op.cc:301] APPLY: starting
0925 13:56:24.996146 (+ 53us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:24.996240 (+ 94us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:24.996352 (+ 112us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:24.996488 (+ 136us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:24.996610 (+ 122us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 13:56:24.996650 (+ 40us) write_op.cc:312] APPLY: finished
0925 13:56:24.996798 (+ 148us) log.cc:844] Serialized 23 byte log entry
0925 13:56:24.996921 (+ 123us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:24.996979 (+ 58us) write_op.cc:454] Released schema lock
0925 13:56:24.997010 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0925 13:56:24.997235 (+ 225us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:24.997267 (+ 32us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":255,"apply.run_cpu_time_us":1095,"apply.run_wall_time_us":1095,"num_ops":1,"prepare.queue_time_us":34703,"prepare.run_cpu_time_us":1856,"prepare.run_wall_time_us":6228,"replication_time_us":37994}]]}
I20260925 13:56:27.644034 19671 tablet_service.cc:1511] Processing CreateTablet for tablet 8b858870f1254e72ae2af17c8192ffa0 (DEFAULT_TABLE table=default.Table_1de14a8309044a888188fc1c228700fe [id=2071747026054f07846f214fc67275d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:27.644918 19673 tablet_service.cc:1558] Processing DeleteTablet for tablet 0e8aa12e95dc434a98a426499593d9a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:25 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:27.645561 19666 tablet_service.cc:1558] Processing DeleteTablet for tablet ec7fe5c834be4a51b627b8b1c3ebe888 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:22 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:27.645731 19669 tablet_service.cc:1558] Processing DeleteTablet for tablet 2ef3efe644364e93a089ddeed328389d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:20 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:27.645898 19670 tablet_service.cc:1511] Processing CreateTablet for tablet 243b9d20890c4ef79e39956239e48c3c (DEFAULT_TABLE table=default.Table_660b4f2919ba4fd4916ff027d5e01ac9 [id=d611df71f13e49989d0711220d9db00c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:27.645928 19280 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_03de3c9f9f844ca98b834d7f6a809daf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:27.646826 19672 tablet_service.cc:1511] Processing CreateTablet for tablet d0caac86b259433e9c490aa71ac9d329 (DEFAULT_TABLE table=default.Table_e0f7b807e8884431898298a1c635c784 [id=e49e015b173c48c1b6ef8112f60b33e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:27.646955 19668 tablet_service.cc:1511] Processing CreateTablet for tablet ef5220aead1f43e98cbaef791d749c60 (DEFAULT_TABLE table=default.Table_f20a7a770a834948b4fb54d6d19ae070 [id=2200d43a3c7547958fc856b9af4d9852]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 13:56:27.647368 19280 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_03de3c9f9f844ca98b834d7f6a809daf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:27.647706 19675 tablet_service.cc:1558] Processing DeleteTablet for tablet 436b8d46e1d244deb8bb9dc128ee6279 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:23 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:27.647612 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d01a791445149e7bee9493e1eec5c6b" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:27.654527 19674 tablet_service.cc:1511] Processing CreateTablet for tablet 77a401585d3f4c28ac75b9096c9c378f (DEFAULT_TABLE table=default.Table_353d24d106dc42b7b8085d28496b3058 [id=d800a775a0e740799f40625659333f7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:27.660796 19671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b858870f1254e72ae2af17c8192ffa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:27.663823 20011 tablet_replica.cc:333] T 0e8aa12e95dc434a98a426499593d9a5 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:27.664565 20011 raft_consensus.cc:2243] T 0e8aa12e95dc434a98a426499593d9a5 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:27.664965 20011 raft_consensus.cc:2272] T 0e8aa12e95dc434a98a426499593d9a5 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:27.665656 19675 tablet_service.cc:1511] Processing CreateTablet for tablet 2b73f00e58c7444b90fd01b685769b2b (DEFAULT_TABLE table=default.Table_c77ea16965bc41eca6f65334f0219a09 [id=18c9f6da927641488972ad4cee22ad09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:27.666567 20011 ts_tablet_manager.cc:1916] T 0e8aa12e95dc434a98a426499593d9a5 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:27.666579 19675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b73f00e58c7444b90fd01b685769b2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:27.667819 19670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 243b9d20890c4ef79e39956239e48c3c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:27.631217 19279 rpcz_store.cc:269] 0925 13:56:25.490669 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:25.718113 (+227444us) service_pool.cc:224] Handling call
0925 13:56:25.819157 (+101044us) spinlock_profiling.cc:243] Waited 95 ms on lock 0x7b5400040454. stack: 00007fdf0a30b323 00007fdf0a30b220 00007fdf09eaf574 0000000000516964 00000000005168f9 00007fdf114095c0 00007fdf114a86fa 00007fdf13219d8c 00007fdf1331801e 00007fdf10ebcfb4 00007fdf10ebcf42 00007fdf10ebce71 00007fdf10ebcdec 00007fdf10ebc0a2 00007fdf0f16e09c 00007fdf0f16d4d6
0925 13:56:27.630810 (+1811653us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":12,"spinlock_wait_cycles":218551168}
I20260925 13:56:27.668926 19279 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_5b3429bafb2248f99555eb27a2780ff6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:27.670405 19279 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5b3429bafb2248f99555eb27a2780ff6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:27.671957 19672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0caac86b259433e9c490aa71ac9d329. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:27.675493 19668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef5220aead1f43e98cbaef791d749c60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:27.677134 19669 tablet_service.cc:1511] Processing CreateTablet for tablet 6a78ac7eb451455e9d2eb6f8cd798a77 (DEFAULT_TABLE table=default.Table_5564a650590f413580292ea7e803b417 [id=4185dfcad1f1499296231833f2fb8edf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:27.678043 19669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a78ac7eb451455e9d2eb6f8cd798a77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:27.679908 19674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77a401585d3f4c28ac75b9096c9c378f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:27.684100 19773 leader_election.cc:343] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:27.684473 19819 tablet_service.cc:1511] Processing CreateTablet for tablet 2b73f00e58c7444b90fd01b685769b2b (DEFAULT_TABLE table=default.Table_c77ea16965bc41eca6f65334f0219a09 [id=18c9f6da927641488972ad4cee22ad09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:27.685544 19819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b73f00e58c7444b90fd01b685769b2b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:27.685663 19267 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 69934b9a49354c92b5b13e182247ef77 (table default.Table_49155edad5db40549fa76e9195967efc [id=50c3843b240b47e1bfd1bfd07e3d4875], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 69934b9a49354c92b5b13e182247ef77 consensus state has no leader
I20260925 13:56:27.687088 19666 tablet_service.cc:1558] Processing DeleteTablet for tablet eb794cef052b43da831531455e2b5f15 with delete_type TABLET_DATA_DELETED (replaced by tablet 3d754e17930549189d65becb98afe2a6) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:27.693176 20604 raft_consensus.cc:493] T 78d8c01c88d24deeae21d647e7f205ba P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:27.693619 20604 raft_consensus.cc:515] T 78d8c01c88d24deeae21d647e7f205ba P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:27.699265 20604 leader_election.cc:290] T 78d8c01c88d24deeae21d647e7f205ba P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
W20260925 13:56:27.700181 19274 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.194:34125 (request call id 40) took 2636 ms (client timeout 1000 ms). Trace:
I20260925 13:56:27.705703 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d8c01c88d24deeae21d647e7f205ba" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:27.707168 19773 leader_election.cc:343] T 78d8c01c88d24deeae21d647e7f205ba P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:27.715831 19280 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_66af2a1d5acd40a4a7ed59932097b68e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:27.540652 20523 raft_consensus.cc:493] T 2711e7a6973546b1822d93f91095dae2 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:27.718032 20523 raft_consensus.cc:515] T 2711e7a6973546b1822d93f91095dae2 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:27.544332 19522 tablet_service.cc:1511] Processing CreateTablet for tablet 120825852c314caf95e67c1cbb95154d (DEFAULT_TABLE table=default.Table_b8c44d75312d4cf48bedf12f8f6a948f [id=3880006011944a55966280e067b54a6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:27.720077 19522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 120825852c314caf95e67c1cbb95154d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:27.721733 19666 tablet_service.cc:1511] Processing CreateTablet for tablet 236e308d8694405cb2b874164fa16485 (DEFAULT_TABLE table=default.Table_25c2c368343741a3923a23f0a7cb2b62 [id=7d4523c5927040ba846c35a4b02172a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:27.562343 19529 tablet_service.cc:1511] Processing CreateTablet for tablet ef5220aead1f43e98cbaef791d749c60 (DEFAULT_TABLE table=default.Table_f20a7a770a834948b4fb54d6d19ae070 [id=2200d43a3c7547958fc856b9af4d9852]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:27.724072 19529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef5220aead1f43e98cbaef791d749c60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:27.571799 19549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d01a791445149e7bee9493e1eec5c6b" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:27.723275 19666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 236e308d8694405cb2b874164fa16485. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:27.576985 19519 tablet_service.cc:1511] Processing CreateTablet for tablet 8b858870f1254e72ae2af17c8192ffa0 (DEFAULT_TABLE table=default.Table_1de14a8309044a888188fc1c228700fe [id=2071747026054f07846f214fc67275d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:27.728964 19519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b858870f1254e72ae2af17c8192ffa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:27.601727 19523 tablet_service.cc:1511] Processing CreateTablet for tablet 243b9d20890c4ef79e39956239e48c3c (DEFAULT_TABLE table=default.Table_660b4f2919ba4fd4916ff027d5e01ac9 [id=d611df71f13e49989d0711220d9db00c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:27.731086 19523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 243b9d20890c4ef79e39956239e48c3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:27.739595 20523 leader_election.cc:290] T 2711e7a6973546b1822d93f91095dae2 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
W20260925 13:56:27.745594 19490 leader_election.cc:343] T 2711e7a6973546b1822d93f91095dae2 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 2711e7a6973546b1822d93f91095dae2
I20260925 13:56:27.751547 19939 tablet_bootstrap.cc:492] T 953d41f26f5f4027acfd374c6ec4ee75 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
W20260925 13:56:27.687206 19273 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.194:34125 (request call id 41) took 1507 ms (client timeout 1000 ms). Trace:
I20260925 13:56:27.607432 19955 ts_tablet_manager.cc:1929] T c1f15b9de572479aae6c2e644da3392a P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:27.628516 20661 raft_consensus.cc:493] T 2ef3efe644364e93a089ddeed328389d P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:27.754525 20661 raft_consensus.cc:515] T 2ef3efe644364e93a089ddeed328389d P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:27.756367 20661 leader_election.cc:290] T 2ef3efe644364e93a089ddeed328389d P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:27.628974 19532 tablet_service.cc:1558] Processing DeleteTablet for tablet 0e8aa12e95dc434a98a426499593d9a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:25 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:27.629940 19518 tablet_service.cc:1511] Processing CreateTablet for tablet 77a401585d3f4c28ac75b9096c9c378f (DEFAULT_TABLE table=default.Table_353d24d106dc42b7b8085d28496b3058 [id=d800a775a0e740799f40625659333f7c]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 13:56:27.754078 19273 rpcz_store.cc:269] 0925 13:56:26.180348 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:26.192829 (+ 12481us) service_pool.cc:224] Handling call
0925 13:56:27.687137 (+1494308us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:27.387771 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:27.388269 (+ 498us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:27.388302 (+ 33us) write_op.cc:435] Acquired schema lock
0925 13:56:27.388321 (+ 19us) tablet.cc:662] Decoding operations
0925 13:56:27.388730 (+ 409us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:27.388781 (+ 51us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:27.388812 (+ 31us) tablet.cc:685] Acquiring locks for 2 operations
0925 13:56:27.389144 (+ 332us) tablet.cc:701] Row locks acquired
0925 13:56:27.389163 (+ 19us) write_op.cc:260] PREPARE: finished
0925 13:56:27.389289 (+ 126us) write_op.cc:270] Start()
0925 13:56:27.389387 (+ 98us) write_op.cc:276] Timestamp: P: 1790344587389248 usec, L: 0
0925 13:56:27.389409 (+ 22us) op_driver.cc:348] REPLICATION: starting
0925 13:56:27.389936 (+ 527us) log.cc:844] Serialized 919 byte log entry
0925 13:56:27.409402 (+ 19466us) op_driver.cc:464] REPLICATION: finished
0925 13:56:27.409650 (+ 248us) write_op.cc:301] APPLY: starting
0925 13:56:27.409710 (+ 60us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:27.409822 (+ 112us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:27.409902 (+ 80us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:27.410186 (+ 284us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:27.410311 (+ 125us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 13:56:27.410352 (+ 41us) write_op.cc:312] APPLY: finished
0925 13:56:27.410517 (+ 165us) log.cc:844] Serialized 29 byte log entry
0925 13:56:27.410681 (+ 164us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:27.410755 (+ 74us) write_op.cc:454] Released schema lock
0925 13:56:27.410789 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0925 13:56:27.411016 (+ 227us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:27.411040 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":57283,"reported_tablets":12,"child_traces":[["op",{"apply.queue_time_us":90,"apply.run_cpu_time_us":1331,"apply.run_wall_time_us":1331,"num_ops":2,"prepare.queue_time_us":121,"prepare.run_cpu_time_us":2478,"prepare.run_wall_time_us":2479,"raft.queue_time_us":2351,"raft.run_cpu_time_us":818,"raft.run_wall_time_us":819,"replication_time_us":19902}]]}
I20260925 13:56:27.757987 19518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77a401585d3f4c28ac75b9096c9c378f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:27.758114 19273 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_8aa1581769ff4b9bb2e158a3b0430a4f" } new_table_name: "default.Table_4f849dc343f5448eb14a195bb0777d12" modify_external_catalogs: true
W20260925 13:56:27.700562 19274 rpcz_store.cc:269] 0925 13:56:25.062845 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:25.113571 (+ 50726us) service_pool.cc:224] Handling call
0925 13:56:27.699599 (+2586028us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:26.070535 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:26.070919 (+ 384us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:26.070942 (+ 23us) write_op.cc:435] Acquired schema lock
0925 13:56:26.070957 (+ 15us) tablet.cc:662] Decoding operations
0925 13:56:26.084345 (+ 13388us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:26.084385 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:26.084410 (+ 25us) tablet.cc:685] Acquiring locks for 2 operations
0925 13:56:26.084691 (+ 281us) tablet.cc:701] Row locks acquired
0925 13:56:26.084704 (+ 13us) write_op.cc:260] PREPARE: finished
0925 13:56:26.084824 (+ 120us) write_op.cc:270] Start()
0925 13:56:26.084910 (+ 86us) write_op.cc:276] Timestamp: P: 1790344586084788 usec, L: 0
0925 13:56:26.084925 (+ 15us) op_driver.cc:348] REPLICATION: starting
0925 13:56:26.085415 (+ 490us) log.cc:844] Serialized 919 byte log entry
0925 13:56:26.313251 (+227836us) op_driver.cc:464] REPLICATION: finished
0925 13:56:26.323753 (+ 10502us) write_op.cc:301] APPLY: starting
0925 13:56:26.323820 (+ 67us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:26.323954 (+ 134us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:26.324093 (+ 139us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:26.324439 (+ 346us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:26.324581 (+ 142us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 13:56:26.324624 (+ 43us) write_op.cc:312] APPLY: finished
0925 13:56:26.324804 (+ 180us) log.cc:844] Serialized 29 byte log entry
0925 13:56:26.324961 (+ 157us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:26.325066 (+ 105us) write_op.cc:454] Released schema lock
0925 13:56:26.325099 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0925 13:56:26.325343 (+ 244us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:26.325368 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":44097,"reported_tablets":12,"child_traces":[["op",{"apply.queue_time_us":10325,"apply.run_cpu_time_us":1558,"apply.run_wall_time_us":1557,"num_ops":2,"prepare.queue_time_us":216612,"prepare.run_cpu_time_us":2223,"prepare.run_wall_time_us":20536,"raft.queue_time_us":1069,"raft.run_cpu_time_us":12,"raft.run_wall_time_us":12,"replication_time_us":228190}]]}
I20260925 13:56:27.760185 19274 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_d0dadd9b31b143fd8dd5adcb7f752c5f" } modify_external_catalogs: true
I20260925 13:56:27.760634 19274 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_d0dadd9b31b143fd8dd5adcb7f752c5f" } modify_external_catalogs: true
I20260925 13:56:27.761818 20571 raft_consensus.cc:493] T 0fdb45b31f93466c8bed1b848c6cdf15 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:27.762248 20571 raft_consensus.cc:515] T 0fdb45b31f93466c8bed1b848c6cdf15 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:27.631016 19526 tablet_service.cc:1511] Processing CreateTablet for tablet d0caac86b259433e9c490aa71ac9d329 (DEFAULT_TABLE table=default.Table_e0f7b807e8884431898298a1c635c784 [id=e49e015b173c48c1b6ef8112f60b33e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:27.764341 19526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0caac86b259433e9c490aa71ac9d329. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:27.764966 20571 leader_election.cc:290] T 0fdb45b31f93466c8bed1b848c6cdf15 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:27.767961 19673 tablet_service.cc:1558] Processing DeleteTablet for tablet 5a9b3dfaa7474611a1824d63eb906b43 with delete_type TABLET_DATA_DELETED (replaced by tablet a218f033a132482a82bd1f60126fa0f9) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:27.768718 19665 tablet_service.cc:1558] Processing DeleteTablet for tablet 908df914cfec490cb7058dbe7b501c97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:22 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:27.769608 19665 tablet_service.cc:1558] Processing DeleteTablet for tablet 2711e7a6973546b1822d93f91095dae2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:21 UTC) from {username='slave'} at 127.0.0.1:45948
W20260925 13:56:27.770046 19275 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.193:37907 (request call id 33) took 3164 ms (client timeout 1000 ms). Trace:
I20260925 13:56:27.673442 19533 tablet_service.cc:1511] Processing CreateTablet for tablet 2b73f00e58c7444b90fd01b685769b2b (DEFAULT_TABLE table=default.Table_c77ea16965bc41eca6f65334f0219a09 [id=18c9f6da927641488972ad4cee22ad09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:27.680402 19520 tablet_service.cc:1511] Processing CreateTablet for tablet 6a78ac7eb451455e9d2eb6f8cd798a77 (DEFAULT_TABLE table=default.Table_5564a650590f413580292ea7e803b417 [id=4185dfcad1f1499296231833f2fb8edf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:27.772142 19533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b73f00e58c7444b90fd01b685769b2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:27.773720 19549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fdb45b31f93466c8bed1b848c6cdf15" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
W20260925 13:56:27.774827 19629 leader_election.cc:343] T 0fdb45b31f93466c8bed1b848c6cdf15 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:27.698290 20375 consensus_queue.cc:1048] T 3aaf29a2310a41d0971eded7ef51fdc9 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:27.704574 19551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d8c01c88d24deeae21d647e7f205ba" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:27.776507 19520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a78ac7eb451455e9d2eb6f8cd798a77. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:27.778867 19278 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.193:37907 (request call id 34) took 1895 ms (client timeout 1000 ms). Trace:
W20260925 13:56:27.779253 19278 rpcz_store.cc:269] 0925 13:56:25.882995 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:25.944989 (+ 61994us) service_pool.cc:224] Handling call
0925 13:56:27.778683 (+1833694us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":78363,"reported_tablets":24}
W20260925 13:56:27.779692 19490 leader_election.cc:343] T 2ef3efe644364e93a089ddeed328389d P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 2ef3efe644364e93a089ddeed328389d
I20260925 13:56:27.780229 19955 log.cc:1199] T c1f15b9de572479aae6c2e644da3392a P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/c1f15b9de572479aae6c2e644da3392a
W20260925 13:56:27.770367 19275 rpcz_store.cc:269] 0925 13:56:24.605340 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:24.728108 (+122768us) service_pool.cc:224] Handling call
0925 13:56:24.927654 (+199546us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b5400040454. stack: 00007fdf0a30b323 00007fdf0a30b220 00007fdf09eaf574 0000000000516964 00000000005168f9 00007fdf114095c0 00007fdf114a86fa 00007fdf13219d8c 00007fdf1331801e 00007fdf10ebcfb4 00007fdf10ebcf42 00007fdf10ebce71 00007fdf10ebcdec 00007fdf10ebc0a2 00007fdf0f16e09c 00007fdf0f16d4d6
0925 13:56:27.769967 (+2842313us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:25.511522 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:25.511889 (+ 367us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:25.511911 (+ 22us) write_op.cc:435] Acquired schema lock
0925 13:56:25.511927 (+ 16us) tablet.cc:662] Decoding operations
0925 13:56:25.512172 (+ 245us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:25.512224 (+ 52us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:25.512245 (+ 21us) tablet.cc:685] Acquiring locks for 1 operations
0925 13:56:25.512388 (+ 143us) tablet.cc:701] Row locks acquired
0925 13:56:25.512401 (+ 13us) write_op.cc:260] PREPARE: finished
0925 13:56:25.512504 (+ 103us) write_op.cc:270] Start()
0925 13:56:25.512585 (+ 81us) write_op.cc:276] Timestamp: P: 1790344585512478 usec, L: 0
0925 13:56:25.512601 (+ 16us) op_driver.cc:348] REPLICATION: starting
0925 13:56:25.513045 (+ 444us) log.cc:844] Serialized 536 byte log entry
0925 13:56:25.658927 (+145882us) op_driver.cc:464] REPLICATION: finished
0925 13:56:25.659328 (+ 401us) write_op.cc:301] APPLY: starting
0925 13:56:25.659423 (+ 95us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:25.659545 (+ 122us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:25.659655 (+ 110us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:25.659795 (+ 140us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:25.659922 (+ 127us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 13:56:25.659964 (+ 42us) write_op.cc:312] APPLY: finished
0925 13:56:25.660124 (+ 160us) log.cc:844] Serialized 23 byte log entry
0925 13:56:25.660258 (+ 134us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:25.660318 (+ 60us) write_op.cc:454] Released schema lock
0925 13:56:25.660347 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0925 13:56:25.696910 (+ 36563us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:25.696948 (+ 38us) write_op.cc:454] Released schema lock
Related trace 'op':
0925 13:56:25.974691 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:25.975305 (+ 614us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:25.975336 (+ 31us) write_op.cc:435] Acquired schema lock
0925 13:56:25.975370 (+ 34us) tablet.cc:662] Decoding operations
0925 13:56:25.975662 (+ 292us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:25.975708 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:25.975735 (+ 27us) tablet.cc:685] Acquiring locks for 1 operations
0925 13:56:25.975912 (+ 177us) tablet.cc:701] Row locks acquired
0925 13:56:25.975931 (+ 19us) write_op.cc:260] PREPARE: finished
0925 13:56:25.976044 (+ 113us) write_op.cc:270] Start()
0925 13:56:25.976133 (+ 89us) write_op.cc:276] Timestamp: P: 1790344585976007 usec, L: 0
0925 13:56:25.976153 (+ 20us) op_driver.cc:348] REPLICATION: starting
0925 13:56:26.040502 (+ 64349us) log.cc:844] Serialized 356 byte log entry
0925 13:56:26.213292 (+172790us) op_driver.cc:464] REPLICATION: finished
0925 13:56:26.213971 (+ 679us) write_op.cc:301] APPLY: starting
0925 13:56:26.214099 (+ 128us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:26.214240 (+ 141us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:26.214357 (+ 117us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:26.214678 (+ 321us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:26.214936 (+ 258us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 13:56:26.215002 (+ 66us) write_op.cc:312] APPLY: finished
0925 13:56:26.215424 (+ 422us) log.cc:844] Serialized 23 byte log entry
0925 13:56:26.215664 (+ 240us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:26.215790 (+ 126us) write_op.cc:454] Released schema lock
0925 13:56:26.215832 (+ 42us) write_op.cc:341] FINISH: Updating metrics
0925 13:56:26.216186 (+ 354us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:26.216222 (+ 36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":122256,"reported_tablets":25,"spinlock_wait_cycles":431445888,"child_traces":[["op",{"apply.queue_time_us":252,"apply.run_cpu_time_us":1194,"apply.run_wall_time_us":1194,"num_ops":1,"prepare.queue_time_us":264328,"prepare.run_cpu_time_us":2009,"prepare.run_wall_time_us":185330,"raft.queue_time_us":624,"raft.run_cpu_time_us":621,"raft.run_wall_time_us":621,"replication_time_us":146227}],["op",{"apply.queue_time_us":363,"apply.run_cpu_time_us":2182,"apply.run_wall_time_us":2182,"num_ops":1,"prepare.queue_time_us":292090,"prepare.run_cpu_time_us":2311,"prepare.run_wall_time_us":72906,"raft.queue_time_us":2896,"raft.run_cpu_time_us":12,"raft.run_wall_time_us":11,"replication_time_us":236927}]]}
I20260925 13:56:27.748143 19279 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_dffaefeede134bde97c6bcdff8bfc010" } new_table_name: "default.Table_cd8babe4e16340b18a41ba2e3aa63cd8" modify_external_catalogs: true
W20260925 13:56:27.791987 19280 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_66af2a1d5acd40a4a7ed59932097b68e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:27.793885 19955 ts_tablet_manager.cc:1950] T c1f15b9de572479aae6c2e644da3392a P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:27.795104 19836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fdb45b31f93466c8bed1b848c6cdf15" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:27.795691 19836 raft_consensus.cc:2468] T 0fdb45b31f93466c8bed1b848c6cdf15 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
W20260925 13:56:27.796695 19773 leader_election.cc:343] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:27.798658 19272 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.195:57799 (request call id 37) took 1233 ms (client timeout 1000 ms). Trace:
I20260925 13:56:27.806715 19773 leader_election.cc:304] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:27.808391 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 155) took 6377 ms. Trace:
I20260925 13:56:27.808707 19955 rpcz_store.cc:276] 0925 13:56:21.430717 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:21.435677 (+ 4960us) service_pool.cc:224] Handling call
0925 13:56:27.244198 (+5808521us) ts_tablet_manager.cc:1151] Deleting tablet c1f15b9de572479aae6c2e644da3392a
0925 13:56:27.244248 (+ 50us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:27.552546 (+308298us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:27.607301 (+ 54755us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:27.808333 (+201032us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5784404}
I20260925 13:56:27.809809 19955 tablet_replica.cc:333] T 2711e7a6973546b1822d93f91095dae2 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
W20260925 13:56:27.809773 19272 rpcz_store.cc:269] 0925 13:56:26.565033 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:26.603938 (+ 38905us) service_pool.cc:224] Handling call
0925 13:56:27.798588 (+1194650us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":24}
I20260925 13:56:27.810530 19955 raft_consensus.cc:2243] T 2711e7a6973546b1822d93f91095dae2 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:27.810956 19955 raft_consensus.cc:2272] T 2711e7a6973546b1822d93f91095dae2 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:27.815529 20604 raft_consensus.cc:2749] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:56:27.817171 19773 leader_election.cc:343] T 78d8c01c88d24deeae21d647e7f205ba P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:27.817520 19773 leader_election.cc:304] T 78d8c01c88d24deeae21d647e7f205ba P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:27.828259 19272 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_1de14a8309044a888188fc1c228700fe" } modify_external_catalogs: true
I20260925 13:56:27.828742 19272 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_1de14a8309044a888188fc1c228700fe" } modify_external_catalogs: true
I20260925 13:56:27.854786 20011 ts_tablet_manager.cc:1929] T 0e8aa12e95dc434a98a426499593d9a5 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:27.855484 20604 raft_consensus.cc:2749] T 78d8c01c88d24deeae21d647e7f205ba P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:27.859851 19631 leader_election.cc:304] T 0fdb45b31f93466c8bed1b848c6cdf15 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters: 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:27.860226 19813 tablet_service.cc:1511] Processing CreateTablet for tablet 236e308d8694405cb2b874164fa16485 (DEFAULT_TABLE table=default.Table_25c2c368343741a3923a23f0a7cb2b62 [id=7d4523c5927040ba846c35a4b02172a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:27.861135 20571 raft_consensus.cc:2804] T 0fdb45b31f93466c8bed1b848c6cdf15 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:27.861406 19813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 236e308d8694405cb2b874164fa16485. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:27.861452 20571 raft_consensus.cc:493] T 0fdb45b31f93466c8bed1b848c6cdf15 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:27.861761 20571 raft_consensus.cc:3060] T 0fdb45b31f93466c8bed1b848c6cdf15 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:27.865461 19938 ts_tablet_manager.cc:1434] T b23948807b2c4df9bf9a4da5f767e3b5 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.362s user 0.001s sys 0.005s
I20260925 13:56:27.877120 20571 raft_consensus.cc:515] T 0fdb45b31f93466c8bed1b848c6cdf15 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:27.883737 20604 raft_consensus.cc:493] T c5a9df833271480aa49feb7d9f925459 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:27.884186 20604 raft_consensus.cc:515] T c5a9df833271480aa49feb7d9f925459 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
W20260925 13:56:27.885739 19273 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7e544c93488541da9ebb588a465b27b7 consensus state has no leader
I20260925 13:56:27.886163 20604 leader_election.cc:290] T c5a9df833271480aa49feb7d9f925459 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:27.891570 19532 tablet_service.cc:1511] Processing CreateTablet for tablet 236e308d8694405cb2b874164fa16485 (DEFAULT_TABLE table=default.Table_25c2c368343741a3923a23f0a7cb2b62 [id=7d4523c5927040ba846c35a4b02172a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:27.892779 19532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 236e308d8694405cb2b874164fa16485. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:27.894918 20649 raft_consensus.cc:493] T 82972fa4868d4c59b3b9bb1240300938 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:27.895339 20649 raft_consensus.cc:515] T 82972fa4868d4c59b3b9bb1240300938 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:27.895743 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a9df833271480aa49feb7d9f925459" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:27.897344 20649 leader_election.cc:290] T 82972fa4868d4c59b3b9bb1240300938 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:27.899703 19805 tablet_service.cc:1511] Processing CreateTablet for tablet 6a78ac7eb451455e9d2eb6f8cd798a77 (DEFAULT_TABLE table=default.Table_5564a650590f413580292ea7e803b417 [id=4185dfcad1f1499296231833f2fb8edf]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 13:56:27.899906 19266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.252:36181, user_credentials={real_user=slave}} blocked reactor thread for 116061us
I20260925 13:56:27.900936 19942 tablet_bootstrap.cc:492] T 953d41f26f5f4027acfd374c6ec4ee75 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:27.901367 19942 ts_tablet_manager.cc:1403] T 953d41f26f5f4027acfd374c6ec4ee75 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.668s user 0.011s sys 0.002s
I20260925 13:56:27.924012 19274 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 3e6c8b9bbf4549b3a8685f886973ed9c
I20260925 13:56:27.928629 19675 tablet_service.cc:1558] Processing DeleteTablet for tablet 3e6c8b9bbf4549b3a8685f886973ed9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:27 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:27.930469 19938 tablet_bootstrap.cc:492] T 7e544c93488541da9ebb588a465b27b7 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:27.930984 20571 leader_election.cc:290] T 0fdb45b31f93466c8bed1b848c6cdf15 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:27.942128 19805 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a78ac7eb451455e9d2eb6f8cd798a77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:27.942608 19942 raft_consensus.cc:359] T 953d41f26f5f4027acfd374c6ec4ee75 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:27.943239 19942 raft_consensus.cc:385] T 953d41f26f5f4027acfd374c6ec4ee75 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:27.943513 19942 raft_consensus.cc:740] T 953d41f26f5f4027acfd374c6ec4ee75 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:27.946146 19955 ts_tablet_manager.cc:1916] T 2711e7a6973546b1822d93f91095dae2 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:27.957647 19275 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_25c2cde2aaca4b76b5d5595a803e37a0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:27.958935 19275 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_25c2cde2aaca4b76b5d5595a803e37a0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:27.959988 19277 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_9ed8ad948c7742f0a0e75c2a7a368a9b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:27.961350 19277 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9ed8ad948c7742f0a0e75c2a7a368a9b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:27.987427 19533 tablet_service.cc:1558] Processing DeleteTablet for tablet 1513c0048bd244028d3dfa8d118bc14e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:19 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:27.999486 19942 consensus_queue.cc:260] T 953d41f26f5f4027acfd374c6ec4ee75 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:28.010644 19942 ts_tablet_manager.cc:1434] T 953d41f26f5f4027acfd374c6ec4ee75 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.109s user 0.006s sys 0.000s
I20260925 13:56:28.019989 19280 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_871b30a592064a0abf86c3339a7d5847"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:28.031699 19280 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_871b30a592064a0abf86c3339a7d5847 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:27.855193 20011 log.cc:1199] T 0e8aa12e95dc434a98a426499593d9a5 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/0e8aa12e95dc434a98a426499593d9a5
I20260925 13:56:28.033538 20011 ts_tablet_manager.cc:1950] T 0e8aa12e95dc434a98a426499593d9a5 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
W20260925 13:56:28.052038 19279 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet de2b3e07bd634bc08368989a9062ba08 consensus state has no leader
I20260925 13:56:28.052601 19279 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:39288 (request call id 289) took 1043 ms. Trace:
I20260925 13:56:28.065635 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 0e8aa12e95dc434a98a426499593d9a5 (table default.Table_b42a6e2de4e74cb7a6f01dba22f3b94c [id=03497b028f664f1794f4ff6b53912124]) successfully deleted
W20260925 13:56:28.067162 19264 catalog_manager.cc:5030] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet ec7fe5c834be4a51b627b8b1c3ebe888 because the tablet was not found. No further retry: Not found: Tablet not found: ec7fe5c834be4a51b627b8b1c3ebe888
W20260925 13:56:28.068998 19264 catalog_manager.cc:5030] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 2ef3efe644364e93a089ddeed328389d because the tablet was not found. No further retry: Not found: Tablet not found: 2ef3efe644364e93a089ddeed328389d
W20260925 13:56:28.070827 19264 catalog_manager.cc:5030] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 436b8d46e1d244deb8bb9dc128ee6279 because the tablet was not found. No further retry: Not found: Tablet not found: 436b8d46e1d244deb8bb9dc128ee6279
W20260925 13:56:28.072700 19264 catalog_manager.cc:5030] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet eb794cef052b43da831531455e2b5f15 because the tablet was not found. No further retry: Not found: Tablet not found: eb794cef052b43da831531455e2b5f15
I20260925 13:56:28.054177 19279 rpcz_store.cc:276] 0925 13:56:27.009624 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:27.747826 (+738202us) service_pool.cc:224] Handling call
0925 13:56:27.849487 (+101661us) spinlock_profiling.cc:243] Waited 3.21 ms on lock 0x7b5800001120. stack: 00007fdf0a30b323 00007fdf0a30b220 00007fdf09eaf574 0000000000516964 00000000005168f9 00007fdf0a1c18a1 00007fdf0a306620 00007fdf131ff19d 00007fdf131f75fb 00007fdf13319c22 00007fdf10ecd634 00007fdf10ecd5c2 00007fdf10ecd4f1 00007fdf10ecd46c 00007fdf10ecc722 00007fdf0f16e09c
0925 13:56:27.849567 (+ 80us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 13:56:27.850261 (+ 694us) catalog_manager.cc:3764] Apply alter schema
0925 13:56:27.850872 (+ 611us) catalog_manager.cc:3796] Acquired catalog manager lock
0925 13:56:27.851097 (+ 225us) catalog_manager.cc:3954] Updating metadata on disk
0925 13:56:28.043550 (+192453us) catalog_manager.cc:3985] Committing alterations to in-memory state
0925 13:56:28.052547 (+ 8997us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:27.973862 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:27.974225 (+ 363us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:27.974245 (+ 20us) write_op.cc:435] Acquired schema lock
0925 13:56:27.974258 (+ 13us) tablet.cc:662] Decoding operations
0925 13:56:27.974479 (+ 221us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:27.974512 (+ 33us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:27.974533 (+ 21us) tablet.cc:685] Acquiring locks for 1 operations
0925 13:56:27.974679 (+ 146us) tablet.cc:701] Row locks acquired
0925 13:56:27.974691 (+ 12us) write_op.cc:260] PREPARE: finished
0925 13:56:27.974789 (+ 98us) write_op.cc:270] Start()
0925 13:56:27.974863 (+ 74us) write_op.cc:276] Timestamp: P: 1790344587974755 usec, L: 0
0925 13:56:27.974877 (+ 14us) op_driver.cc:348] REPLICATION: starting
0925 13:56:27.975326 (+ 449us) log.cc:844] Serialized 405 byte log entry
0925 13:56:28.018308 (+ 42982us) op_driver.cc:464] REPLICATION: finished
0925 13:56:28.018631 (+ 323us) write_op.cc:301] APPLY: starting
0925 13:56:28.018684 (+ 53us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:28.018779 (+ 95us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:28.018867 (+ 88us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:28.019013 (+ 146us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:28.019133 (+ 120us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 13:56:28.019177 (+ 44us) write_op.cc:312] APPLY: finished
0925 13:56:28.019333 (+ 156us) log.cc:844] Serialized 23 byte log entry
0925 13:56:28.022502 (+ 3169us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:28.022557 (+ 55us) write_op.cc:454] Released schema lock
0925 13:56:28.022586 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0925 13:56:28.022801 (+ 215us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:28.022824 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":7379200,"child_traces":[["op",{"apply.queue_time_us":203,"apply.run_cpu_time_us":1138,"apply.run_wall_time_us":4138,"num_ops":1,"prepare.queue_time_us":25954,"prepare.run_cpu_time_us":1798,"prepare.run_wall_time_us":42009,"raft.queue_time_us":1131,"raft.run_cpu_time_us":11,"raft.run_wall_time_us":13,"replication_time_us":43353}]]}
I20260925 13:56:28.090240 19533 tablet_service.cc:1558] Processing DeleteTablet for tablet 26644768f516412f8c95d68a76bb99c5 with delete_type TABLET_DATA_DELETED (replaced by tablet b3172d99afee4a4c878e71b645037917) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:28.094748 19279 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_1aa8069da6504812a85380a42ca07583" } new_table_name: "default.Table_dd8a1797a728429697e35fd628338f05" modify_external_catalogs: true
I20260925 13:56:28.101404 19533 tablet_service.cc:1558] Processing DeleteTablet for tablet 2a1f34dba13143b097a38bab3782c404 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:14 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:28.108340 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2711e7a6973546b1822d93f91095dae2" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:28.108975 19692 raft_consensus.cc:2468] T 2711e7a6973546b1822d93f91095dae2 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:28.111143 20083 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260925 13:56:28.113210 20083 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:56:28.113853 20083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:56:27.995886 19524 tablet_service.cc:1558] Processing DeleteTablet for tablet c161ae3edfed4f8488c770ecc6332607 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:17 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:28.133020 20644 ts_tablet_manager.cc:1929] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:28.135084 19942 tablet_bootstrap.cc:492] T 1d641f09429546d683f2884c02152f73 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:28.144470 19533 tablet_service.cc:1558] Processing DeleteTablet for tablet e6e59d6f4f6e4de1be1cf450620cfa59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:20 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:28.157320 19521 tablet_service.cc:1558] Processing DeleteTablet for tablet 5788849b7336496488e5401b4ac91191 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:20 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:28.162339 19551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82972fa4868d4c59b3b9bb1240300938" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:28.108629 19549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a9df833271480aa49feb7d9f925459" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:28.184971 19532 tablet_service.cc:1558] Processing DeleteTablet for tablet 3e6c8b9bbf4549b3a8685f886973ed9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:27 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:28.136389 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fdb45b31f93466c8bed1b848c6cdf15" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:28.191574 19524 tablet_service.cc:1558] Processing DeleteTablet for tablet 137c43fb74ec4cc48d30ed3edac81d98 with delete_type TABLET_DATA_DELETED (replaced by tablet 540132db164c403da72b9377078c4bf0) from {username='slave'} at 127.0.0.1:40018
W20260925 13:56:28.191663 19629 leader_election.cc:343] T 82972fa4868d4c59b3b9bb1240300938 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:28.192224 19629 leader_election.cc:343] T 0fdb45b31f93466c8bed1b848c6cdf15 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:28.193305 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet c1f15b9de572479aae6c2e644da3392a (table default.Table_8b37ba6f398943df8ab95b19f0b7dec6 [id=9cb2dc12a91e4de087414b492cd0dcfb]) successfully deleted
I20260925 13:56:28.203253 19275 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_b2092cda68f44cd4a3a79e3360f2e7ff"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:28.205114 19275 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b2092cda68f44cd4a3a79e3360f2e7ff in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:28.221729 19274 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_732ec1f43dec4e6b8cd73691cdd91358" } modify_external_catalogs: true
I20260925 13:56:28.222661 19274 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_732ec1f43dec4e6b8cd73691cdd91358" } modify_external_catalogs: true
I20260925 13:56:28.238616 20685 raft_consensus.cc:493] T 953d41f26f5f4027acfd374c6ec4ee75 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:28.239068 20685 raft_consensus.cc:515] T 953d41f26f5f4027acfd374c6ec4ee75 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:28.241415 20685 leader_election.cc:290] T 953d41f26f5f4027acfd374c6ec4ee75 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:28.248307 19549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "953d41f26f5f4027acfd374c6ec4ee75" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:28.251545 19939 tablet_bootstrap.cc:654] T 953d41f26f5f4027acfd374c6ec4ee75 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
W20260925 13:56:28.256013 19629 leader_election.cc:343] T 953d41f26f5f4027acfd374c6ec4ee75 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:28.256825 19629 consensus_peers.cc:597] T a8a679a453e0430b866edc707fe7c234 P 7c1b52895a14482fb54a6758f5054f8f -> Peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Couldn't send request to peer 8f578869bc084a5d8a9a69f143954eb3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:28.239460 19525 tablet_service.cc:1558] Processing DeleteTablet for tablet 12984db09f34472ba365fdf63a33b720 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:20 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:28.247237 20709 raft_consensus.cc:493] T eb794cef052b43da831531455e2b5f15 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:28.244041 19523 tablet_service.cc:1558] Processing DeleteTablet for tablet bbfc4f99a5964e2ba6146e390e8dba44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:19 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:28.260574 20709 raft_consensus.cc:515] T eb794cef052b43da831531455e2b5f15 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:28.262369 20709 leader_election.cc:290] T eb794cef052b43da831531455e2b5f15 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:28.263350 19276 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_3bc5f887568f417aa8c1269ddf12544a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:28.265342 19276 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3bc5f887568f417aa8c1269ddf12544a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:28.272009 19271 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_4a32345f45fd4ca2a6bdce69c6c2073b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:28.273916 19271 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4a32345f45fd4ca2a6bdce69c6c2073b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:28.275278 20011 tablet_replica.cc:333] T 5a9b3dfaa7474611a1824d63eb906b43 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:28.276129 20011 raft_consensus.cc:2243] T 5a9b3dfaa7474611a1824d63eb906b43 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:28.276597 20011 raft_consensus.cc:2272] T 5a9b3dfaa7474611a1824d63eb906b43 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:28.278359 20011 ts_tablet_manager.cc:1916] T 5a9b3dfaa7474611a1824d63eb906b43 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:28.291687 19277 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_1e8d4bb8dcbb45a3b882ed664417c23b" } new_table_name: "default.Table_b43a08d7f02f4b5b8ef48ab6d296efa0" modify_external_catalogs: true
I20260925 13:56:28.314344 19939 tablet_bootstrap.cc:492] T 953d41f26f5f4027acfd374c6ec4ee75 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:28.314800 19939 ts_tablet_manager.cc:1403] T 953d41f26f5f4027acfd374c6ec4ee75 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.565s user 0.011s sys 0.005s
I20260925 13:56:28.388815 19939 raft_consensus.cc:359] T 953d41f26f5f4027acfd374c6ec4ee75 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:28.389916 19939 raft_consensus.cc:385] T 953d41f26f5f4027acfd374c6ec4ee75 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:28.390170 19939 raft_consensus.cc:740] T 953d41f26f5f4027acfd374c6ec4ee75 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:28.390787 19939 consensus_queue.cc:260] T 953d41f26f5f4027acfd374c6ec4ee75 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:28.392616 19939 ts_tablet_manager.cc:1434] T 953d41f26f5f4027acfd374c6ec4ee75 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.074s user 0.008s sys 0.008s
I20260925 13:56:28.407464 19488 leader_election.cc:304] T 2711e7a6973546b1822d93f91095dae2 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3; no voters: f790493ba7ad403993b678f2df2b55be
W20260925 13:56:28.408032 19488 raft_consensus.cc:2686] T 2711e7a6973546b1822d93f91095dae2 P 8f578869bc084a5d8a9a69f143954eb3: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260925 13:56:28.408653 19488 leader_election.cc:343] T 2ef3efe644364e93a089ddeed328389d P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Not found: Tablet not found: 2ef3efe644364e93a089ddeed328389d
I20260925 13:56:28.408982 19488 leader_election.cc:304] T 2ef3efe644364e93a089ddeed328389d P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3; no voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be
W20260925 13:56:28.409642 19265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.254:40825, user_credentials={real_user=slave}} blocked reactor thread for 116513us
I20260925 13:56:28.411100 19939 tablet_bootstrap.cc:492] T 5a9b3dfaa7474611a1824d63eb906b43 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:28.413604 20723 raft_consensus.cc:2749] T 2ef3efe644364e93a089ddeed328389d P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:56:28.416742 19280 master_service.cc:960] ReplaceTablet: received request to replace tablet 3aaf29a2310a41d0971eded7ef51fdc9 from {username='slave'} at 127.0.0.1:39288
W20260925 13:56:28.421339 19488 leader_election.cc:343] T eb794cef052b43da831531455e2b5f15 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Not found: Tablet not found: eb794cef052b43da831531455e2b5f15
I20260925 13:56:28.428517 19939 tablet_bootstrap.cc:654] T 5a9b3dfaa7474611a1824d63eb906b43 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:28.443027 19939 tablet_bootstrap.cc:492] T 5a9b3dfaa7474611a1824d63eb906b43 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:28.443485 19939 ts_tablet_manager.cc:1403] T 5a9b3dfaa7474611a1824d63eb906b43 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.033s user 0.014s sys 0.006s
I20260925 13:56:28.447304 19273 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet f16050752ad44720ba8703ef10bbb359 (table default.Table_1da8b04f68ab4682a8ec32ad47854930 [id=b954461264164e969a40af810627fcb2]). Expected version 1 got 0
I20260925 13:56:28.447964 19273 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 9059655f2c3d4e86b3b389b5b9a68891 (table default.Table_1b1f36048d2a4a3c86e513e46bcf5471 [id=0c13bfd171f2464b887f15d085439fa3]). Expected version 1 got 0
I20260925 13:56:28.448530 19273 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet e3cf9676160c4af08f1f62ae952b4c57 (table default.Table_4d6d747ea4034b3ab48710e10ff4544f [id=d6e6dd0772bf4901b696e11b10e41a48]). Expected version 1 got 0
I20260925 13:56:28.449004 19273 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 82972fa4868d4c59b3b9bb1240300938 (table default.Table_d3d2076fbb6241a68f04e83462a34182 [id=4232cecee9104120ac2521eb250bbd0d]). Expected version 1 got 0
I20260925 13:56:28.462164 19671 tablet_service.cc:1511] Processing CreateTablet for tablet b45c85b8ff7c4b8f8cc8bab6fe20780b (DEFAULT_TABLE table=default.Table_746abdd6da964b3592615357002e76bd [id=b6556fe94fdc4c0288eb18c83c015b2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:28.463549 19671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b45c85b8ff7c4b8f8cc8bab6fe20780b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:28.467875 19667 tablet_service.cc:1511] Processing CreateTablet for tablet 931c54cac90849ab831a8d41cd6ff567 (DEFAULT_TABLE table=default.Table_db5fac75a1364fbda83732d71eabb1e2 [id=d4153ef523db47408495d5f9b76422f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:28.469038 19667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 931c54cac90849ab831a8d41cd6ff567. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:28.445881 19939 raft_consensus.cc:359] T 5a9b3dfaa7474611a1824d63eb906b43 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:28.479852 19939 raft_consensus.cc:385] T 5a9b3dfaa7474611a1824d63eb906b43 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:28.480139 19939 raft_consensus.cc:740] T 5a9b3dfaa7474611a1824d63eb906b43 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:28.480743 19939 consensus_queue.cc:260] T 5a9b3dfaa7474611a1824d63eb906b43 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:28.481793 19942 tablet_bootstrap.cc:654] T 1d641f09429546d683f2884c02152f73 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:28.485108 19674 tablet_service.cc:1511] Processing CreateTablet for tablet 4b36d5f94de04dca98b62e8128602cbb (DEFAULT_TABLE table=default.Table_770e2b27b6d4455787f7b0eff2a7221a [id=fcda99f490014a2f8a88fed041ad5228]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:28.486089 19674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b36d5f94de04dca98b62e8128602cbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:28.489075 19278 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet b3172d99afee4a4c878e71b645037917 (table default.Table_bd52a86b8ff14bdbb06fcf88a60b1e82 [id=203024d67e444f839391c8670d7c7d3b]). Expected version 2 got 0
I20260925 13:56:28.491531 19666 tablet_service.cc:1511] Processing CreateTablet for tablet 765b022051e84f5487b389c3ff29d0e5 (DEFAULT_TABLE table=default.Table_1e8d4bb8dcbb45a3b882ed664417c23b [id=8668cef08fbb4364830675233ded3e80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:28.492552 19666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 765b022051e84f5487b389c3ff29d0e5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:28.518572 19488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.194:38641, user_credentials={real_user=slave}} blocked reactor thread for 60269.7us
I20260925 13:56:28.520572 19939 ts_tablet_manager.cc:1434] T 5a9b3dfaa7474611a1824d63eb906b43 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.077s user 0.004s sys 0.001s
I20260925 13:56:28.510838 19275 master_service.cc:960] ReplaceTablet: received request to replace tablet 61f141c40ce64fbcb68e01ac15009e3c from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:28.525319 19278 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 5f8a96ae6eec4f15a221f9aad537c094 (table default.Table_3a06919b658e46f893318b4e1ec8b52f [id=e6f2a872536b4e55b071231020903581]). Expected version 1 got 0
I20260925 13:56:28.526827 19939 tablet_bootstrap.cc:492] T 0e8aa12e95dc434a98a426499593d9a5 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:28.532464 19939 tablet_bootstrap.cc:654] T 0e8aa12e95dc434a98a426499593d9a5 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:28.530529 19278 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 000aad39953c4688b543c7d882a08ec8 (table default.Table_3a3e48a0641a43a1a783c6ba6374a708 [id=e64fe44a237f48859a288a87a565ee5e]). Expected version 1 got 0
I20260925 13:56:28.536888 19278 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet f16050752ad44720ba8703ef10bbb359 (table default.Table_1da8b04f68ab4682a8ec32ad47854930 [id=b954461264164e969a40af810627fcb2]). Expected version 1 got 0
I20260925 13:56:28.537719 19670 tablet_service.cc:1511] Processing CreateTablet for tablet 96213c58b3bb4a8c8760da3da73d4bbe (DEFAULT_TABLE table=default.Table_d1ac7e953d034f28babf9808d47e7fbf [id=2a357a24c19941c294b59e28f95fc50d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:28.538794 19670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96213c58b3bb4a8c8760da3da73d4bbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:28.539216 19278 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet e3cf9676160c4af08f1f62ae952b4c57 (table default.Table_4d6d747ea4034b3ab48710e10ff4544f [id=d6e6dd0772bf4901b696e11b10e41a48]). Expected version 1 got 0
I20260925 13:56:28.548339 19278 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet f50ea68e3b3c4459b3f5e5b0df724724 (table default.Table_01f0109a149148dfaf1974cee0c3e42b [id=f1353f53d28748c8a8a4a2c6259581bf]). Expected version 3 got 0
I20260925 13:56:28.547519 20723 raft_consensus.cc:493] T 5a9b3dfaa7474611a1824d63eb906b43 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:28.549091 19521 tablet_service.cc:1511] Processing CreateTablet for tablet b45c85b8ff7c4b8f8cc8bab6fe20780b (DEFAULT_TABLE table=default.Table_746abdd6da964b3592615357002e76bd [id=b6556fe94fdc4c0288eb18c83c015b2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:28.548921 20723 raft_consensus.cc:515] T 5a9b3dfaa7474611a1824d63eb906b43 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:28.550041 19278 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 82972fa4868d4c59b3b9bb1240300938 (table default.Table_d3d2076fbb6241a68f04e83462a34182 [id=4232cecee9104120ac2521eb250bbd0d]). Expected version 1 got 0
I20260925 13:56:28.550736 19278 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 9059655f2c3d4e86b3b389b5b9a68891 (table default.Table_1b1f36048d2a4a3c86e513e46bcf5471 [id=0c13bfd171f2464b887f15d085439fa3]). Expected version 1 got 0
I20260925 13:56:28.551640 19278 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet e9dbcd85dc8647c68c120ba3fcfe1790 (table default.Table_5bf9b87b5083469fad04eb3a80a4b8d5 [id=443536a0bd4d4d568d52f6ffe3d06ecb]). Expected version 1 got 0
I20260925 13:56:28.552531 20723 leader_election.cc:290] T 5a9b3dfaa7474611a1824d63eb906b43 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:28.554625 19526 tablet_service.cc:1511] Processing CreateTablet for tablet 931c54cac90849ab831a8d41cd6ff567 (DEFAULT_TABLE table=default.Table_db5fac75a1364fbda83732d71eabb1e2 [id=d4153ef523db47408495d5f9b76422f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:28.556324 19525 tablet_service.cc:1511] Processing CreateTablet for tablet 4b36d5f94de04dca98b62e8128602cbb (DEFAULT_TABLE table=default.Table_770e2b27b6d4455787f7b0eff2a7221a [id=fcda99f490014a2f8a88fed041ad5228]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:28.557746 19519 tablet_service.cc:1511] Processing CreateTablet for tablet 765b022051e84f5487b389c3ff29d0e5 (DEFAULT_TABLE table=default.Table_1e8d4bb8dcbb45a3b882ed664417c23b [id=8668cef08fbb4364830675233ded3e80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:28.558755 19529 tablet_service.cc:1511] Processing CreateTablet for tablet 96213c58b3bb4a8c8760da3da73d4bbe (DEFAULT_TABLE table=default.Table_d1ac7e953d034f28babf9808d47e7fbf [id=2a357a24c19941c294b59e28f95fc50d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:28.568569 19521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b45c85b8ff7c4b8f8cc8bab6fe20780b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:28.569980 19529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96213c58b3bb4a8c8760da3da73d4bbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:28.571269 19519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 765b022051e84f5487b389c3ff29d0e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:28.578399 19672 tablet_service.cc:1511] Processing CreateTablet for tablet 7f267b3b911248419a5d18cc222a0057 (DEFAULT_TABLE table=default.Table_03de3c9f9f844ca98b834d7f6a809daf [id=aa44369061f04ea0b77ff64c6aaf414e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:28.579519 19672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f267b3b911248419a5d18cc222a0057. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:28.584372 19526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 931c54cac90849ab831a8d41cd6ff567. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:28.585721 19525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b36d5f94de04dca98b62e8128602cbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:28.592448 19675 tablet_service.cc:1511] Processing CreateTablet for tablet 3d754e17930549189d65becb98afe2a6 (DEFAULT_TABLE table=default.Table_c389f408e051410dac7a521879c22729 [id=d2bc135460ab4f2fad4740a281bfc6e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:28.593627 19675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d754e17930549189d65becb98afe2a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:28.597982 19666 tablet_service.cc:1511] Processing CreateTablet for tablet e51e1e553857435eba28a979de0a6a23 (DEFAULT_TABLE table=default.Table_d33ac5b10f974ab592b99e99309c0c8c [id=c89e4c78da164cf9aa57fbbb6513bb4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:28.595913 20730 tablet.cc:1751] T 3aaf29a2310a41d0971eded7ef51fdc9 P 8f578869bc084a5d8a9a69f143954eb3: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260925 13:56:28.505636 19773 leader_election.cc:343] T c5a9df833271480aa49feb7d9f925459 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:28.639747 20644 log.cc:1199] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/0e8aa12e95dc434a98a426499593d9a5
W20260925 13:56:28.639658 19773 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.194:38641, user_credentials={real_user=slave}} blocked reactor thread for 134493us
I20260925 13:56:28.640771 20644 ts_tablet_manager.cc:1950] T 0e8aa12e95dc434a98a426499593d9a5 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:28.641324 19818 tablet_service.cc:1558] Processing DeleteTablet for tablet e2e44b2ca25c4bbcb91288424b3e18b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:24 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:28.642709 19818 tablet_service.cc:1558] Processing DeleteTablet for tablet 3e6c8b9bbf4549b3a8685f886973ed9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:27 UTC) from {username='slave'} at 127.0.0.1:51442
W20260925 13:56:28.579099 19819 consensus_meta.cc:287] T 2b73f00e58c7444b90fd01b685769b2b P f790493ba7ad403993b678f2df2b55be: Time spent flushing consensus metadata: real 0.650s user 0.007s sys 0.000s
I20260925 13:56:28.635639 19938 tablet_bootstrap.cc:654] T 7e544c93488541da9ebb588a465b27b7 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:28.646199 19813 tablet_service.cc:1511] Processing CreateTablet for tablet b45c85b8ff7c4b8f8cc8bab6fe20780b (DEFAULT_TABLE table=default.Table_746abdd6da964b3592615357002e76bd [id=b6556fe94fdc4c0288eb18c83c015b2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:28.647580 19813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b45c85b8ff7c4b8f8cc8bab6fe20780b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:28.654402 19801 tablet_service.cc:1511] Processing CreateTablet for tablet 4b36d5f94de04dca98b62e8128602cbb (DEFAULT_TABLE table=default.Table_770e2b27b6d4455787f7b0eff2a7221a [id=fcda99f490014a2f8a88fed041ad5228]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:28.654629 19817 tablet_service.cc:1511] Processing CreateTablet for tablet 96213c58b3bb4a8c8760da3da73d4bbe (DEFAULT_TABLE table=default.Table_d1ac7e953d034f28babf9808d47e7fbf [id=2a357a24c19941c294b59e28f95fc50d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:28.655571 19801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b36d5f94de04dca98b62e8128602cbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:28.656239 19817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96213c58b3bb4a8c8760da3da73d4bbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:28.657584 19804 tablet_service.cc:1511] Processing CreateTablet for tablet 765b022051e84f5487b389c3ff29d0e5 (DEFAULT_TABLE table=default.Table_1e8d4bb8dcbb45a3b882ed664417c23b [id=8668cef08fbb4364830675233ded3e80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:28.658619 19804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 765b022051e84f5487b389c3ff29d0e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:28.659730 19814 tablet_service.cc:1511] Processing CreateTablet for tablet 7f267b3b911248419a5d18cc222a0057 (DEFAULT_TABLE table=default.Table_03de3c9f9f844ca98b834d7f6a809daf [id=aa44369061f04ea0b77ff64c6aaf414e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:28.660391 20644 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 216) took 1040 ms. Trace:
I20260925 13:56:28.660481 19803 tablet_service.cc:1511] Processing CreateTablet for tablet e51e1e553857435eba28a979de0a6a23 (DEFAULT_TABLE table=default.Table_d33ac5b10f974ab592b99e99309c0c8c [id=c89e4c78da164cf9aa57fbbb6513bb4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:28.660920 19814 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f267b3b911248419a5d18cc222a0057. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:28.660820 20644 rpcz_store.cc:276] 0925 13:56:27.619787 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:27.621548 (+ 1761us) service_pool.cc:224] Handling call
0925 13:56:27.623476 (+ 1928us) ts_tablet_manager.cc:1151] Deleting tablet 0e8aa12e95dc434a98a426499593d9a5
0925 13:56:27.623523 (+ 47us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:27.903579 (+280056us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:28.132847 (+229268us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:28.660307 (+527460us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1394}
I20260925 13:56:28.661818 19803 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e51e1e553857435eba28a979de0a6a23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:28.662493 20644 tablet_replica.cc:333] T 953d41f26f5f4027acfd374c6ec4ee75 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
W20260925 13:56:28.667572 19490 leader_election.cc:343] T 5a9b3dfaa7474611a1824d63eb906b43 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 5a9b3dfaa7474611a1824d63eb906b43
I20260925 13:56:28.668751 19810 tablet_service.cc:1511] Processing CreateTablet for tablet 3d754e17930549189d65becb98afe2a6 (DEFAULT_TABLE table=default.Table_c389f408e051410dac7a521879c22729 [id=d2bc135460ab4f2fad4740a281bfc6e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:28.669655 19810 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d754e17930549189d65becb98afe2a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:28.674088 19819 tablet_service.cc:1511] Processing CreateTablet for tablet 931c54cac90849ab831a8d41cd6ff567 (DEFAULT_TABLE table=default.Table_db5fac75a1364fbda83732d71eabb1e2 [id=d4153ef523db47408495d5f9b76422f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:28.675105 19819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 931c54cac90849ab831a8d41cd6ff567. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:28.676017 19278 catalog_manager.cc:6038] Failed to send AlterTable RPC for tablet b3172d99afee4a4c878e71b645037917 (table default.Table_bd52a86b8ff14bdbb06fcf88a60b1e82 [id=203024d67e444f839391c8670d7c7d3b], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet b3172d99afee4a4c878e71b645037917 consensus state has no leader
I20260925 13:56:28.679313 19809 tablet_service.cc:1511] Processing CreateTablet for tablet f806a82ee59c4da79e2f30647adc3c60 (DEFAULT_TABLE table=default.Table_727855fadd224779a3638b48e6b0266f [id=4e57e9392b3f43e3a767b7c4b641d498]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:28.684573 19816 tablet_service.cc:1511] Processing CreateTablet for tablet 1dad030c86b54b7d87dd3b825c811640 (DEFAULT_TABLE table=default.Table_d5b57f5cf2574edcbe5bd91f38508763 [id=08eba7b5fc604580a8359e3cfb7d65fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:28.687476 19802 tablet_service.cc:1511] Processing CreateTablet for tablet 940d14678f89470c85371b5a2972ac3c (DEFAULT_TABLE table=default.Table_3c6b9bb73ad444eca6106c4aa3289fdc [id=925b603294c446779ce50e83a6955d06]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 13:56:28.697154 19490 leader_election.cc:343] T eb794cef052b43da831531455e2b5f15 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: eb794cef052b43da831531455e2b5f15
I20260925 13:56:28.697685 19490 leader_election.cc:304] T eb794cef052b43da831531455e2b5f15 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3; no voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:28.698935 19840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82972fa4868d4c59b3b9bb1240300938" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:28.699783 19840 raft_consensus.cc:2468] T 82972fa4868d4c59b3b9bb1240300938 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:28.700727 20721 raft_consensus.cc:2749] T eb794cef052b43da831531455e2b5f15 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:56:28.704317 19278 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.195:57799 (request call id 38) took 767 ms (client timeout 1000 ms). Trace:
I20260925 13:56:28.706833 19802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 940d14678f89470c85371b5a2972ac3c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:28.708199 19278 rpcz_store.cc:269] 0925 13:56:27.936401 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:27.936517 (+ 116us) service_pool.cc:224] Handling call
0925 13:56:27.956715 (+ 20198us) spinlock_profiling.cc:243] Waited 16.1 ms on lock 0x7b5400040454. stack: 00007fdf0a30b323 00007fdf0a30b220 00007fdf09eaf574 0000000000516964 00000000005168f9 00007fdf114095c0 00007fdf114a86fa 00007fdf13219d8c 00007fdf1331801e 00007fdf10ebcfb4 00007fdf10ebcf42 00007fdf10ebce71 00007fdf10ebcdec 00007fdf10ebc0a2 00007fdf0f16e09c 00007fdf0f16d4d6
0925 13:56:28.704250 (+747535us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":30,"spinlock_wait_cycles":37138688}
I20260925 13:56:28.711637 19816 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1dad030c86b54b7d87dd3b825c811640. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:28.722153 19264 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet de2b3e07bd634bc08368989a9062ba08 (table default.Table_cd8babe4e16340b18a41ba2e3aa63cd8 [id=be2ddd2647094a8788d52ef46500426f], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet de2b3e07bd634bc08368989a9062ba08 consensus state has no leader
I20260925 13:56:28.723803 19809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f806a82ee59c4da79e2f30647adc3c60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:28.738374 20692 consensus_queue.cc:1048] T 3aaf29a2310a41d0971eded7ef51fdc9 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 13:56:28.751585 19839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fdb45b31f93466c8bed1b848c6cdf15" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:28.752169 19839 raft_consensus.cc:3060] T 0fdb45b31f93466c8bed1b848c6cdf15 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:28.768054 19812 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:51442 (request call id 211) took 1227 ms. Trace:
I20260925 13:56:28.768465 19812 rpcz_store.cc:276] 0925 13:56:27.540336 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:27.540986 (+ 650us) service_pool.cc:224] Handling call
0925 13:56:27.542193 (+ 1207us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0925 13:56:27.542264 (+ 71us) ts_tablet_manager.cc:682] Creating new metadata...
0925 13:56:28.620568 (+1078304us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:28.767516 (+146948us) tablet_replica.cc:206] Creating consensus instance
0925 13:56:28.767893 (+ 377us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260925 13:56:28.610567 19669 tablet_service.cc:1511] Processing CreateTablet for tablet f806a82ee59c4da79e2f30647adc3c60 (DEFAULT_TABLE table=default.Table_727855fadd224779a3638b48e6b0266f [id=4e57e9392b3f43e3a767b7c4b641d498]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:28.788602 19669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f806a82ee59c4da79e2f30647adc3c60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:28.666095 19676 tablet_service.cc:1511] Processing CreateTablet for tablet 940d14678f89470c85371b5a2972ac3c (DEFAULT_TABLE table=default.Table_3c6b9bb73ad444eca6106c4aa3289fdc [id=925b603294c446779ce50e83a6955d06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:28.679147 19665 tablet_service.cc:1511] Processing CreateTablet for tablet 84188f52b54b477393d6a8433cbd1cb9 (DEFAULT_TABLE table=default.Table_29a7a24743f647108b2fa4629b620df5 [id=b3bd097a5199481290fe76162a2017a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:28.687680 19675 tablet_service.cc:1511] Processing CreateTablet for tablet 1dad030c86b54b7d87dd3b825c811640 (DEFAULT_TABLE table=default.Table_d5b57f5cf2574edcbe5bd91f38508763 [id=08eba7b5fc604580a8359e3cfb7d65fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:28.702455 19673 tablet_service.cc:1511] Processing CreateTablet for tablet 197232acf0844b1d813e7699c2531b84 (DEFAULT_TABLE table=default.Table_5b3429bafb2248f99555eb27a2780ff6 [id=035f7129265e44d0bfd81afbf9f57f96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:28.792364 19676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 940d14678f89470c85371b5a2972ac3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:28.796126 19675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1dad030c86b54b7d87dd3b825c811640. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:28.797449 19837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "953d41f26f5f4027acfd374c6ec4ee75" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:28.801147 19837 raft_consensus.cc:2468] T 953d41f26f5f4027acfd374c6ec4ee75 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:28.801381 19631 leader_election.cc:304] T 82972fa4868d4c59b3b9bb1240300938 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters: 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:28.804311 19673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 197232acf0844b1d813e7699c2531b84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:28.810848 19278 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_9529f5b35c21450db26ad3efb6bece7b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:28.812466 19278 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9529f5b35c21450db26ad3efb6bece7b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:28.812695 19665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84188f52b54b477393d6a8433cbd1cb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:28.812841 19518 tablet_service.cc:1511] Processing CreateTablet for tablet e51e1e553857435eba28a979de0a6a23 (DEFAULT_TABLE table=default.Table_d33ac5b10f974ab592b99e99309c0c8c [id=c89e4c78da164cf9aa57fbbb6513bb4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:28.813951 19518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e51e1e553857435eba28a979de0a6a23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:28.820147 20685 raft_consensus.cc:2804] T 82972fa4868d4c59b3b9bb1240300938 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:28.820510 20685 raft_consensus.cc:493] T 82972fa4868d4c59b3b9bb1240300938 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:28.820814 20685 raft_consensus.cc:3060] T 82972fa4868d4c59b3b9bb1240300938 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:28.825320 19274 catalog_manager.cc:6128] Not sending DeleteTablet requests; no consensus state for tablet 14f2665ccab5471db2d1081ec7bad179
I20260925 13:56:28.829557 19522 tablet_service.cc:1511] Processing CreateTablet for tablet 7f267b3b911248419a5d18cc222a0057 (DEFAULT_TABLE table=default.Table_03de3c9f9f844ca98b834d7f6a809daf [id=aa44369061f04ea0b77ff64c6aaf414e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:28.830524 19522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f267b3b911248419a5d18cc222a0057. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:28.854956 19274 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_d57c934ba0b84027b1830f751a466939"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:28.839232 20685 raft_consensus.cc:515] T 82972fa4868d4c59b3b9bb1240300938 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:28.865955 19666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e51e1e553857435eba28a979de0a6a23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:28.871923 19839 raft_consensus.cc:2468] T 0fdb45b31f93466c8bed1b848c6cdf15 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
I20260925 13:56:28.877873 20644 raft_consensus.cc:2243] T 953d41f26f5f4027acfd374c6ec4ee75 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:28.878453 20644 raft_consensus.cc:2272] T 953d41f26f5f4027acfd374c6ec4ee75 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:28.880522 20644 ts_tablet_manager.cc:1916] T 953d41f26f5f4027acfd374c6ec4ee75 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:28.883882 20685 leader_election.cc:290] T 82972fa4868d4c59b3b9bb1240300938 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
W20260925 13:56:28.894970 19631 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 92637.6us
I20260925 13:56:28.901743 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 0e8aa12e95dc434a98a426499593d9a5 (table default.Table_b42a6e2de4e74cb7a6f01dba22f3b94c [id=03497b028f664f1794f4ff6b53912124]) successfully deleted
I20260925 13:56:28.906316 19523 tablet_service.cc:1511] Processing CreateTablet for tablet 3d754e17930549189d65becb98afe2a6 (DEFAULT_TABLE table=default.Table_c389f408e051410dac7a521879c22729 [id=d2bc135460ab4f2fad4740a281bfc6e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:28.907495 19523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d754e17930549189d65becb98afe2a6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:28.902612 19274 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d57c934ba0b84027b1830f751a466939 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:28.918062 20735 tablet.cc:1751] T 3aaf29a2310a41d0971eded7ef51fdc9 P 7c1b52895a14482fb54a6758f5054f8f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260925 13:56:28.925625 20083 thread.cc:641] rpc worker (service pool kudu.consensus.ConsensusService) Time spent creating pthread: real 0.557s user 0.030s sys 0.071s
W20260925 13:56:28.932799 20083 thread.cc:608] rpc worker (service pool kudu.consensus.ConsensusService) Time spent starting thread: real 0.565s user 0.030s sys 0.072s
I20260925 13:56:28.864173 19279 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_6f3fcafe361344c6920c158efa979438" } new_table_name: "default.Table_33e9cd401c9d45d0a1af408bda5e3bfa" modify_external_catalogs: true
I20260925 13:56:28.980785 19942 tablet_bootstrap.cc:492] T 1d641f09429546d683f2884c02152f73 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:28.981364 19942 ts_tablet_manager.cc:1403] T 1d641f09429546d683f2884c02152f73 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.846s user 0.022s sys 0.060s
I20260925 13:56:28.987618 20748 consensus_queue.cc:1048] T a8a679a453e0430b866edc707fe7c234 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:28.990684 19955 ts_tablet_manager.cc:1929] T 2711e7a6973546b1822d93f91095dae2 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:28.991139 19955 log.cc:1199] T 2711e7a6973546b1822d93f91095dae2 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/2711e7a6973546b1822d93f91095dae2
I20260925 13:56:28.991943 19955 ts_tablet_manager.cc:1950] T 2711e7a6973546b1822d93f91095dae2 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:28.994059 20604 raft_consensus.cc:493] T b3172d99afee4a4c878e71b645037917 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:28.994537 20604 raft_consensus.cc:515] T b3172d99afee4a4c878e71b645037917 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:28.997006 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9b3dfaa7474611a1824d63eb906b43" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:29.001287 20604 leader_election.cc:290] T b3172d99afee4a4c878e71b645037917 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
W20260925 13:56:29.002539 19773 leader_election.cc:343] T c5a9df833271480aa49feb7d9f925459 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:29.003000 20604 raft_consensus.cc:493] T b23948807b2c4df9bf9a4da5f767e3b5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:29.003933 19773 leader_election.cc:304] T c5a9df833271480aa49feb7d9f925459 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:29.005393 20604 raft_consensus.cc:515] T b23948807b2c4df9bf9a4da5f767e3b5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:29.010769 20604 leader_election.cc:290] T b23948807b2c4df9bf9a4da5f767e3b5 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:29.012591 19938 tablet_bootstrap.cc:492] T 7e544c93488541da9ebb588a465b27b7 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:29.013463 19938 ts_tablet_manager.cc:1403] T 7e544c93488541da9ebb588a465b27b7 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 1.088s user 0.017s sys 0.002s
I20260925 13:56:29.024880 20604 raft_consensus.cc:2749] T c5a9df833271480aa49feb7d9f925459 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:29.028075 19818 tablet_service.cc:1558] Processing DeleteTablet for tablet 78d8c01c88d24deeae21d647e7f205ba with delete_type TABLET_DATA_DELETED (replaced by tablet ab1ba6ef73a046b59ffac150f7c73953) from {username='slave'} at 127.0.0.1:51442
W20260925 13:56:29.051394 19277 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 765b022051e84f5487b389c3ff29d0e5 consensus state has no leader
I20260925 13:56:29.070230 19818 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d01a791445149e7bee9493e1eec5c6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:21 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:29.072036 19938 raft_consensus.cc:359] T 7e544c93488541da9ebb588a465b27b7 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:29.072811 19938 raft_consensus.cc:385] T 7e544c93488541da9ebb588a465b27b7 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:29.073098 19938 raft_consensus.cc:740] T 7e544c93488541da9ebb588a465b27b7 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:29.073098 19942 raft_consensus.cc:359] T 1d641f09429546d683f2884c02152f73 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:29.073935 19942 raft_consensus.cc:385] T 1d641f09429546d683f2884c02152f73 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:29.073758 19938 consensus_queue.cc:260] T 7e544c93488541da9ebb588a465b27b7 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:29.074242 19942 raft_consensus.cc:740] T 1d641f09429546d683f2884c02152f73 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:29.087174 19631 leader_election.cc:304] T 953d41f26f5f4027acfd374c6ec4ee75 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters: 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:29.088786 19631 leader_election.cc:304] T 0fdb45b31f93466c8bed1b848c6cdf15 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters: 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:29.089344 20748 raft_consensus.cc:2804] T 953d41f26f5f4027acfd374c6ec4ee75 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:29.089692 20748 raft_consensus.cc:493] T 953d41f26f5f4027acfd374c6ec4ee75 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:29.089860 20746 raft_consensus.cc:2804] T 0fdb45b31f93466c8bed1b848c6cdf15 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:29.090018 20748 raft_consensus.cc:3060] T 953d41f26f5f4027acfd374c6ec4ee75 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:29.090281 20746 raft_consensus.cc:697] T 0fdb45b31f93466c8bed1b848c6cdf15 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Becoming Leader. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Running, Role: LEADER
I20260925 13:56:29.091051 20746 consensus_queue.cc:237] T 0fdb45b31f93466c8bed1b848c6cdf15 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:29.091854 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3172d99afee4a4c878e71b645037917" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:29.091872 19691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23948807b2c4df9bf9a4da5f767e3b5" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:29.097962 19809 tablet_service.cc:1511] Processing CreateTablet for tablet 84188f52b54b477393d6a8433cbd1cb9 (DEFAULT_TABLE table=default.Table_29a7a24743f647108b2fa4629b620df5 [id=b3bd097a5199481290fe76162a2017a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:29.099099 19809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84188f52b54b477393d6a8433cbd1cb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:29.103399 20748 raft_consensus.cc:515] T 953d41f26f5f4027acfd374c6ec4ee75 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:29.104475 19802 tablet_service.cc:1511] Processing CreateTablet for tablet 197232acf0844b1d813e7699c2531b84 (DEFAULT_TABLE table=default.Table_5b3429bafb2248f99555eb27a2780ff6 [id=035f7129265e44d0bfd81afbf9f57f96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:29.105674 19802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 197232acf0844b1d813e7699c2531b84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:29.105697 19276 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_67306a96449b4a7bbb4b96498a396cea" } modify_external_catalogs: true
I20260925 13:56:29.106231 19276 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_67306a96449b4a7bbb4b96498a396cea" } modify_external_catalogs: true
I20260925 13:56:29.106909 19816 tablet_service.cc:1558] Processing DeleteTablet for tablet c5a9df833271480aa49feb7d9f925459 with delete_type TABLET_DATA_DELETED (replaced by tablet 236e308d8694405cb2b874164fa16485) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:29.107606 20773 tablet.cc:1751] T 9059655f2c3d4e86b3b389b5b9a68891 P 7c1b52895a14482fb54a6758f5054f8f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:29.120893 19938 ts_tablet_manager.cc:1434] T 7e544c93488541da9ebb588a465b27b7 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.107s user 0.007s sys 0.000s
I20260925 13:56:29.122068 19938 tablet_bootstrap.cc:492] T 82c1002d2bcb4b11839e2f4a93b899ad P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:29.075271 19942 consensus_queue.cc:260] T 1d641f09429546d683f2884c02152f73 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:29.144665 19773 leader_election.cc:343] T b3172d99afee4a4c878e71b645037917 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:29.083083 19805 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:51442 (request call id 222) took 1226 ms. Trace:
W20260925 13:56:29.145938 19776 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.634s user 0.014s sys 0.009s
W20260925 13:56:29.149680 19776 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.638s user 0.015s sys 0.009s
W20260925 13:56:29.145335 19773 leader_election.cc:343] T b23948807b2c4df9bf9a4da5f767e3b5 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:29.156387 19488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.194:38641, user_credentials={real_user=slave}} blocked reactor thread for 62999.2us
W20260925 13:56:29.157073 19488 leader_election.cc:343] T 5a9b3dfaa7474611a1824d63eb906b43 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: Tablet not RUNNING: STOPPED
I20260925 13:56:29.157419 19488 leader_election.cc:304] T 5a9b3dfaa7474611a1824d63eb906b43 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3; no voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:29.159323 20748 leader_election.cc:290] T 953d41f26f5f4027acfd374c6ec4ee75 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
W20260925 13:56:29.160008 19939 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.624s user 0.062s sys 0.094s
W20260925 13:56:29.160394 19939 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.626s user 0.062s sys 0.094s
I20260925 13:56:29.159541 19549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82972fa4868d4c59b3b9bb1240300938" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:29.168452 19942 ts_tablet_manager.cc:1434] T 1d641f09429546d683f2884c02152f73 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.187s user 0.006s sys 0.000s
I20260925 13:56:29.163820 20692 raft_consensus.cc:2749] T 5a9b3dfaa7474611a1824d63eb906b43 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:29.170194 19835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82972fa4868d4c59b3b9bb1240300938" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:29.170765 19835 raft_consensus.cc:3060] T 82972fa4868d4c59b3b9bb1240300938 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:29.145886 19805 rpcz_store.cc:276] 0925 13:56:27.856727 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:27.898976 (+ 42249us) service_pool.cc:224] Handling call
0925 13:56:27.930382 (+ 31406us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0925 13:56:27.930461 (+ 79us) ts_tablet_manager.cc:682] Creating new metadata...
0925 13:56:28.577207 (+646746us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:29.082121 (+504914us) tablet_replica.cc:206] Creating consensus instance
0925 13:56:29.082895 (+ 774us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260925 13:56:29.181537 19532 tablet_service.cc:1511] Processing CreateTablet for tablet 940d14678f89470c85371b5a2972ac3c (DEFAULT_TABLE table=default.Table_3c6b9bb73ad444eca6106c4aa3289fdc [id=925b603294c446779ce50e83a6955d06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:29.184206 19524 tablet_service.cc:1511] Processing CreateTablet for tablet 1dad030c86b54b7d87dd3b825c811640 (DEFAULT_TABLE table=default.Table_d5b57f5cf2574edcbe5bd91f38508763 [id=08eba7b5fc604580a8359e3cfb7d65fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:29.185628 19520 tablet_service.cc:1511] Processing CreateTablet for tablet f806a82ee59c4da79e2f30647adc3c60 (DEFAULT_TABLE table=default.Table_727855fadd224779a3638b48e6b0266f [id=4e57e9392b3f43e3a767b7c4b641d498]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:29.204129 19942 tablet_bootstrap.cc:492] T 321c740ba8eb4cbb8974305c5f8702af P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:29.173849 20011 ts_tablet_manager.cc:1929] T 5a9b3dfaa7474611a1824d63eb906b43 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:29.208068 20011 log.cc:1199] T 5a9b3dfaa7474611a1824d63eb906b43 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/5a9b3dfaa7474611a1824d63eb906b43
I20260925 13:56:29.208050 20773 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:45948 (request call id 239) took 1201 ms. Trace:
I20260925 13:56:29.209337 19532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 940d14678f89470c85371b5a2972ac3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:29.209962 20773 rpcz_store.cc:276] 0925 13:56:28.006729 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:28.006837 (+ 108us) service_pool.cc:224] Handling call
0925 13:56:29.207946 (+1201109us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:28.072419 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0925 13:56:28.072994 (+ 575us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0925 13:56:28.073032 (+ 38us) alter_schema_op.cc:66] Acquired schema lock
0925 13:56:28.073076 (+ 44us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0925 13:56:28.073183 (+ 107us) alter_schema_op.cc:128] START. Timestamp: 1790344588073131
0925 13:56:28.073204 (+ 21us) op_driver.cc:348] REPLICATION: starting
0925 13:56:28.073737 (+ 533us) log.cc:844] Serialized 173 byte log entry
0925 13:56:29.097343 (+1023606us) op_driver.cc:464] REPLICATION: finished
0925 13:56:29.107194 (+ 9851us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0925 13:56:29.109115 (+ 1921us) log.cc:844] Serialized 5 byte log entry
0925 13:56:29.193870 (+ 84755us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:29.194959 (+ 1089us) log.cc:844] Serialized 14 byte log entry
0925 13:56:29.207764 (+ 12805us) alter_schema_op.cc:73] Released schema lock
0925 13:56:29.207801 (+ 37us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":9625,"prepare.queue_time_us":64748,"prepare.run_cpu_time_us":1392,"prepare.run_wall_time_us":3559,"raft.queue_time_us":83,"raft.run_cpu_time_us":628,"raft.run_wall_time_us":629,"replication_time_us":1024016,"thread_start_us":132349,"threads_started":3,"wal-append.queue_time_us":57515}]]}
I20260925 13:56:29.214993 19271 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_ed1744bfe9644572b68ed9a8ccbb8d19"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:29.217549 19533 tablet_service.cc:1511] Processing CreateTablet for tablet 84188f52b54b477393d6a8433cbd1cb9 (DEFAULT_TABLE table=default.Table_29a7a24743f647108b2fa4629b620df5 [id=b3bd097a5199481290fe76162a2017a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:29.209194 20011 ts_tablet_manager.cc:1950] T 5a9b3dfaa7474611a1824d63eb906b43 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:29.218614 19533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84188f52b54b477393d6a8433cbd1cb9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:29.216774 19271 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ed1744bfe9644572b68ed9a8ccbb8d19 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:29.219769 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3172d99afee4a4c878e71b645037917" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:29.221082 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 235) took 1453 ms. Trace:
I20260925 13:56:29.221580 20011 rpcz_store.cc:276] 0925 13:56:27.767685 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:27.767795 (+ 110us) service_pool.cc:224] Handling call
0925 13:56:28.275083 (+507288us) ts_tablet_manager.cc:1151] Deleting tablet 5a9b3dfaa7474611a1824d63eb906b43
0925 13:56:28.275138 (+ 55us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:28.503510 (+228372us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:29.173703 (+670193us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:29.220994 (+ 47291us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":506280}
I20260925 13:56:29.224669 19277 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_ba7902001a1e48df84e25b9942bd98f9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:29.226271 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 5a9b3dfaa7474611a1824d63eb906b43 (table default.Table_67306a96449b4a7bbb4b96498a396cea [id=bcf9d7c5e4f44267b9204bc3c2d90584]) successfully deleted
I20260925 13:56:29.228153 20011 tablet_replica.cc:333] T 908df914cfec490cb7058dbe7b501c97 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:29.230141 20011 raft_consensus.cc:2243] T 908df914cfec490cb7058dbe7b501c97 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:29.232069 20011 raft_consensus.cc:2272] T 908df914cfec490cb7058dbe7b501c97 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:29.232934 19831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "953d41f26f5f4027acfd374c6ec4ee75" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
W20260925 13:56:29.234009 19277 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ba7902001a1e48df84e25b9942bd98f9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:29.235139 20011 ts_tablet_manager.cc:1916] T 908df914cfec490cb7058dbe7b501c97 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:29.241290 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23948807b2c4df9bf9a4da5f767e3b5" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
W20260925 13:56:29.244058 19629 leader_election.cc:343] T 82972fa4868d4c59b3b9bb1240300938 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:29.248279 19939 tablet_bootstrap.cc:492] T 0e8aa12e95dc434a98a426499593d9a5 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:29.248685 19939 ts_tablet_manager.cc:1403] T 0e8aa12e95dc434a98a426499593d9a5 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.722s user 0.068s sys 0.094s
I20260925 13:56:29.262564 19835 raft_consensus.cc:2468] T 82972fa4868d4c59b3b9bb1240300938 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
I20260925 13:56:29.255129 19939 raft_consensus.cc:359] T 0e8aa12e95dc434a98a426499593d9a5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:29.263971 19939 raft_consensus.cc:385] T 0e8aa12e95dc434a98a426499593d9a5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:29.264348 19939 raft_consensus.cc:740] T 0e8aa12e95dc434a98a426499593d9a5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:29.266063 19531 tablet_service.cc:1511] Processing CreateTablet for tablet 197232acf0844b1d813e7699c2531b84 (DEFAULT_TABLE table=default.Table_5b3429bafb2248f99555eb27a2780ff6 [id=035f7129265e44d0bfd81afbf9f57f96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:29.267211 19531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 197232acf0844b1d813e7699c2531b84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:29.208253 19551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "953d41f26f5f4027acfd374c6ec4ee75" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:29.269933 19551 raft_consensus.cc:3060] T 953d41f26f5f4027acfd374c6ec4ee75 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:29.270234 20011 ts_tablet_manager.cc:1929] T 908df914cfec490cb7058dbe7b501c97 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:29.270568 20011 log.cc:1199] T 908df914cfec490cb7058dbe7b501c97 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/908df914cfec490cb7058dbe7b501c97
I20260925 13:56:29.271178 20011 ts_tablet_manager.cc:1950] T 908df914cfec490cb7058dbe7b501c97 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:29.271705 19520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f806a82ee59c4da79e2f30647adc3c60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:29.277123 19524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1dad030c86b54b7d87dd3b825c811640. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:29.281461 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 236) took 1513 ms. Trace:
I20260925 13:56:29.281843 20011 rpcz_store.cc:276] 0925 13:56:27.767942 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:27.768130 (+ 188us) service_pool.cc:224] Handling call
0925 13:56:29.227920 (+1459790us) ts_tablet_manager.cc:1151] Deleting tablet 908df914cfec490cb7058dbe7b501c97
0925 13:56:29.227982 (+ 62us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:29.259733 (+ 31751us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:29.270117 (+ 10384us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:29.281389 (+ 11272us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1458771}
I20260925 13:56:29.283290 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 908df914cfec490cb7058dbe7b501c97 (table default.Table_c90b2c8024944acb9ffe76b506fff5a2 [id=d06e00bf26bd4c47a9e27f7eb35835df]) successfully deleted
I20260925 13:56:29.287698 20011 tablet_replica.cc:333] T 2711e7a6973546b1822d93f91095dae2 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:29.289233 20011 raft_consensus.cc:2243] T 2711e7a6973546b1822d93f91095dae2 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:29.290525 20011 raft_consensus.cc:2272] T 2711e7a6973546b1822d93f91095dae2 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:29.293015 20011 ts_tablet_manager.cc:1916] T 2711e7a6973546b1822d93f91095dae2 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 13:56:29.303923 19555 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.692s user 0.001s sys 0.015s
W20260925 13:56:29.304345 19555 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.693s user 0.002s sys 0.015s
W20260925 13:56:29.306150 19631 leader_election.cc:343] T 953d41f26f5f4027acfd374c6ec4ee75 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Illegal state: Tablet not RUNNING: STOPPED
I20260925 13:56:29.312803 19280 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 3aaf29a2310a41d0971eded7ef51fdc9
I20260925 13:56:29.314320 19280 catalog_manager.cc:6934] ReplaceTablet: tablet 3aaf29a2310a41d0971eded7ef51fdc9 deleted and replaced by tablet aea3670acb654810a05eda9ddd52e79b
I20260925 13:56:29.316339 19666 tablet_service.cc:1558] Processing DeleteTablet for tablet 3aaf29a2310a41d0971eded7ef51fdc9 with delete_type TABLET_DATA_DELETED (replaced by tablet aea3670acb654810a05eda9ddd52e79b) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:29.316738 19938 tablet_bootstrap.cc:654] T 82c1002d2bcb4b11839e2f4a93b899ad P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:29.315279 19280 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_b43a08d7f02f4b5b8ef48ab6d296efa0" } new_table_name: "default.Table_93a1c747f8d542578d106fa44b06b349" modify_external_catalogs: true
W20260925 13:56:29.320668 19525 consensus_meta.cc:287] T 4b36d5f94de04dca98b62e8128602cbb P 8f578869bc084a5d8a9a69f143954eb3: Time spent flushing consensus metadata: real 0.591s user 0.009s sys 0.004s
W20260925 13:56:29.324508 19273 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.194:34125 (request call id 44) took 848 ms (client timeout 1000 ms). Trace:
W20260925 13:56:29.324893 19273 rpcz_store.cc:269] 0925 13:56:28.476211 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:28.480541 (+ 4330us) service_pool.cc:224] Handling call
0925 13:56:28.850684 (+370143us) spinlock_profiling.cc:243] Waited 365 ms on lock 0x7b5400040454. stack: 00007fdf0a30b323 00007fdf0a30b220 00007fdf09eaf574 0000000000516964 00000000005168f9 00007fdf114095c0 00007fdf114a86fa 00007fdf13219d8c 00007fdf1331801e 00007fdf10ebcfb4 00007fdf10ebcf42 00007fdf10ebce71 00007fdf10ebcdec 00007fdf10ebc0a2 00007fdf0f16e09c 00007fdf0f16d4d6
0925 13:56:29.324455 (+473771us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":3,"spinlock_wait_cycles":838560512}
I20260925 13:56:29.325959 19666 tablet_service.cc:1558] Processing DeleteTablet for tablet 953d41f26f5f4027acfd374c6ec4ee75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:22 UTC) from {username='slave'} at 127.0.0.1:45948
W20260925 13:56:29.333171 19529 consensus_meta.cc:287] T 96213c58b3bb4a8c8760da3da73d4bbe P 8f578869bc084a5d8a9a69f143954eb3: Time spent flushing consensus metadata: real 0.622s user 0.009s sys 0.000s
W20260925 13:56:29.336519 19521 consensus_meta.cc:287] T b45c85b8ff7c4b8f8cc8bab6fe20780b P 8f578869bc084a5d8a9a69f143954eb3: Time spent flushing consensus metadata: real 0.643s user 0.012s sys 0.000s
I20260925 13:56:29.338563 20730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:40018 (request call id 243) took 1153 ms. Trace:
W20260925 13:56:29.344136 19265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.254:40825, user_credentials={real_user=slave}} blocked reactor thread for 206443us
I20260925 13:56:29.341193 19939 consensus_queue.cc:260] T 0e8aa12e95dc434a98a426499593d9a5 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:29.345301 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 161) took 7587 ms. Trace:
I20260925 13:56:29.347754 19955 rpcz_store.cc:276] 0925 13:56:21.757615 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:21.787433 (+ 29818us) service_pool.cc:224] Handling call
0925 13:56:27.809674 (+6022241us) ts_tablet_manager.cc:1151] Deleting tablet 2711e7a6973546b1822d93f91095dae2
0925 13:56:27.809713 (+ 39us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:28.321715 (+512002us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:28.990542 (+668827us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:29.345244 (+354702us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":289104,"tablet-delete.queue_time_us":6020944}
I20260925 13:56:29.349205 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet 2711e7a6973546b1822d93f91095dae2 (table default.Table_83acf26db75e4c02b312cef4df19901f [id=7ec3d38b986747ca99d6a49e6a7dbe1a]) successfully deleted
I20260925 13:56:29.350170 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 162) took 7530 ms. Trace:
I20260925 13:56:29.350507 19955 rpcz_store.cc:276] 0925 13:56:21.820044 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:21.824279 (+ 4235us) service_pool.cc:224] Handling call
0925 13:56:29.349855 (+7525576us) ts_tablet_manager.cc:1151] Deleting tablet 78d8c01c88d24deeae21d647e7f205ba
0925 13:56:29.349895 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:29.350121 (+ 226us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7524947}
I20260925 13:56:29.350921 20011 ts_tablet_manager.cc:1929] T 2711e7a6973546b1822d93f91095dae2 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260925 13:56:29.352010 19264 catalog_manager.cc:5043] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): delete failed for tablet 78d8c01c88d24deeae21d647e7f205ba because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 78d8c01c88d24deeae21d647e7f205ba already in progress: creating tablet
I20260925 13:56:29.351274 20011 log.cc:1199] T 2711e7a6973546b1822d93f91095dae2 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/2711e7a6973546b1822d93f91095dae2
I20260925 13:56:29.355715 20011 ts_tablet_manager.cc:1950] T 2711e7a6973546b1822d93f91095dae2 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:29.361397 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 237) took 1590 ms. Trace:
I20260925 13:56:29.362152 20011 rpcz_store.cc:276] 0925 13:56:27.769405 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:27.769494 (+ 89us) service_pool.cc:224] Handling call
0925 13:56:29.287510 (+1518016us) ts_tablet_manager.cc:1151] Deleting tablet 2711e7a6973546b1822d93f91095dae2
0925 13:56:29.287563 (+ 53us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:29.336413 (+ 48850us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:29.350803 (+ 14390us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:29.359318 (+ 8515us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1517556}
I20260925 13:56:29.363415 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 2711e7a6973546b1822d93f91095dae2 (table default.Table_83acf26db75e4c02b312cef4df19901f [id=7ec3d38b986747ca99d6a49e6a7dbe1a]) successfully deleted
I20260925 13:56:29.365373 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 164) took 7129 ms. Trace:
I20260925 13:56:29.366173 20011 tablet_replica.cc:333] T 3e6c8b9bbf4549b3a8685f886973ed9c P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:29.365800 19955 rpcz_store.cc:276] 0925 13:56:22.236214 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:22.243487 (+ 7273us) service_pool.cc:224] Handling call
0925 13:56:29.363105 (+7119618us) ts_tablet_manager.cc:1151] Deleting tablet 4d01a791445149e7bee9493e1eec5c6b
0925 13:56:29.363146 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:29.365300 (+ 2154us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7118922}
I20260925 13:56:29.367573 20011 raft_consensus.cc:2243] T 3e6c8b9bbf4549b3a8685f886973ed9c P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:29.369259 20011 raft_consensus.cc:2272] T 3e6c8b9bbf4549b3a8685f886973ed9c P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:29.370783 20748 raft_consensus.cc:493] T 1d641f09429546d683f2884c02152f73 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:29.371197 20748 raft_consensus.cc:515] T 1d641f09429546d683f2884c02152f73 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:29.373142 20748 leader_election.cc:290] T 1d641f09429546d683f2884c02152f73 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:29.374050 20011 ts_tablet_manager.cc:1916] T 3e6c8b9bbf4549b3a8685f886973ed9c P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:29.377597 20735 tablet.cc:1751] T f16050752ad44720ba8703ef10bbb359 P 7c1b52895a14482fb54a6758f5054f8f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:29.380111 19525 tablet_service.cc:1558] Processing DeleteTablet for tablet 3aaf29a2310a41d0971eded7ef51fdc9 with delete_type TABLET_DATA_DELETED (replaced by tablet aea3670acb654810a05eda9ddd52e79b) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:29.383951 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d641f09429546d683f2884c02152f73" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
W20260925 13:56:29.386250 19629 leader_election.cc:343] T 1d641f09429546d683f2884c02152f73 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:29.400040 19264 catalog_manager.cc:5043] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): delete failed for tablet 4d01a791445149e7bee9493e1eec5c6b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4d01a791445149e7bee9493e1eec5c6b already in progress: creating tablet
W20260925 13:56:29.400547 19773 leader_election.cc:343] T b3172d99afee4a4c878e71b645037917 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:29.400930 19773 leader_election.cc:304] T b3172d99afee4a4c878e71b645037917 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:29.346225 20730 rpcz_store.cc:276] 0925 13:56:28.184997 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:28.210836 (+ 25839us) service_pool.cc:224] Handling call
0925 13:56:29.338506 (+1127670us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:28.336508 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0925 13:56:28.336866 (+ 358us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0925 13:56:28.336894 (+ 28us) alter_schema_op.cc:66] Acquired schema lock
0925 13:56:28.336935 (+ 41us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0925 13:56:28.337050 (+ 115us) alter_schema_op.cc:128] START. Timestamp: 1790344588328623
0925 13:56:28.337068 (+ 18us) op_driver.cc:348] REPLICATION: starting
0925 13:56:28.347211 (+ 10143us) log.cc:844] Serialized 173 byte log entry
0925 13:56:28.447981 (+100770us) op_driver.cc:464] REPLICATION: finished
0925 13:56:28.471240 (+ 23259us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0925 13:56:28.621492 (+150252us) log.cc:844] Serialized 5 byte log entry
0925 13:56:29.337223 (+715731us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:29.337835 (+ 612us) log.cc:844] Serialized 14 byte log entry
0925 13:56:29.338396 (+ 561us) alter_schema_op.cc:73] Released schema lock
0925 13:56:29.338425 (+ 29us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":23078,"mutex_wait_us":60252,"prepare.queue_time_us":82922,"prepare.run_cpu_time_us":2817,"prepare.run_wall_time_us":109267,"raft.queue_time_us":60652,"raft.run_cpu_time_us":9061,"raft.run_wall_time_us":86711,"replication_time_us":110824,"thread_start_us":211667,"threads_started":3,"wal-append.queue_time_us":98881}]]}
I20260925 13:56:29.410856 20730 tablet.cc:1751] T f16050752ad44720ba8703ef10bbb359 P 8f578869bc084a5d8a9a69f143954eb3: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:29.413127 19955 tablet_replica.cc:333] T 2ef3efe644364e93a089ddeed328389d P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:29.413870 19955 raft_consensus.cc:2243] T 2ef3efe644364e93a089ddeed328389d P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:29.414299 19955 raft_consensus.cc:2272] T 2ef3efe644364e93a089ddeed328389d P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 13:56:29.416105 19773 leader_election.cc:343] T b23948807b2c4df9bf9a4da5f767e3b5 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:29.416487 19773 leader_election.cc:304] T b23948807b2c4df9bf9a4da5f767e3b5 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:29.417930 20604 raft_consensus.cc:493] T 000aad39953c4688b543c7d882a08ec8 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:29.418354 20604 raft_consensus.cc:515] T 000aad39953c4688b543c7d882a08ec8 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:29.420347 20731 raft_consensus.cc:2749] T b3172d99afee4a4c878e71b645037917 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:29.421406 19830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d641f09429546d683f2884c02152f73" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:29.421991 19830 raft_consensus.cc:2468] T 1d641f09429546d683f2884c02152f73 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:29.432471 20604 leader_election.cc:290] T 000aad39953c4688b543c7d882a08ec8 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:29.451998 20011 ts_tablet_manager.cc:1929] T 3e6c8b9bbf4549b3a8685f886973ed9c P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:56:29.452338 20011 log.cc:1199] T 3e6c8b9bbf4549b3a8685f886973ed9c P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/3e6c8b9bbf4549b3a8685f886973ed9c
I20260925 13:56:29.452983 20011 ts_tablet_manager.cc:1950] T 3e6c8b9bbf4549b3a8685f886973ed9c P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:29.454805 20011 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45948 (request call id 238) took 1526 ms. Trace:
I20260925 13:56:29.456703 19691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "000aad39953c4688b543c7d882a08ec8" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:29.456848 19819 tablet_service.cc:1558] Processing DeleteTablet for tablet 3aaf29a2310a41d0971eded7ef51fdc9 with delete_type TABLET_DATA_DELETED (replaced by tablet aea3670acb654810a05eda9ddd52e79b) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:29.457062 19942 tablet_bootstrap.cc:654] T 321c740ba8eb4cbb8974305c5f8702af P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:29.458300 20785 tablet.cc:1751] T 9059655f2c3d4e86b3b389b5b9a68891 P f790493ba7ad403993b678f2df2b55be: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:29.455111 20011 rpcz_store.cc:276] 0925 13:56:27.928385 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:27.928500 (+ 115us) service_pool.cc:224] Handling call
0925 13:56:29.366006 (+1437506us) ts_tablet_manager.cc:1151] Deleting tablet 3e6c8b9bbf4549b3a8685f886973ed9c
0925 13:56:29.366061 (+ 55us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:29.421044 (+ 54983us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:29.451873 (+ 30829us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:29.454752 (+ 2879us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1436736}
I20260925 13:56:29.461035 20604 raft_consensus.cc:2749] T b23948807b2c4df9bf9a4da5f767e3b5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:29.461362 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 3e6c8b9bbf4549b3a8685f886973ed9c (table default.Table_d0dadd9b31b143fd8dd5adcb7f752c5f [id=beee0ac537e44859b0f7c76d6808b0e0]) successfully deleted
I20260925 13:56:29.462195 20011 tablet_replica.cc:333] T 3aaf29a2310a41d0971eded7ef51fdc9 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:29.462867 20011 raft_consensus.cc:2243] T 3aaf29a2310a41d0971eded7ef51fdc9 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:29.463413 20011 raft_consensus.cc:2272] T 3aaf29a2310a41d0971eded7ef51fdc9 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:29.470206 19272 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 8b858870f1254e72ae2af17c8192ffa0
I20260925 13:56:29.471736 19272 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:39288 (request call id 292) took 1644 ms. Trace:
I20260925 13:56:29.473023 20011 ts_tablet_manager.cc:1916] T 3aaf29a2310a41d0971eded7ef51fdc9 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:29.472030 19272 rpcz_store.cc:276] 0925 13:56:27.827831 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:27.827964 (+ 133us) service_pool.cc:224] Handling call
0925 13:56:27.829121 (+ 1157us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 13:56:27.829666 (+ 545us) catalog_manager.cc:2854] Modifying in-memory table state
0925 13:56:27.829779 (+ 113us) catalog_manager.cc:2863] Locking tablets
0925 13:56:28.443874 (+614095us) catalog_manager.cc:2877] Removing table and tablets from system table
0925 13:56:29.469821 (+1025947us) catalog_manager.cc:2895] Removing table from by-name map
0925 13:56:29.470021 (+ 200us) catalog_manager.cc:2911] Committing in-memory state
0925 13:56:29.470107 (+ 86us) catalog_manager.cc:2915] Aborting table tasks
0925 13:56:29.471682 (+ 1575us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:29.148133 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:29.148554 (+ 421us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:29.148584 (+ 30us) write_op.cc:435] Acquired schema lock
0925 13:56:29.148603 (+ 19us) tablet.cc:662] Decoding operations
0925 13:56:29.148930 (+ 327us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:29.148978 (+ 48us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:29.149004 (+ 26us) tablet.cc:685] Acquiring locks for 2 operations
0925 13:56:29.149317 (+ 313us) tablet.cc:701] Row locks acquired
0925 13:56:29.149338 (+ 21us) write_op.cc:260] PREPARE: finished
0925 13:56:29.149462 (+ 124us) write_op.cc:270] Start()
0925 13:56:29.149557 (+ 95us) write_op.cc:276] Timestamp: P: 1790344589149421 usec, L: 0
0925 13:56:29.149576 (+ 19us) op_driver.cc:348] REPLICATION: starting
0925 13:56:29.150193 (+ 617us) log.cc:844] Serialized 734 byte log entry
0925 13:56:29.467725 (+317532us) op_driver.cc:464] REPLICATION: finished
0925 13:56:29.467963 (+ 238us) write_op.cc:301] APPLY: starting
0925 13:56:29.468026 (+ 63us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:29.468127 (+ 101us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:29.468210 (+ 83us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:29.468453 (+ 243us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:29.468589 (+ 136us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 13:56:29.468628 (+ 39us) write_op.cc:312] APPLY: finished
0925 13:56:29.468800 (+ 172us) log.cc:844] Serialized 29 byte log entry
0925 13:56:29.468951 (+ 151us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:29.469013 (+ 62us) write_op.cc:454] Released schema lock
0925 13:56:29.469048 (+ 35us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":91,"apply.run_cpu_time_us":1270,"apply.run_wall_time_us":1270,"num_ops":2,"prepare.queue_time_us":702616,"prepare.run_cpu_time_us":2335,"prepare.run_wall_time_us":3494,"raft.queue_time_us":24623,"raft.run_cpu_time_us":21,"raft.run_wall_time_us":23,"replication_time_us":318024}]]}
I20260925 13:56:29.477389 19666 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b858870f1254e72ae2af17c8192ffa0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:27 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:29.480754 19955 ts_tablet_manager.cc:1916] T 2ef3efe644364e93a089ddeed328389d P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:29.481982 19275 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 61f141c40ce64fbcb68e01ac15009e3c
I20260925 13:56:29.483264 19275 catalog_manager.cc:6934] ReplaceTablet: tablet 61f141c40ce64fbcb68e01ac15009e3c deleted and replaced by tablet f5791a011b104ec59721b6ed8ce773f1
I20260925 13:56:29.483594 19275 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:39288 (request call id 303) took 1008 ms. Trace:
I20260925 13:56:29.483860 19275 rpcz_store.cc:276] 0925 13:56:28.475120 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:28.510702 (+ 35582us) service_pool.cc:224] Handling call
0925 13:56:28.815283 (+304581us) spinlock_profiling.cc:243] Waited 290 ms on lock 0x7b5400040454. stack: 00007fdf0a30b323 00007fdf0a30b220 00007fdf09eaf574 0000000000516964 00000000005168f9 00007fdf114095c0 00007fdf114a86fa 00007fdf13219d8c 00007fdf1331c0e1 00007fdf10ee4d37 00007fdf10ee4cc2 00007fdf10ee4bf1 00007fdf10ee4b6c 00007fdf10ee3e22 00007fdf0f16e09c 00007fdf0f16d4d6
0925 13:56:29.483542 (+668259us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:29.151724 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:29.152143 (+ 419us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:29.152168 (+ 25us) write_op.cc:435] Acquired schema lock
0925 13:56:29.152182 (+ 14us) tablet.cc:662] Decoding operations
0925 13:56:29.152468 (+ 286us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:29.152521 (+ 53us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:29.152557 (+ 36us) tablet.cc:685] Acquiring locks for 2 operations
0925 13:56:29.152834 (+ 277us) tablet.cc:701] Row locks acquired
0925 13:56:29.152853 (+ 19us) write_op.cc:260] PREPARE: finished
0925 13:56:29.152969 (+ 116us) write_op.cc:270] Start()
0925 13:56:29.153054 (+ 85us) write_op.cc:276] Timestamp: P: 1790344589152923 usec, L: 0
0925 13:56:29.153068 (+ 14us) op_driver.cc:348] REPLICATION: starting
0925 13:56:29.153546 (+ 478us) log.cc:844] Serialized 631 byte log entry
0925 13:56:29.478497 (+324951us) op_driver.cc:464] REPLICATION: finished
0925 13:56:29.478728 (+ 231us) write_op.cc:301] APPLY: starting
0925 13:56:29.478799 (+ 71us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:29.478947 (+ 148us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:29.479051 (+ 104us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:29.479487 (+ 436us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:29.480094 (+ 607us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 13:56:29.480143 (+ 49us) write_op.cc:312] APPLY: finished
0925 13:56:29.480295 (+ 152us) log.cc:844] Serialized 29 byte log entry
0925 13:56:29.480851 (+ 556us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:29.480906 (+ 55us) write_op.cc:454] Released schema lock
0925 13:56:29.480935 (+ 29us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":456,"spinlock_wait_cycles":667087232,"child_traces":[["op",{"apply.queue_time_us":89,"apply.run_cpu_time_us":1663,"apply.run_wall_time_us":2853,"num_ops":2,"prepare.queue_time_us":329082,"prepare.run_cpu_time_us":2071,"prepare.run_wall_time_us":198499,"replication_time_us":325325}]]}
I20260925 13:56:29.488683 19666 tablet_service.cc:1558] Processing DeleteTablet for tablet 61f141c40ce64fbcb68e01ac15009e3c with delete_type TABLET_DATA_DELETED (replaced by tablet f5791a011b104ec59721b6ed8ce773f1) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:29.493716 19631 leader_election.cc:304] T 1d641f09429546d683f2884c02152f73 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters: 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:29.494647 20746 raft_consensus.cc:2804] T 1d641f09429546d683f2884c02152f73 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:29.494959 20746 raft_consensus.cc:493] T 1d641f09429546d683f2884c02152f73 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:29.495257 20746 raft_consensus.cc:3060] T 1d641f09429546d683f2884c02152f73 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:29.500514 19551 raft_consensus.cc:2468] T 953d41f26f5f4027acfd374c6ec4ee75 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
I20260925 13:56:29.502032 20723 raft_consensus.cc:493] T 0e8aa12e95dc434a98a426499593d9a5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:29.502425 20746 raft_consensus.cc:515] T 1d641f09429546d683f2884c02152f73 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:29.502522 20723 raft_consensus.cc:515] T 0e8aa12e95dc434a98a426499593d9a5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:29.504876 19525 tablet_service.cc:1558] Processing DeleteTablet for tablet 61f141c40ce64fbcb68e01ac15009e3c with delete_type TABLET_DATA_DELETED (replaced by tablet f5791a011b104ec59721b6ed8ce773f1) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:29.505400 20746 leader_election.cc:290] T 1d641f09429546d683f2884c02152f73 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:29.506996 19631 leader_election.cc:304] T 82972fa4868d4c59b3b9bb1240300938 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters: 8f578869bc084a5d8a9a69f143954eb3
W20260925 13:56:29.507688 19279 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet eff33a56361946edb556ac611d2299eb consensus state has no leader
I20260925 13:56:29.508930 20748 raft_consensus.cc:2804] T 82972fa4868d4c59b3b9bb1240300938 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:29.509352 20748 raft_consensus.cc:697] T 82972fa4868d4c59b3b9bb1240300938 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Becoming Leader. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Running, Role: LEADER
I20260925 13:56:29.510090 20748 consensus_queue.cc:237] T 82972fa4868d4c59b3b9bb1240300938 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:29.511669 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "000aad39953c4688b543c7d882a08ec8" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
W20260925 13:56:29.525323 19488 consensus_peers.cc:597] T 3aaf29a2310a41d0971eded7ef51fdc9 P 8f578869bc084a5d8a9a69f143954eb3 -> Peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Couldn't send request to peer 7c1b52895a14482fb54a6758f5054f8f. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:29.526979 19939 ts_tablet_manager.cc:1434] T 0e8aa12e95dc434a98a426499593d9a5 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.278s user 0.008s sys 0.011s
I20260925 13:56:29.546674 19939 tablet_bootstrap.cc:492] T 9059655f2c3d4e86b3b389b5b9a68891 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
W20260925 13:56:29.537905 19773 leader_election.cc:343] T 000aad39953c4688b543c7d882a08ec8 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:29.562871 20011 ts_tablet_manager.cc:1929] T 3aaf29a2310a41d0971eded7ef51fdc9 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:56:29.567510 20011 log.cc:1199] T 3aaf29a2310a41d0971eded7ef51fdc9 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/3aaf29a2310a41d0971eded7ef51fdc9
I20260925 13:56:29.568347 20011 ts_tablet_manager.cc:1950] T 3aaf29a2310a41d0971eded7ef51fdc9 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
W20260925 13:56:29.570907 19488 leader_election.cc:343] T 0e8aa12e95dc434a98a426499593d9a5 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Not found: Tablet not found: 0e8aa12e95dc434a98a426499593d9a5
I20260925 13:56:29.571921 19532 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b858870f1254e72ae2af17c8192ffa0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:27 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:29.576116 20735 tablet.cc:1751] T 0ad6442b2de249119ba0ef85dfc06da9 P 7c1b52895a14482fb54a6758f5054f8f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:29.571750 20723 leader_election.cc:290] T 0e8aa12e95dc434a98a426499593d9a5 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:29.583133 19629 leader_election.cc:304] T 953d41f26f5f4027acfd374c6ec4ee75 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3; no voters: f790493ba7ad403993b678f2df2b55be
I20260925 13:56:29.585686 19939 tablet_bootstrap.cc:654] T 9059655f2c3d4e86b3b389b5b9a68891 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:29.585656 20749 raft_consensus.cc:2804] T 953d41f26f5f4027acfd374c6ec4ee75 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:29.586585 20749 raft_consensus.cc:697] T 953d41f26f5f4027acfd374c6ec4ee75 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Becoming Leader. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Running, Role: LEADER
I20260925 13:56:29.587241 20735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:45948 (request call id 248) took 1008 ms. Trace:
I20260925 13:56:29.587287 20749 consensus_queue.cc:237] T 953d41f26f5f4027acfd374c6ec4ee75 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:29.587710 20735 rpcz_store.cc:276] 0925 13:56:28.578881 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:28.578980 (+ 99us) service_pool.cc:224] Handling call
0925 13:56:29.587172 (+1008192us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:28.927835 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0925 13:56:28.928265 (+ 430us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0925 13:56:28.928302 (+ 37us) alter_schema_op.cc:66] Acquired schema lock
0925 13:56:28.928355 (+ 53us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0925 13:56:28.928509 (+ 154us) alter_schema_op.cc:128] START. Timestamp: 1790344588928426
0925 13:56:28.928533 (+ 24us) op_driver.cc:348] REPLICATION: starting
0925 13:56:28.929202 (+ 669us) log.cc:844] Serialized 173 byte log entry
0925 13:56:29.575734 (+646532us) op_driver.cc:464] REPLICATION: finished
0925 13:56:29.576036 (+ 302us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0925 13:56:29.577270 (+ 1234us) log.cc:844] Serialized 5 byte log entry
0925 13:56:29.585663 (+ 8393us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:29.586399 (+ 736us) log.cc:844] Serialized 14 byte log entry
0925 13:56:29.587037 (+ 638us) alter_schema_op.cc:73] Released schema lock
0925 13:56:29.587064 (+ 27us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":113,"prepare.queue_time_us":345792,"prepare.run_cpu_time_us":2180,"prepare.run_wall_time_us":110176,"raft.queue_time_us":103,"raft.run_cpu_time_us":1126,"raft.run_wall_time_us":1126,"replication_time_us":647053,"thread_start_us":475752,"threads_started":2,"wal-append.queue_time_us":105065}]]}
I20260925 13:56:29.591462 20810 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.182.252:36181 every 8 connection(s)
I20260925 13:56:29.593758 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 3aaf29a2310a41d0971eded7ef51fdc9 (table default.Table_95a1130b07a34c82a6a6d7bb1c8245c8 [id=f4beec04dea942f98007b1f4cf33c9ec]) successfully deleted
I20260925 13:56:29.591957 20083 rpc_server.cc:307] RPC server started. Bound to: 127.18.182.252:36181
I20260925 13:56:29.597993 20083 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/master-2/data/info.pb
I20260925 13:56:29.605213 19163 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskYmJttM/build/tsan/bin/kudu as pid 20083
I20260925 13:56:29.608071 20011 tablet_replica.cc:333] T 953d41f26f5f4027acfd374c6ec4ee75 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:29.625892 19939 tablet_bootstrap.cc:492] T 9059655f2c3d4e86b3b389b5b9a68891 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:29.626312 19939 ts_tablet_manager.cc:1403] T 9059655f2c3d4e86b3b389b5b9a68891 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.099s user 0.007s sys 0.004s
I20260925 13:56:29.629328 19939 raft_consensus.cc:359] T 9059655f2c3d4e86b3b389b5b9a68891 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:29.629973 19939 raft_consensus.cc:385] T 9059655f2c3d4e86b3b389b5b9a68891 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:29.630231 19939 raft_consensus.cc:740] T 9059655f2c3d4e86b3b389b5b9a68891 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:29.630534 20011 raft_consensus.cc:2243] T 953d41f26f5f4027acfd374c6ec4ee75 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:29.631642 19939 consensus_queue.cc:260] T 9059655f2c3d4e86b3b389b5b9a68891 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:29.632609 20011 pending_rounds.cc:70] T 953d41f26f5f4027acfd374c6ec4ee75 P 7c1b52895a14482fb54a6758f5054f8f: Trying to abort 1 pending ops.
I20260925 13:56:29.632833 20011 pending_rounds.cc:77] T 953d41f26f5f4027acfd374c6ec4ee75 P 7c1b52895a14482fb54a6758f5054f8f: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251439091970048 op_type: NO_OP noop_request { }
I20260925 13:56:29.633132 20011 raft_consensus.cc:2889] T 953d41f26f5f4027acfd374c6ec4ee75 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:56:29.633437 20011 raft_consensus.cc:2272] T 953d41f26f5f4027acfd374c6ec4ee75 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:29.633945 19939 ts_tablet_manager.cc:1434] T 9059655f2c3d4e86b3b389b5b9a68891 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.007s user 0.002s sys 0.005s
I20260925 13:56:29.634795 19939 tablet_bootstrap.cc:492] T 1215d875209a40d38cc527e2795d6c75 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:29.637354 19549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d641f09429546d683f2884c02152f73" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
W20260925 13:56:29.652951 19629 leader_election.cc:343] T 1d641f09429546d683f2884c02152f73 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:29.656421 20796 tablet.cc:1751] T 0ad6442b2de249119ba0ef85dfc06da9 P 8f578869bc084a5d8a9a69f143954eb3: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:29.656881 19939 tablet_bootstrap.cc:654] T 1215d875209a40d38cc527e2795d6c75 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:29.665697 20804 raft_consensus.cc:493] T 7e544c93488541da9ebb588a465b27b7 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:29.666203 20804 raft_consensus.cc:515] T 7e544c93488541da9ebb588a465b27b7 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:29.668478 20804 leader_election.cc:290] T 7e544c93488541da9ebb588a465b27b7 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:29.670066 20011 ts_tablet_manager.cc:1916] T 953d41f26f5f4027acfd374c6ec4ee75 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:29.675539 19276 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet a218f033a132482a82bd1f60126fa0f9
I20260925 13:56:29.679628 19520 tablet_service.cc:1558] Processing DeleteTablet for tablet 5a9b3dfaa7474611a1824d63eb906b43 with delete_type TABLET_DATA_DELETED (replaced by tablet a218f033a132482a82bd1f60126fa0f9) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:29.680748 19666 tablet_service.cc:1558] Processing DeleteTablet for tablet a218f033a132482a82bd1f60126fa0f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:29 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:29.690577 20814 sys_catalog.cc:263] Verifying existing consensus state
I20260925 13:56:29.697024 19271 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_750d58e5a30a4b43ba8d84fb7821273e" } modify_external_catalogs: true
I20260925 13:56:29.697566 19271 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_750d58e5a30a4b43ba8d84fb7821273e" } modify_external_catalogs: true
I20260925 13:56:29.699677 19533 tablet_service.cc:1558] Processing DeleteTablet for tablet a218f033a132482a82bd1f60126fa0f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:29 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:29.700738 19939 tablet_bootstrap.cc:492] T 1215d875209a40d38cc527e2795d6c75 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:29.701169 19939 ts_tablet_manager.cc:1403] T 1215d875209a40d38cc527e2795d6c75 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.067s user 0.007s sys 0.003s
I20260925 13:56:29.701719 19837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d641f09429546d683f2884c02152f73" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:29.702338 19837 raft_consensus.cc:3060] T 1d641f09429546d683f2884c02152f73 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:29.703709 19939 raft_consensus.cc:359] T 1215d875209a40d38cc527e2795d6c75 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:29.704276 19939 raft_consensus.cc:385] T 1215d875209a40d38cc527e2795d6c75 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:29.704511 19939 raft_consensus.cc:740] T 1215d875209a40d38cc527e2795d6c75 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:29.704743 19276 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_2d803b07af1d41a789c69141ec7f60f7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:29.705305 19955 ts_tablet_manager.cc:1929] T 2ef3efe644364e93a089ddeed328389d P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:29.705641 19955 log.cc:1199] T 2ef3efe644364e93a089ddeed328389d P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/2ef3efe644364e93a089ddeed328389d
I20260925 13:56:29.706270 19955 ts_tablet_manager.cc:1950] T 2ef3efe644364e93a089ddeed328389d P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
W20260925 13:56:29.706176 19276 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2d803b07af1d41a789c69141ec7f60f7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 13:56:29.707078 19490 leader_election.cc:343] T 0e8aa12e95dc434a98a426499593d9a5 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 0e8aa12e95dc434a98a426499593d9a5
I20260925 13:56:29.707458 19490 leader_election.cc:304] T 0e8aa12e95dc434a98a426499593d9a5 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3; no voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:29.708151 19939 consensus_queue.cc:260] T 1215d875209a40d38cc527e2795d6c75 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:29.708967 20794 raft_consensus.cc:2749] T 0e8aa12e95dc434a98a426499593d9a5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:29.709944 19275 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_caeed989c841465a9b384a6641618002"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:29.711298 19275 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_caeed989c841465a9b384a6641618002 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:29.715313 19279 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_6f8114f6052845bb85b139bfedbed143" } new_table_name: "default.Table_2efda975ab0a464eaf96f1f9a48eff1c" modify_external_catalogs: true
I20260925 13:56:29.716233 19939 ts_tablet_manager.cc:1434] T 1215d875209a40d38cc527e2795d6c75 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.015s user 0.006s sys 0.001s
I20260925 13:56:29.726388 19942 tablet_bootstrap.cc:492] T 321c740ba8eb4cbb8974305c5f8702af P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:29.733595 19600 heartbeater.cc:344] Connected to a master server at 127.18.182.252:36181
I20260925 13:56:29.750980 20814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728: Bootstrap starting.
I20260925 13:56:29.772315 19939 tablet_bootstrap.cc:492] T f50ea68e3b3c4459b3f5e5b0df724724 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:29.778055 19939 tablet_bootstrap.cc:654] T f50ea68e3b3c4459b3f5e5b0df724724 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:29.779330 19274 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet 9059655f2c3d4e86b3b389b5b9a68891 (table default.Table_1b1f36048d2a4a3c86e513e46bcf5471 [id=0c13bfd171f2464b887f15d085439fa3]). Expected version 1 got 0
I20260925 13:56:29.738546 19741 heartbeater.cc:344] Connected to a master server at 127.18.182.252:36181
I20260925 13:56:29.791610 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 166) took 7437 ms. Trace:
I20260925 13:56:29.792374 19549 raft_consensus.cc:3060] T 9059655f2c3d4e86b3b389b5b9a68891 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:29.791918 19955 rpcz_store.cc:276] 0925 13:56:22.354397 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:22.354685 (+ 288us) service_pool.cc:224] Handling call
0925 13:56:29.402199 (+7047514us) ts_tablet_manager.cc:1151] Deleting tablet 2ef3efe644364e93a089ddeed328389d
0925 13:56:29.412969 (+ 10770us) spinlock_profiling.cc:243] Waited 1.07 ms on lock 0x7b5800000f98. stack: 00007f1358052323 00007f1358052220 00007f1357bf6574 0000000000516964 00000000005168f9 00007f1357f088a1 00007f135804cb51 00007f135804d182 00007f1360fc02c9 00007f13601d1918 00007f13602982c9 00007f13602988c3 00007f13602c9960 00007f13602afe6d 00007f13602afe19 00007f13602afda9
0925 13:56:29.413014 (+ 45us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:29.643384 (+230370us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:29.705192 (+ 61808us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:29.763482 (+ 58290us) spinlock_profiling.cc:243] Waited 28.7 ms on lock 0x7b5800000f98. stack: 00007f1358052323 00007f1358052220 00007f1357bf6574 0000000000516964 00000000005168f9 00007f1357f088a1 00007f135804cb51 00007f135804d182 00007f1360fc02c9 00007f13601d1918 00007f1360298d82 00007f13602c9960 00007f13602afe6d 00007f13602afe19 00007f13602afda9 00007f13602afd71
0925 13:56:29.791557 (+ 28075us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":68542720,"tablet-delete.queue_time_us":7046853}
I20260925 13:56:29.793359 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet 2ef3efe644364e93a089ddeed328389d (table default.Table_dfe9d9863d364dfc8df4c405e87dd635 [id=7b4bc216530346b6ab44c1ce25fe2112]) successfully deleted
I20260925 13:56:29.794232 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 169) took 7380 ms. Trace:
I20260925 13:56:29.794513 19955 rpcz_store.cc:276] 0925 13:56:22.414124 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:22.414235 (+ 111us) service_pool.cc:224] Handling call
0925 13:56:29.794024 (+7379789us) ts_tablet_manager.cc:1151] Deleting tablet dc14c37aa393407a87af0d0eb8b88d82
0925 13:56:29.794061 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:29.794191 (+ 130us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7379239}
W20260925 13:56:29.795626 19264 catalog_manager.cc:5030] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): delete failed for tablet dc14c37aa393407a87af0d0eb8b88d82 because the tablet was not found. No further retry: Not found: Tablet not found: dc14c37aa393407a87af0d0eb8b88d82
I20260925 13:56:29.795825 19942 ts_tablet_manager.cc:1403] T 321c740ba8eb4cbb8974305c5f8702af P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.592s user 0.009s sys 0.004s
I20260925 13:56:29.796494 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 170) took 7381 ms. Trace:
I20260925 13:56:29.796777 19955 rpcz_store.cc:276] 0925 13:56:22.415104 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:22.415193 (+ 89us) service_pool.cc:224] Handling call
0925 13:56:29.796249 (+7381056us) ts_tablet_manager.cc:1151] Deleting tablet 4d3dbd9796564472b44a31ccaf56d898
0925 13:56:29.796287 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:29.796444 (+ 157us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7380530}
W20260925 13:56:29.797888 19264 catalog_manager.cc:5030] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): delete failed for tablet 4d3dbd9796564472b44a31ccaf56d898 because the tablet was not found. No further retry: Not found: Tablet not found: 4d3dbd9796564472b44a31ccaf56d898
I20260925 13:56:29.799095 19942 raft_consensus.cc:359] T 321c740ba8eb4cbb8974305c5f8702af P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:29.799873 19955 tablet_replica.cc:333] T ec7fe5c834be4a51b627b8b1c3ebe888 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:29.799908 19942 raft_consensus.cc:385] T 321c740ba8eb4cbb8974305c5f8702af P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:29.800163 19942 raft_consensus.cc:740] T 321c740ba8eb4cbb8974305c5f8702af P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:29.800616 19955 raft_consensus.cc:2243] T ec7fe5c834be4a51b627b8b1c3ebe888 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:29.801134 19955 raft_consensus.cc:2272] T ec7fe5c834be4a51b627b8b1c3ebe888 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:29.802919 19955 ts_tablet_manager.cc:1916] T ec7fe5c834be4a51b627b8b1c3ebe888 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:29.805366 19939 tablet_bootstrap.cc:492] T f50ea68e3b3c4459b3f5e5b0df724724 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:29.805778 19939 ts_tablet_manager.cc:1403] T f50ea68e3b3c4459b3f5e5b0df724724 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.034s user 0.002s sys 0.008s
W20260925 13:56:29.807686 19264 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet eff33a56361946edb556ac611d2299eb (table default.Table_33e9cd401c9d45d0a1af408bda5e3bfa [id=2c74fb31a35541c4a2dd0b2171ba41ce], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet eff33a56361946edb556ac611d2299eb consensus state has no leader
I20260925 13:56:29.811506 19939 raft_consensus.cc:359] T f50ea68e3b3c4459b3f5e5b0df724724 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:29.812171 19939 raft_consensus.cc:385] T f50ea68e3b3c4459b3f5e5b0df724724 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:29.812460 19939 raft_consensus.cc:740] T f50ea68e3b3c4459b3f5e5b0df724724 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:29.813181 19939 consensus_queue.cc:260] T f50ea68e3b3c4459b3f5e5b0df724724 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:29.816116 20730 tablet.cc:1751] T 9059655f2c3d4e86b3b389b5b9a68891 P 8f578869bc084a5d8a9a69f143954eb3: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:29.817185 19939 ts_tablet_manager.cc:1434] T f50ea68e3b3c4459b3f5e5b0df724724 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20260925 13:56:29.820075 19939 tablet_bootstrap.cc:492] T 31049bccd170436893512a7e9ef435cc P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:29.835198 20011 ts_tablet_manager.cc:1929] T 953d41f26f5f4027acfd374c6ec4ee75 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:29.835644 20011 log.cc:1199] T 953d41f26f5f4027acfd374c6ec4ee75 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/953d41f26f5f4027acfd374c6ec4ee75
I20260925 13:56:29.836334 20011 ts_tablet_manager.cc:1950] T 953d41f26f5f4027acfd374c6ec4ee75 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:29.837482 19939 tablet_bootstrap.cc:654] T 31049bccd170436893512a7e9ef435cc P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:29.837684 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 953d41f26f5f4027acfd374c6ec4ee75 (table default.Table_9cdc814c021f49dea82875834691ba9e [id=e3fd45d69e384f9eb2ebce81bf195e5b]) successfully deleted
I20260925 13:56:29.843737 20814 log.cc:826] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728: Log is configured to *not* fsync() on all Append() calls
W20260925 13:56:29.852957 19773 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.193:38269, user_credentials={real_user=slave}} blocked reactor thread for 258806us
W20260925 13:56:29.853658 19773 leader_election.cc:343] T 000aad39953c4688b543c7d882a08ec8 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:29.854063 19773 leader_election.cc:304] T 000aad39953c4688b543c7d882a08ec8 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:29.856408 19939 tablet_bootstrap.cc:492] T 31049bccd170436893512a7e9ef435cc P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:29.856279 20805 tablet.cc:1751] T f50ea68e3b3c4459b3f5e5b0df724724 P f790493ba7ad403993b678f2df2b55be: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260925 13:56:29.856904 19939 ts_tablet_manager.cc:1403] T 31049bccd170436893512a7e9ef435cc P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.037s user 0.010s sys 0.002s
I20260925 13:56:29.859416 19939 raft_consensus.cc:359] T 31049bccd170436893512a7e9ef435cc P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:29.860123 19819 tablet_service.cc:1558] Processing DeleteTablet for tablet 61f141c40ce64fbcb68e01ac15009e3c with delete_type TABLET_DATA_DELETED (replaced by tablet f5791a011b104ec59721b6ed8ce773f1) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:29.860239 19939 raft_consensus.cc:385] T 31049bccd170436893512a7e9ef435cc P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:29.860502 19939 raft_consensus.cc:740] T 31049bccd170436893512a7e9ef435cc P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
W20260925 13:56:29.861475 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 8b858870f1254e72ae2af17c8192ffa0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b858870f1254e72ae2af17c8192ffa0 already in progress: creating tablet
W20260925 13:56:29.863240 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 61f141c40ce64fbcb68e01ac15009e3c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 61f141c40ce64fbcb68e01ac15009e3c already in progress: creating tablet
W20260925 13:56:29.865224 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet a218f033a132482a82bd1f60126fa0f9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a218f033a132482a82bd1f60126fa0f9 already in progress: creating tablet
I20260925 13:56:29.853309 19837 raft_consensus.cc:2468] T 1d641f09429546d683f2884c02152f73 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
I20260925 13:56:29.855583 19809 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b858870f1254e72ae2af17c8192ffa0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:27 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:29.869577 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e544c93488541da9ebb588a465b27b7" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:29.870292 19549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e544c93488541da9ebb588a465b27b7" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:29.870241 19939 consensus_queue.cc:260] T 31049bccd170436893512a7e9ef435cc P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:29.872649 20735 tablet.cc:1751] T f50ea68e3b3c4459b3f5e5b0df724724 P 7c1b52895a14482fb54a6758f5054f8f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260925 13:56:29.868139 19819 tablet_service.cc:1558] Processing DeleteTablet for tablet a218f033a132482a82bd1f60126fa0f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:29 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:29.873162 19939 ts_tablet_manager.cc:1434] T 31049bccd170436893512a7e9ef435cc P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.016s user 0.005s sys 0.000s
I20260925 13:56:29.876829 19938 tablet_bootstrap.cc:492] T 82c1002d2bcb4b11839e2f4a93b899ad P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:29.877286 19938 ts_tablet_manager.cc:1403] T 82c1002d2bcb4b11839e2f4a93b899ad P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.755s user 0.015s sys 0.003s
I20260925 13:56:29.857808 20811 raft_consensus.cc:2749] T 000aad39953c4688b543c7d882a08ec8 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:29.886255 19278 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_f35518532f584ec5b480332f43e688ad"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:29.887442 19272 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_66af2a1d5acd40a4a7ed59932097b68e" } modify_external_catalogs: true
W20260925 13:56:29.887727 19278 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f35518532f584ec5b480332f43e688ad in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:29.887926 19272 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_66af2a1d5acd40a4a7ed59932097b68e" } modify_external_catalogs: true
I20260925 13:56:29.900489 19939 tablet_bootstrap.cc:492] T 5f8a96ae6eec4f15a221f9aad537c094 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:29.906189 19939 tablet_bootstrap.cc:654] T 5f8a96ae6eec4f15a221f9aad537c094 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:29.927033 20814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260925 13:56:29.917187 20604 raft_consensus.cc:493] T f3b46e4fac694084b06b7086bcc596b9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:29.936326 20814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728: Bootstrap complete.
I20260925 13:56:29.938126 19939 tablet_bootstrap.cc:492] T 5f8a96ae6eec4f15a221f9aad537c094 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:29.938506 19939 ts_tablet_manager.cc:1403] T 5f8a96ae6eec4f15a221f9aad537c094 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.038s user 0.009s sys 0.009s
I20260925 13:56:29.941004 19939 raft_consensus.cc:359] T 5f8a96ae6eec4f15a221f9aad537c094 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:29.941615 19939 raft_consensus.cc:385] T 5f8a96ae6eec4f15a221f9aad537c094 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:29.941864 19939 raft_consensus.cc:740] T 5f8a96ae6eec4f15a221f9aad537c094 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:29.942500 19939 consensus_queue.cc:260] T 5f8a96ae6eec4f15a221f9aad537c094 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:29.949643 19939 ts_tablet_manager.cc:1434] T 5f8a96ae6eec4f15a221f9aad537c094 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.011s user 0.005s sys 0.001s
I20260925 13:56:29.951467 19939 tablet_bootstrap.cc:492] T e9dbcd85dc8647c68c120ba3fcfe1790 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:29.950696 19938 raft_consensus.cc:359] T 82c1002d2bcb4b11839e2f4a93b899ad P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:29.957440 19938 raft_consensus.cc:385] T 82c1002d2bcb4b11839e2f4a93b899ad P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:29.957721 19938 raft_consensus.cc:740] T 82c1002d2bcb4b11839e2f4a93b899ad P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:29.968065 19939 tablet_bootstrap.cc:654] T e9dbcd85dc8647c68c120ba3fcfe1790 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:29.976305 19938 consensus_queue.cc:260] T 82c1002d2bcb4b11839e2f4a93b899ad P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:29.977964 20723 raft_consensus.cc:493] T 31049bccd170436893512a7e9ef435cc P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:29.978405 20723 raft_consensus.cc:515] T 31049bccd170436893512a7e9ef435cc P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:29.980397 20723 leader_election.cc:290] T 31049bccd170436893512a7e9ef435cc P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:29.982151 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31049bccd170436893512a7e9ef435cc" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:29.983575 19488 leader_election.cc:343] T 31049bccd170436893512a7e9ef435cc P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:29.958143 20604 raft_consensus.cc:515] T f3b46e4fac694084b06b7086bcc596b9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:29.985540 20604 leader_election.cc:290] T f3b46e4fac694084b06b7086bcc596b9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:29.985663 19837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31049bccd170436893512a7e9ef435cc" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:29.986382 19837 raft_consensus.cc:2468] T 31049bccd170436893512a7e9ef435cc P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:29.986750 19938 ts_tablet_manager.cc:1434] T 82c1002d2bcb4b11839e2f4a93b899ad P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.109s user 0.003s sys 0.002s
I20260925 13:56:29.987564 19490 leader_election.cc:304] T 31049bccd170436893512a7e9ef435cc P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:29.988461 20723 raft_consensus.cc:2804] T 31049bccd170436893512a7e9ef435cc P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:29.988742 20723 raft_consensus.cc:493] T 31049bccd170436893512a7e9ef435cc P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:29.989035 20723 raft_consensus.cc:3060] T 31049bccd170436893512a7e9ef435cc P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:29.998018 20723 raft_consensus.cc:515] T 31049bccd170436893512a7e9ef435cc P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:30.001109 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31049bccd170436893512a7e9ef435cc" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
W20260925 13:56:30.002683 19488 leader_election.cc:343] T 31049bccd170436893512a7e9ef435cc P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:30.002552 19942 consensus_queue.cc:260] T 321c740ba8eb4cbb8974305c5f8702af P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:30.003541 20749 raft_consensus.cc:493] T 321c740ba8eb4cbb8974305c5f8702af P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:30.003953 20749 raft_consensus.cc:515] T 321c740ba8eb4cbb8974305c5f8702af P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:30.004325 20723 leader_election.cc:290] T 31049bccd170436893512a7e9ef435cc P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:30.007496 19939 tablet_bootstrap.cc:492] T e9dbcd85dc8647c68c120ba3fcfe1790 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:30.007934 19939 ts_tablet_manager.cc:1403] T e9dbcd85dc8647c68c120ba3fcfe1790 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.057s user 0.007s sys 0.003s
I20260925 13:56:30.007853 19549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "321c740ba8eb4cbb8974305c5f8702af" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:30.008342 20749 leader_election.cc:290] T 321c740ba8eb4cbb8974305c5f8702af P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
W20260925 13:56:30.009012 19629 leader_election.cc:343] T 321c740ba8eb4cbb8974305c5f8702af P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:30.010533 19939 raft_consensus.cc:359] T e9dbcd85dc8647c68c120ba3fcfe1790 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:30.011149 19939 raft_consensus.cc:385] T e9dbcd85dc8647c68c120ba3fcfe1790 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:30.011422 19939 raft_consensus.cc:740] T e9dbcd85dc8647c68c120ba3fcfe1790 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:30.018993 19942 ts_tablet_manager.cc:1434] T 321c740ba8eb4cbb8974305c5f8702af P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.223s user 0.006s sys 0.000s
I20260925 13:56:30.019917 19942 tablet_bootstrap.cc:492] T e2e44b2ca25c4bbcb91288424b3e18b9 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:30.019497 19939 consensus_queue.cc:260] T e9dbcd85dc8647c68c120ba3fcfe1790 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:30.021513 19939 ts_tablet_manager.cc:1434] T e9dbcd85dc8647c68c120ba3fcfe1790 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.013s user 0.002s sys 0.003s
I20260925 13:56:30.022289 19939 tablet_bootstrap.cc:492] T 1d641f09429546d683f2884c02152f73 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
W20260925 13:56:30.025338 19280 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 765b022051e84f5487b389c3ff29d0e5 consensus state has no leader
I20260925 13:56:30.023541 20814 raft_consensus.cc:359] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } }
I20260925 13:56:30.029524 19955 ts_tablet_manager.cc:1929] T ec7fe5c834be4a51b627b8b1c3ebe888 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:30.029870 19955 log.cc:1199] T ec7fe5c834be4a51b627b8b1c3ebe888 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/ec7fe5c834be4a51b627b8b1c3ebe888
I20260925 13:56:30.030658 19955 ts_tablet_manager.cc:1950] T ec7fe5c834be4a51b627b8b1c3ebe888 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:30.032686 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 174) took 7524 ms. Trace:
I20260925 13:56:30.016960 20785 tablet.cc:1751] T 0ad6442b2de249119ba0ef85dfc06da9 P f790493ba7ad403993b678f2df2b55be: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:30.032960 19955 rpcz_store.cc:276] 0925 13:56:22.508666 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:22.509192 (+ 526us) service_pool.cc:224] Handling call
0925 13:56:29.799588 (+7290396us) ts_tablet_manager.cc:1151] Deleting tablet ec7fe5c834be4a51b627b8b1c3ebe888
0925 13:56:29.799623 (+ 35us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:29.904961 (+105338us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:30.029402 (+124441us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:30.032639 (+ 3237us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7289797}
I20260925 13:56:30.040207 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet ec7fe5c834be4a51b627b8b1c3ebe888 (table default.Table_86e7099f24054da589b5a6f7de9ab131 [id=191bb6f9e3b14882aabe499934d8eb57]) successfully deleted
I20260925 13:56:29.975864 19840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.18.182.193:58915 (request call id 254) took 1210 ms. Trace:
I20260925 13:56:30.044479 19280 master_service.cc:960] ReplaceTablet: received request to replace tablet 7d55b56fa80143248f4ccc14898381b6 from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:30.045010 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 176) took 7501 ms. Trace:
I20260925 13:56:30.045516 19955 rpcz_store.cc:276] 0925 13:56:22.543762 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:22.543899 (+ 137us) service_pool.cc:224] Handling call
0925 13:56:30.044675 (+7500776us) ts_tablet_manager.cc:1151] Deleting tablet 908df914cfec490cb7058dbe7b501c97
0925 13:56:30.044715 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:30.044959 (+ 244us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7499575}
I20260925 13:56:30.048686 19832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31049bccd170436893512a7e9ef435cc" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:30.048897 20644 ts_tablet_manager.cc:1929] T 953d41f26f5f4027acfd374c6ec4ee75 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:30.049249 20644 log.cc:1199] T 953d41f26f5f4027acfd374c6ec4ee75 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/953d41f26f5f4027acfd374c6ec4ee75
I20260925 13:56:30.049947 20644 ts_tablet_manager.cc:1950] T 953d41f26f5f4027acfd374c6ec4ee75 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:30.052539 20644 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 220) took 2416 ms. Trace:
I20260925 13:56:30.052918 19938 tablet_bootstrap.cc:492] T 1ce5d15e67644247b95960f444acc621 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:30.049263 19832 raft_consensus.cc:3060] T 31049bccd170436893512a7e9ef435cc P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:30.053766 20823 tablet.cc:1751] T 3aaf29a2310a41d0971eded7ef51fdc9 P f790493ba7ad403993b678f2df2b55be: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:30.055828 20805 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:51442 (request call id 226) took 1412 ms. Trace:
I20260925 13:56:30.056290 19939 tablet_bootstrap.cc:654] T 1d641f09429546d683f2884c02152f73 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:30.059514 19831 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "321c740ba8eb4cbb8974305c5f8702af" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:30.060040 19831 raft_consensus.cc:2468] T 321c740ba8eb4cbb8974305c5f8702af P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:30.065112 19939 tablet_bootstrap.cc:492] T 1d641f09429546d683f2884c02152f73 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:30.065486 19939 ts_tablet_manager.cc:1403] T 1d641f09429546d683f2884c02152f73 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.043s user 0.007s sys 0.002s
I20260925 13:56:30.066606 20814 raft_consensus.cc:740] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc3953a2ba984a658630c423ec9b9728, State: Initialized, Role: FOLLOWER
W20260925 13:56:30.067499 19264 catalog_manager.cc:5043] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): delete failed for tablet 908df914cfec490cb7058dbe7b501c97 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 908df914cfec490cb7058dbe7b501c97 already in progress: creating tablet
I20260925 13:56:30.067725 19955 tablet_replica.cc:333] T 953d41f26f5f4027acfd374c6ec4ee75 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:30.067857 20814 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } }
I20260925 13:56:30.069021 19939 raft_consensus.cc:359] T 1d641f09429546d683f2884c02152f73 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:30.070192 19939 raft_consensus.cc:385] T 1d641f09429546d683f2884c02152f73 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:30.070569 19939 raft_consensus.cc:740] T 1d641f09429546d683f2884c02152f73 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:30.071784 19939 consensus_queue.cc:260] T 1d641f09429546d683f2884c02152f73 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:30.072486 19549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b46e4fac694084b06b7086bcc596b9" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
W20260925 13:56:30.073800 19773 leader_election.cc:343] T 7e544c93488541da9ebb588a465b27b7 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:30.073995 19554 raft_consensus.cc:3060] T f50ea68e3b3c4459b3f5e5b0df724724 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:30.074043 19939 ts_tablet_manager.cc:1434] T 1d641f09429546d683f2884c02152f73 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.008s user 0.006s sys 0.001s
I20260925 13:56:30.075054 19939 tablet_bootstrap.cc:492] T 908df914cfec490cb7058dbe7b501c97 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:30.082257 20837 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } } }
I20260925 13:56:30.083384 20837 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:56:30.084404 19939 tablet_bootstrap.cc:654] T 908df914cfec490cb7058dbe7b501c97 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:30.086470 20814 sys_catalog.cc:565] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:56:30.044448 19840 rpcz_store.cc:276] 0925 13:56:28.746987 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:28.747111 (+ 124us) service_pool.cc:224] Handling call
0925 13:56:28.747580 (+ 469us) raft_consensus.cc:1471] Updating replica for 2 ops
0925 13:56:28.748022 (+ 442us) raft_consensus.cc:1534] Early marking committed up to index 0
0925 13:56:28.748046 (+ 24us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0925 13:56:28.748550 (+ 504us) alter_schema_op.cc:128] START. Timestamp: 1790344588328623
0925 13:56:29.026057 (+277507us) log.cc:844] Serialized 199 byte log entry
0925 13:56:29.462250 (+436193us) raft_consensus.cc:1645] Marking committed up to 2
0925 13:56:29.539622 (+ 77372us) log.cc:844] Serialized 14 byte log entry
0925 13:56:29.904781 (+365159us) raft_consensus.cc:1697] Filling consensus response to leader.
0925 13:56:29.927308 (+ 22527us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0925 13:56:29.928186 (+ 878us) raft_consensus.cc:1685] finished
0925 13:56:29.928199 (+ 13us) raft_consensus.cc:1691] UpdateReplicas() finished
0925 13:56:29.957088 (+ 28889us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:28.975100 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0925 13:56:28.975530 (+ 430us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0925 13:56:28.975561 (+ 31us) alter_schema_op.cc:66] Acquired schema lock
0925 13:56:28.975630 (+ 69us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0925 13:56:29.664608 (+688978us) op_driver.cc:464] REPLICATION: finished
0925 13:56:30.044191 (+379583us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
Metrics: {"mutex_wait_us":570,"prepare.queue_time_us":453,"prepare.run_cpu_time_us":23,"prepare.run_wall_time_us":24,"raft.queue_time_us":6539,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":274,"spinlock_wait_cycles":1754496,"thread_start_us":276227,"threads_started":1,"wal-append.queue_time_us":274501,"child_traces":[["op",{"apply.queue_time_us":322523,"prepare.queue_time_us":226205,"prepare.run_cpu_time_us":593,"prepare.run_wall_time_us":595,"replication_time_us":804829,"thread_start_us":389322,"threads_started":2}]]}
I20260925 13:56:30.096012 19939 tablet_bootstrap.cc:492] T 908df914cfec490cb7058dbe7b501c97 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:30.096599 19939 ts_tablet_manager.cc:1403] T 908df914cfec490cb7058dbe7b501c97 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.022s user 0.011s sys 0.005s
I20260925 13:56:30.101194 19277 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_b54d79ef34824edd919b01a1cdd16f8d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:30.099381 19939 raft_consensus.cc:359] T 908df914cfec490cb7058dbe7b501c97 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:30.103579 19939 raft_consensus.cc:385] T 908df914cfec490cb7058dbe7b501c97 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:30.104056 19939 raft_consensus.cc:740] T 908df914cfec490cb7058dbe7b501c97 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:30.105454 19939 consensus_queue.cc:260] T 908df914cfec490cb7058dbe7b501c97 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:30.108952 19939 ts_tablet_manager.cc:1434] T 908df914cfec490cb7058dbe7b501c97 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.012s user 0.004s sys 0.002s
I20260925 13:56:30.111001 19939 tablet_bootstrap.cc:492] T a8a679a453e0430b866edc707fe7c234 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:30.056169 20805 rpcz_store.cc:276] 0925 13:56:28.643483 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:28.643911 (+ 428us) service_pool.cc:224] Handling call
0925 13:56:30.055747 (+1411836us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:28.804289 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0925 13:56:28.804796 (+ 507us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0925 13:56:28.804851 (+ 55us) alter_schema_op.cc:66] Acquired schema lock
0925 13:56:28.804947 (+ 96us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0925 13:56:28.805206 (+ 259us) alter_schema_op.cc:128] START. Timestamp: 1790344588805130
0925 13:56:28.805237 (+ 31us) op_driver.cc:348] REPLICATION: starting
0925 13:56:28.806038 (+ 801us) log.cc:844] Serialized 173 byte log entry
0925 13:56:29.554577 (+748539us) op_driver.cc:464] REPLICATION: finished
0925 13:56:29.855544 (+300967us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0925 13:56:29.858331 (+ 2787us) log.cc:844] Serialized 5 byte log entry
0925 13:56:30.046313 (+187982us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:30.055521 (+ 9208us) log.cc:844] Serialized 14 byte log entry
0925 13:56:30.055607 (+ 86us) alter_schema_op.cc:73] Released schema lock
0925 13:56:30.055636 (+ 29us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":300789,"prepare.queue_time_us":159242,"prepare.run_cpu_time_us":3383,"prepare.run_wall_time_us":264317,"replication_time_us":749219,"thread_start_us":859374,"threads_started":3,"wal-append.queue_time_us":251880}]]}
I20260925 13:56:30.124125 19939 tablet_bootstrap.cc:654] T a8a679a453e0430b866edc707fe7c234 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:30.128182 19885 heartbeater.cc:344] Connected to a master server at 127.18.182.252:36181
I20260925 13:56:30.054342 20644 rpcz_store.cc:276] 0925 13:56:27.636591 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:27.636726 (+ 135us) service_pool.cc:224] Handling call
0925 13:56:28.662321 (+1025595us) ts_tablet_manager.cc:1151] Deleting tablet 953d41f26f5f4027acfd374c6ec4ee75
0925 13:56:28.662390 (+ 69us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:29.090666 (+428276us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:30.048772 (+958106us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:30.052472 (+ 3700us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":214516,"tablet-delete.queue_time_us":1025032}
I20260925 13:56:30.133242 20644 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 219) took 2497 ms. Trace:
I20260925 13:56:30.133585 20644 rpcz_store.cc:276] 0925 13:56:27.636306 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:27.637382 (+ 1076us) service_pool.cc:224] Handling call
0925 13:56:30.132915 (+2495533us) ts_tablet_manager.cc:1151] Deleting tablet 908df914cfec490cb7058dbe7b501c97
0925 13:56:30.132971 (+ 56us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:30.133186 (+ 215us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2495033}
I20260925 13:56:30.134617 20644 tablet_replica.cc:333] T e2e44b2ca25c4bbcb91288424b3e18b9 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:30.119426 20804 raft_consensus.cc:493] T eff33a56361946edb556ac611d2299eb P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:30.135182 20804 raft_consensus.cc:515] T eff33a56361946edb556ac611d2299eb P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:30.137110 20804 leader_election.cc:290] T eff33a56361946edb556ac611d2299eb P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:30.147504 19938 tablet_bootstrap.cc:654] T 1ce5d15e67644247b95960f444acc621 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:30.148335 19939 tablet_bootstrap.cc:492] T a8a679a453e0430b866edc707fe7c234 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:30.149027 19939 ts_tablet_manager.cc:1403] T a8a679a453e0430b866edc707fe7c234 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.038s user 0.014s sys 0.007s
I20260925 13:56:30.149871 19955 raft_consensus.cc:2243] T 953d41f26f5f4027acfd374c6ec4ee75 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:30.151842 19955 raft_consensus.cc:2272] T 953d41f26f5f4027acfd374c6ec4ee75 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:30.153160 19939 raft_consensus.cc:359] T a8a679a453e0430b866edc707fe7c234 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:30.154191 19939 raft_consensus.cc:385] T a8a679a453e0430b866edc707fe7c234 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:30.154663 19939 raft_consensus.cc:740] T a8a679a453e0430b866edc707fe7c234 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:30.153774 19955 ts_tablet_manager.cc:1916] T 953d41f26f5f4027acfd374c6ec4ee75 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:30.159343 19832 raft_consensus.cc:2468] T 31049bccd170436893512a7e9ef435cc P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:30.156251 19939 consensus_queue.cc:260] T a8a679a453e0430b866edc707fe7c234 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:30.164258 20805 tablet.cc:1751] T f16050752ad44720ba8703ef10bbb359 P f790493ba7ad403993b678f2df2b55be: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:30.170715 19490 leader_election.cc:304] T 31049bccd170436893512a7e9ef435cc P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:30.171137 19549 raft_consensus.cc:3060] T a8a679a453e0430b866edc707fe7c234 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:30.172243 20795 raft_consensus.cc:2804] T 31049bccd170436893512a7e9ef435cc P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:30.173125 20795 raft_consensus.cc:697] T 31049bccd170436893512a7e9ef435cc P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Running, Role: LEADER
I20260925 13:56:30.173918 20795 consensus_queue.cc:237] T 31049bccd170436893512a7e9ef435cc P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:30.180488 19773 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.194:38641, user_credentials={real_user=slave}} blocked reactor thread for 51880.1us
W20260925 13:56:30.181133 19773 leader_election.cc:343] T 7e544c93488541da9ebb588a465b27b7 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:30.181489 19773 leader_election.cc:304] T 7e544c93488541da9ebb588a465b27b7 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
W20260925 13:56:30.200357 19773 leader_election.cc:343] T f3b46e4fac694084b06b7086bcc596b9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:30.202073 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 953d41f26f5f4027acfd374c6ec4ee75 (table default.Table_9cdc814c021f49dea82875834691ba9e [id=e3fd45d69e384f9eb2ebce81bf195e5b]) successfully deleted
W20260925 13:56:30.202739 19266 catalog_manager.cc:5030] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): delete failed for tablet 908df914cfec490cb7058dbe7b501c97 because the tablet was not found. No further retry: Not found: Tablet not found: 908df914cfec490cb7058dbe7b501c97
I20260925 13:56:30.202885 20831 raft_consensus.cc:2749] T 7e544c93488541da9ebb588a465b27b7 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:56:30.197057 19277 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b54d79ef34824edd919b01a1cdd16f8d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:30.218909 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff33a56361946edb556ac611d2299eb" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:30.220816 19939 ts_tablet_manager.cc:1434] T a8a679a453e0430b866edc707fe7c234 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.064s user 0.001s sys 0.008s
I20260925 13:56:30.060443 19631 leader_election.cc:304] T 1d641f09429546d683f2884c02152f73 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters: 8f578869bc084a5d8a9a69f143954eb3
W20260925 13:56:30.221935 19631 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 161565us
I20260925 13:56:30.222714 19939 tablet_bootstrap.cc:492] T 089dc4dfeb5243788dc03061c8743a81 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:30.071067 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b46e4fac694084b06b7086bcc596b9" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:30.211131 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff33a56361946edb556ac611d2299eb" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:30.215605 19629 consensus_peers.cc:597] T 0fdb45b31f93466c8bed1b848c6cdf15 P 7c1b52895a14482fb54a6758f5054f8f -> Peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Couldn't send request to peer 8f578869bc084a5d8a9a69f143954eb3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:30.224644 20748 raft_consensus.cc:2804] T 1d641f09429546d683f2884c02152f73 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:30.230365 20749 consensus_queue.cc:1048] T 0fdb45b31f93466c8bed1b848c6cdf15 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:30.231223 20748 raft_consensus.cc:697] T 1d641f09429546d683f2884c02152f73 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Becoming Leader. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Running, Role: LEADER
I20260925 13:56:30.232081 20748 consensus_queue.cc:237] T 1d641f09429546d683f2884c02152f73 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:30.220908 19773 leader_election.cc:343] T eff33a56361946edb556ac611d2299eb P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:30.235297 19555 raft_consensus.cc:3060] T 1215d875209a40d38cc527e2795d6c75 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
W20260925 13:56:30.239722 19773 leader_election.cc:343] T eff33a56361946edb556ac611d2299eb P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:30.240170 19773 leader_election.cc:304] T eff33a56361946edb556ac611d2299eb P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:30.240943 20831 raft_consensus.cc:2749] T eff33a56361946edb556ac611d2299eb P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:56:30.242040 19773 leader_election.cc:343] T f3b46e4fac694084b06b7086bcc596b9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:30.242439 19773 leader_election.cc:304] T f3b46e4fac694084b06b7086bcc596b9 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:30.243194 20831 raft_consensus.cc:2749] T f3b46e4fac694084b06b7086bcc596b9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:30.249190 19938 tablet_bootstrap.cc:492] T 1ce5d15e67644247b95960f444acc621 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:30.251132 19939 tablet_bootstrap.cc:654] T 089dc4dfeb5243788dc03061c8743a81 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:30.251677 20860 catalog_manager.cc:818] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 13:56:30.255890 19631 leader_election.cc:304] T 321c740ba8eb4cbb8974305c5f8702af P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters: 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:30.256577 20749 raft_consensus.cc:2804] T 321c740ba8eb4cbb8974305c5f8702af P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:30.256953 20749 raft_consensus.cc:493] T 321c740ba8eb4cbb8974305c5f8702af P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:30.257368 20749 raft_consensus.cc:3060] T 321c740ba8eb4cbb8974305c5f8702af P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:30.263960 19939 tablet_bootstrap.cc:492] T 089dc4dfeb5243788dc03061c8743a81 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:30.264384 19939 ts_tablet_manager.cc:1403] T 089dc4dfeb5243788dc03061c8743a81 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.042s user 0.013s sys 0.000s
I20260925 13:56:30.267117 20719 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "f790493ba7ad403993b678f2df2b55be" instance_seqno: 1790344568772730) as {username='slave'} at 127.18.182.195:41233; Asking this server to re-register.
I20260925 13:56:30.266865 19939 raft_consensus.cc:359] T 089dc4dfeb5243788dc03061c8743a81 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:30.267902 19939 raft_consensus.cc:385] T 089dc4dfeb5243788dc03061c8743a81 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:30.268251 19939 raft_consensus.cc:740] T 089dc4dfeb5243788dc03061c8743a81 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:30.269135 19939 consensus_queue.cc:260] T 089dc4dfeb5243788dc03061c8743a81 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:30.271899 19939 ts_tablet_manager.cc:1434] T 089dc4dfeb5243788dc03061c8743a81 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260925 13:56:30.272935 19939 tablet_bootstrap.cc:492] T e2e44b2ca25c4bbcb91288424b3e18b9 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:30.278515 19939 tablet_bootstrap.cc:654] T e2e44b2ca25c4bbcb91288424b3e18b9 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:30.285641 20718 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" instance_seqno: 1790344564101965) as {username='slave'} at 127.18.182.193:60609; Asking this server to re-register.
I20260925 13:56:30.288444 20795 raft_consensus.cc:493] T 089dc4dfeb5243788dc03061c8743a81 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:30.288983 20795 raft_consensus.cc:515] T 089dc4dfeb5243788dc03061c8743a81 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:30.290323 19275 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_c389f408e051410dac7a521879c22729" } new_table_name: "default.Table_6dd81ec6a73d459581359a7c93b23f9c" modify_external_catalogs: true
I20260925 13:56:30.291334 20795 leader_election.cc:290] T 089dc4dfeb5243788dc03061c8743a81 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:30.294440 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "089dc4dfeb5243788dc03061c8743a81" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:30.295856 19488 leader_election.cc:343] T 089dc4dfeb5243788dc03061c8743a81 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:30.302363 19600 heartbeater.cc:461] Registering TS with master...
I20260925 13:56:30.305423 19939 tablet_bootstrap.cc:492] T e2e44b2ca25c4bbcb91288424b3e18b9 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:30.310400 19939 ts_tablet_manager.cc:1403] T e2e44b2ca25c4bbcb91288424b3e18b9 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.038s user 0.016s sys 0.000s
W20260925 13:56:30.310428 19279 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f3b46e4fac694084b06b7086bcc596b9 consensus state has no leader
I20260925 13:56:30.312592 19939 raft_consensus.cc:359] T e2e44b2ca25c4bbcb91288424b3e18b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:30.313174 19939 raft_consensus.cc:385] T e2e44b2ca25c4bbcb91288424b3e18b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:30.313513 19939 raft_consensus.cc:740] T e2e44b2ca25c4bbcb91288424b3e18b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:30.314179 19939 consensus_queue.cc:260] T e2e44b2ca25c4bbcb91288424b3e18b9 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:30.316154 19939 ts_tablet_manager.cc:1434] T e2e44b2ca25c4bbcb91288424b3e18b9 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260925 13:56:30.316936 19939 tablet_bootstrap.cc:492] T 321c740ba8eb4cbb8974305c5f8702af P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:30.334215 19939 tablet_bootstrap.cc:654] T 321c740ba8eb4cbb8974305c5f8702af P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:30.276196 20644 raft_consensus.cc:2243] T e2e44b2ca25c4bbcb91288424b3e18b9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:30.280243 19885 heartbeater.cc:461] Registering TS with master...
I20260925 13:56:30.344918 20644 raft_consensus.cc:2272] T e2e44b2ca25c4bbcb91288424b3e18b9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:30.293805 19833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "089dc4dfeb5243788dc03061c8743a81" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:30.345674 19833 raft_consensus.cc:2468] T 089dc4dfeb5243788dc03061c8743a81 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:30.347478 19490 leader_election.cc:304] T 089dc4dfeb5243788dc03061c8743a81 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
W20260925 13:56:30.348780 20831 raft_consensus.cc:670] T e2e44b2ca25c4bbcb91288424b3e18b9 P f790493ba7ad403993b678f2df2b55be: failed to trigger leader election: Illegal state: RaftConsensus is not running: State = Stopped
I20260925 13:56:30.351099 20794 raft_consensus.cc:2804] T 089dc4dfeb5243788dc03061c8743a81 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:30.351285 19939 tablet_bootstrap.cc:492] T 321c740ba8eb4cbb8974305c5f8702af P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:30.351161 20644 ts_tablet_manager.cc:1916] T e2e44b2ca25c4bbcb91288424b3e18b9 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:30.351508 20794 raft_consensus.cc:493] T 089dc4dfeb5243788dc03061c8743a81 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:30.351857 20794 raft_consensus.cc:3060] T 089dc4dfeb5243788dc03061c8743a81 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:30.351703 19939 ts_tablet_manager.cc:1403] T 321c740ba8eb4cbb8974305c5f8702af P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.035s user 0.010s sys 0.000s
I20260925 13:56:30.356830 19939 raft_consensus.cc:359] T 321c740ba8eb4cbb8974305c5f8702af P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:30.357578 19939 raft_consensus.cc:385] T 321c740ba8eb4cbb8974305c5f8702af P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:30.357856 19939 raft_consensus.cc:740] T 321c740ba8eb4cbb8974305c5f8702af P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:30.360286 20794 raft_consensus.cc:515] T 089dc4dfeb5243788dc03061c8743a81 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:30.361389 19938 ts_tablet_manager.cc:1403] T 1ce5d15e67644247b95960f444acc621 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.309s user 0.007s sys 0.011s
I20260925 13:56:30.363325 20794 leader_election.cc:290] T 089dc4dfeb5243788dc03061c8743a81 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:30.364176 20831 raft_consensus.cc:493] T 5f8a96ae6eec4f15a221f9aad537c094 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:30.364431 19833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "089dc4dfeb5243788dc03061c8743a81" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:30.365005 19833 raft_consensus.cc:3060] T 089dc4dfeb5243788dc03061c8743a81 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:30.364792 20831 raft_consensus.cc:515] T 5f8a96ae6eec4f15a221f9aad537c094 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:30.367427 20831 leader_election.cc:290] T 5f8a96ae6eec4f15a221f9aad537c094 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:30.372818 19833 raft_consensus.cc:2468] T 089dc4dfeb5243788dc03061c8743a81 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:30.362666 19939 consensus_queue.cc:260] T 321c740ba8eb4cbb8974305c5f8702af P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:30.374801 19490 leader_election.cc:304] T 089dc4dfeb5243788dc03061c8743a81 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:30.377965 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f8a96ae6eec4f15a221f9aad537c094" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:30.378516 19555 raft_consensus.cc:2468] T 5f8a96ae6eec4f15a221f9aad537c094 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 0.
I20260925 13:56:30.378902 20795 raft_consensus.cc:2804] T 089dc4dfeb5243788dc03061c8743a81 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:30.382232 20795 raft_consensus.cc:697] T 089dc4dfeb5243788dc03061c8743a81 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Running, Role: LEADER
I20260925 13:56:30.382484 19279 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_32a95710635c459294ae80be58eed594"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:30.383068 20795 consensus_queue.cc:237] T 089dc4dfeb5243788dc03061c8743a81 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:30.384125 19279 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_32a95710635c459294ae80be58eed594 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:30.386190 20794 raft_consensus.cc:493] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:30.386646 20794 raft_consensus.cc:515] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:30.387575 19773 leader_election.cc:304] T 5f8a96ae6eec4f15a221f9aad537c094 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:30.388362 20831 raft_consensus.cc:2804] T 5f8a96ae6eec4f15a221f9aad537c094 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:30.389042 20831 raft_consensus.cc:493] T 5f8a96ae6eec4f15a221f9aad537c094 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:30.389405 20831 raft_consensus.cc:3060] T 5f8a96ae6eec4f15a221f9aad537c094 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
W20260925 13:56:30.391676 19490 leader_election.cc:343] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 436b8d46e1d244deb8bb9dc128ee6279
I20260925 13:56:30.393829 19939 ts_tablet_manager.cc:1434] T 321c740ba8eb4cbb8974305c5f8702af P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.041s user 0.005s sys 0.000s
I20260925 13:56:30.395018 20795 raft_consensus.cc:493] T 908df914cfec490cb7058dbe7b501c97 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:30.395471 20795 raft_consensus.cc:515] T 908df914cfec490cb7058dbe7b501c97 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:30.397205 19938 raft_consensus.cc:359] T 1ce5d15e67644247b95960f444acc621 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:30.397823 20795 leader_election.cc:290] T 908df914cfec490cb7058dbe7b501c97 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
W20260925 13:56:30.400013 19490 leader_election.cc:343] T 908df914cfec490cb7058dbe7b501c97 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 908df914cfec490cb7058dbe7b501c97
I20260925 13:56:30.400813 20790 mvcc.cc:204] Tried to move back new op lower bound from 7333251435745812480 to 7333251423429824512. Current Snapshot: MvccSnapshot[applied={T|T < 7333251435745812480}]
I20260925 13:56:30.394718 19939 tablet_bootstrap.cc:492] T 0fdb45b31f93466c8bed1b848c6cdf15 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:30.400893 20794 leader_election.cc:290] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:30.410192 20870 tablet.cc:1751] T f50ea68e3b3c4459b3f5e5b0df724724 P 8f578869bc084a5d8a9a69f143954eb3: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260925 13:56:30.410604 20718 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" instance_seqno: 1790344566482664) as {username='slave'} at 127.18.182.194:33741; Asking this server to re-register.
I20260925 13:56:30.421733 19939 tablet_bootstrap.cc:654] T 0fdb45b31f93466c8bed1b848c6cdf15 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
W20260925 13:56:30.452394 19488 leader_election.cc:343] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Not found: Tablet not found: 436b8d46e1d244deb8bb9dc128ee6279
I20260925 13:56:30.452798 19488 leader_election.cc:304] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3; no voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be
W20260925 13:56:30.453814 19488 leader_election.cc:343] T 908df914cfec490cb7058dbe7b501c97 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Not found: Tablet not found: 908df914cfec490cb7058dbe7b501c97
I20260925 13:56:30.454196 19488 leader_election.cc:304] T 908df914cfec490cb7058dbe7b501c97 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3; no voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:30.455060 20795 raft_consensus.cc:2749] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:56:30.455642 20795 raft_consensus.cc:2749] T 908df914cfec490cb7058dbe7b501c97 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:30.459769 20718 ts_manager.cc:194] Registered new tserver with Master: 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:30.471127 20719 ts_manager.cc:194] Registered new tserver with Master: f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:30.376179 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f8a96ae6eec4f15a221f9aad537c094" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:30.343302 20749 raft_consensus.cc:515] T 321c740ba8eb4cbb8974305c5f8702af P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:30.481624 20749 leader_election.cc:290] T 321c740ba8eb4cbb8974305c5f8702af P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:30.487675 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "321c740ba8eb4cbb8974305c5f8702af" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:30.488322 19554 raft_consensus.cc:3060] T 321c740ba8eb4cbb8974305c5f8702af P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:30.500924 19741 heartbeater.cc:461] Registering TS with master...
I20260925 13:56:30.424649 20644 ts_tablet_manager.cc:1929] T e2e44b2ca25c4bbcb91288424b3e18b9 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:30.519724 19938 raft_consensus.cc:385] T 1ce5d15e67644247b95960f444acc621 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:30.363106 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "089dc4dfeb5243788dc03061c8743a81" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
W20260925 13:56:30.524561 19488 leader_election.cc:343] T 089dc4dfeb5243788dc03061c8743a81 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:30.470773 19942 tablet_bootstrap.cc:654] T e2e44b2ca25c4bbcb91288424b3e18b9 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:30.519944 20644 log.cc:1199] T e2e44b2ca25c4bbcb91288424b3e18b9 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/e2e44b2ca25c4bbcb91288424b3e18b9
I20260925 13:56:30.526969 20644 ts_tablet_manager.cc:1950] T e2e44b2ca25c4bbcb91288424b3e18b9 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:30.527572 20837 raft_consensus.cc:493] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:30.528064 20837 raft_consensus.cc:515] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } }
I20260925 13:56:30.529129 20644 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 224) took 1888 ms. Trace:
I20260925 13:56:30.529430 20644 rpcz_store.cc:276] 0925 13:56:28.640614 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:28.641037 (+ 423us) service_pool.cc:224] Handling call
0925 13:56:30.134467 (+1493430us) ts_tablet_manager.cc:1151] Deleting tablet e2e44b2ca25c4bbcb91288424b3e18b9
0925 13:56:30.134514 (+ 47us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:30.411085 (+276571us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:30.424523 (+ 13438us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:30.529059 (+104536us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":140629,"tablet-delete.queue_time_us":1492505}
I20260925 13:56:30.530727 20644 tablet_replica.cc:333] T 3e6c8b9bbf4549b3a8685f886973ed9c P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:30.530673 20837 leader_election.cc:290] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7dcc50a8531144a69ab0cc00df1f0df4 (127.18.182.254:40825), bcc92728d49643efab5528d77c3d7828 (127.18.182.253:39641)
I20260925 13:56:30.541175 20831 raft_consensus.cc:515] T 5f8a96ae6eec4f15a221f9aad537c094 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:30.543505 20831 leader_election.cc:290] T 5f8a96ae6eec4f15a221f9aad537c094 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
W20260925 13:56:30.549571 19629 consensus_peers.cc:597] T 82972fa4868d4c59b3b9bb1240300938 P 7c1b52895a14482fb54a6758f5054f8f -> Peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Couldn't send request to peer 8f578869bc084a5d8a9a69f143954eb3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:30.558434 19554 raft_consensus.cc:2468] T 321c740ba8eb4cbb8974305c5f8702af P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
I20260925 13:56:30.560427 19629 leader_election.cc:304] T 321c740ba8eb4cbb8974305c5f8702af P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3; no voters:
I20260925 13:56:30.561623 19955 ts_tablet_manager.cc:1929] T 953d41f26f5f4027acfd374c6ec4ee75 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:30.562011 19955 log.cc:1199] T 953d41f26f5f4027acfd374c6ec4ee75 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/953d41f26f5f4027acfd374c6ec4ee75
I20260925 13:56:30.562680 19955 ts_tablet_manager.cc:1950] T 953d41f26f5f4027acfd374c6ec4ee75 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:30.567786 20748 raft_consensus.cc:2804] T 321c740ba8eb4cbb8974305c5f8702af P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:30.570147 20748 raft_consensus.cc:697] T 321c740ba8eb4cbb8974305c5f8702af P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Becoming Leader. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Running, Role: LEADER
I20260925 13:56:30.571081 20748 consensus_queue.cc:237] T 321c740ba8eb4cbb8974305c5f8702af P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:30.581849 19290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fc3953a2ba984a658630c423ec9b9728" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "bcc92728d49643efab5528d77c3d7828" is_pre_election: true
I20260925 13:56:30.569566 19515 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:40014 (request call id 35) took 1040 ms. Trace:
I20260925 13:56:30.593657 20794 raft_consensus.cc:493] T 1d641f09429546d683f2884c02152f73 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:30.602550 19939 tablet_bootstrap.cc:492] T 0fdb45b31f93466c8bed1b848c6cdf15 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:30.602962 19939 ts_tablet_manager.cc:1403] T 0fdb45b31f93466c8bed1b848c6cdf15 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.208s user 0.015s sys 0.025s
I20260925 13:56:30.602726 20794 raft_consensus.cc:515] T 1d641f09429546d683f2884c02152f73 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:30.604877 20794 leader_election.cc:290] T 1d641f09429546d683f2884c02152f73 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:30.605104 19939 raft_consensus.cc:359] T 0fdb45b31f93466c8bed1b848c6cdf15 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:30.605319 19515 rpcz_store.cc:276] 0925 13:56:29.529198 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:29.529316 (+ 118us) service_pool.cc:224] Handling call
0925 13:56:29.928690 (+399374us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b5c0006190c. stack: 00007f1358052323 00007f1358052220 00007f1357bf6574 0000000000516964 00000000005168f9 00007f135f5a6151 00007f135f74bfd9 00007f135f632560 00007f1360251960 00007f135e9a1104 00007f135e9a1092 00007f135e9a0fc1 00007f135e9a0f3c 00007f135e9a01f2 00007f135ceb509c 00007f135ceb44d6
0925 13:56:30.569339 (+640649us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":255561088}
I20260925 13:56:30.608464 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d641f09429546d683f2884c02152f73" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:30.605696 19939 raft_consensus.cc:385] T 0fdb45b31f93466c8bed1b848c6cdf15 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:30.611469 19939 raft_consensus.cc:740] T 0fdb45b31f93466c8bed1b848c6cdf15 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:30.612193 19939 consensus_queue.cc:260] T 0fdb45b31f93466c8bed1b848c6cdf15 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:30.613565 19939 ts_tablet_manager.cc:1434] T 0fdb45b31f93466c8bed1b848c6cdf15 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
W20260925 13:56:30.616863 19490 consensus_peers.cc:597] T 3e6c8b9bbf4549b3a8685f886973ed9c P 8f578869bc084a5d8a9a69f143954eb3 -> Peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Couldn't send request to peer f790493ba7ad403993b678f2df2b55be. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
W20260925 13:56:30.624902 19488 consensus_peers.cc:597] T 3e6c8b9bbf4549b3a8685f886973ed9c P 8f578869bc084a5d8a9a69f143954eb3 -> Peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Couldn't send request to peer 7c1b52895a14482fb54a6758f5054f8f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e6c8b9bbf4549b3a8685f886973ed9c. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:30.526435 19938 raft_consensus.cc:740] T 1ce5d15e67644247b95960f444acc621 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:30.626435 19938 consensus_queue.cc:260] T 1ce5d15e67644247b95960f444acc621 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:30.628095 19938 ts_tablet_manager.cc:1434] T 1ce5d15e67644247b95960f444acc621 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.266s user 0.005s sys 0.000s
W20260925 13:56:30.522156 19773 leader_election.cc:343] T 5f8a96ae6eec4f15a221f9aad537c094 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:30.631784 19773 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.194:38641, user_credentials={real_user=slave}} blocked reactor thread for 109665us
I20260925 13:56:30.597087 20804 raft_consensus.cc:493] T 82c1002d2bcb4b11839e2f4a93b899ad P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:30.632694 20804 raft_consensus.cc:515] T 82c1002d2bcb4b11839e2f4a93b899ad P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:30.605963 19830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d641f09429546d683f2884c02152f73" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:30.634217 19830 raft_consensus.cc:2393] T 1d641f09429546d683f2884c02152f73 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f578869bc084a5d8a9a69f143954eb3 in current term 1: Already voted for candidate 7c1b52895a14482fb54a6758f5054f8f in this term.
I20260925 13:56:30.634820 20804 leader_election.cc:290] T 82c1002d2bcb4b11839e2f4a93b899ad P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:30.636016 19490 leader_election.cc:304] T 1d641f09429546d683f2884c02152f73 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3; no voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:30.637030 20868 raft_consensus.cc:3060] T 1d641f09429546d683f2884c02152f73 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:30.674587 19830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "321c740ba8eb4cbb8974305c5f8702af" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:30.675117 19830 raft_consensus.cc:3060] T 321c740ba8eb4cbb8974305c5f8702af P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:30.682487 19830 raft_consensus.cc:2468] T 321c740ba8eb4cbb8974305c5f8702af P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
I20260925 13:56:30.688673 20868 raft_consensus.cc:2749] T 1d641f09429546d683f2884c02152f73 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:30.697966 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet e2e44b2ca25c4bbcb91288424b3e18b9 (table default.Table_261023ab15a845df82deea31f0eac8c8 [id=f03b9baa231c40999b38848e9059553e]) successfully deleted
I20260925 13:56:30.698670 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f8a96ae6eec4f15a221f9aad537c094" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:30.699178 19554 raft_consensus.cc:3060] T 5f8a96ae6eec4f15a221f9aad537c094 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:30.699431 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f8a96ae6eec4f15a221f9aad537c094" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:30.702112 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82c1002d2bcb4b11839e2f4a93b899ad" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:30.705300 19554 raft_consensus.cc:2468] T 5f8a96ae6eec4f15a221f9aad537c094 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 1.
I20260925 13:56:30.712543 19938 tablet_bootstrap.cc:492] T 505a24ebe7014f80b3c184329cc74a9a P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:30.732934 19939 tablet_bootstrap.cc:492] T 4d01a791445149e7bee9493e1eec5c6b P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:30.733361 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 184) took 7780 ms. Trace:
I20260925 13:56:30.733664 19955 rpcz_store.cc:276] 0925 13:56:22.952596 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:22.952726 (+ 130us) service_pool.cc:224] Handling call
0925 13:56:30.067583 (+7114857us) ts_tablet_manager.cc:1151] Deleting tablet 953d41f26f5f4027acfd374c6ec4ee75
0925 13:56:30.067623 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:30.151598 (+ 83975us) spinlock_profiling.cc:243] Waited 7.07 ms on lock 0x7b54000b2f54. stack: 00007f1358052323 00007f1358052220 00007f1357bf6574 0000000000516964 00000000005168f9 00007f135f14faa1 00007f135f1eeb22 00007f135f62f50c 00007f1360298c3a 00007f13602c9960 00007f13602afe6d 00007f13602afe19 00007f13602afda9 00007f13602afd71 00007f13602aef9d 00000000005bd4e4
0925 13:56:30.429209 (+277611us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:30.561508 (+132299us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:30.733303 (+171795us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74298,"spinlock_wait_cycles":16259072,"tablet-delete.queue_time_us":7114281}
I20260925 13:56:30.734936 19955 tablet_replica.cc:333] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:30.736034 19955 raft_consensus.cc:2243] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:30.736219 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet 953d41f26f5f4027acfd374c6ec4ee75 (table default.Table_9cdc814c021f49dea82875834691ba9e [id=e3fd45d69e384f9eb2ebce81bf195e5b]) successfully deleted
I20260925 13:56:30.736472 19955 raft_consensus.cc:2272] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:30.736764 19942 tablet_bootstrap.cc:492] T e2e44b2ca25c4bbcb91288424b3e18b9 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:30.737250 19942 ts_tablet_manager.cc:1403] T e2e44b2ca25c4bbcb91288424b3e18b9 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.718s user 0.010s sys 0.001s
I20260925 13:56:30.738121 19955 ts_tablet_manager.cc:1916] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:30.739081 19939 tablet_bootstrap.cc:654] T 4d01a791445149e7bee9493e1eec5c6b P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:30.739595 19942 raft_consensus.cc:359] T e2e44b2ca25c4bbcb91288424b3e18b9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:30.744436 19942 raft_consensus.cc:385] T e2e44b2ca25c4bbcb91288424b3e18b9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:30.745342 19942 raft_consensus.cc:740] T e2e44b2ca25c4bbcb91288424b3e18b9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:30.746560 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82c1002d2bcb4b11839e2f4a93b899ad" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
W20260925 13:56:30.749097 19275 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3d754e17930549189d65becb98afe2a6 consensus state has no leader
I20260925 13:56:30.749634 19939 tablet_bootstrap.cc:492] T 4d01a791445149e7bee9493e1eec5c6b P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:30.751124 19939 ts_tablet_manager.cc:1403] T 4d01a791445149e7bee9493e1eec5c6b P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.009s
I20260925 13:56:30.754236 19276 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 7e544c93488541da9ebb588a465b27b7 (table default.Table_4f849dc343f5448eb14a195bb0777d12 [id=f71e790b94b34e3d88f7539e396bc94d]). Expected version 1 got 0
I20260925 13:56:30.754057 19939 raft_consensus.cc:359] T 4d01a791445149e7bee9493e1eec5c6b P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:30.754822 19939 raft_consensus.cc:385] T 4d01a791445149e7bee9493e1eec5c6b P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:30.755095 19939 raft_consensus.cc:740] T 4d01a791445149e7bee9493e1eec5c6b P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:30.758246 19939 consensus_queue.cc:260] T 4d01a791445149e7bee9493e1eec5c6b P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:30.760849 19939 ts_tablet_manager.cc:1434] T 4d01a791445149e7bee9493e1eec5c6b P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260925 13:56:30.761453 19279 master_service.cc:960] ReplaceTablet: received request to replace tablet 8e5f54ef08054e7fb231c361615e2ab2 from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:30.777671 19273 catalog_manager.cc:5838] T 0fdb45b31f93466c8bed1b848c6cdf15 P 7c1b52895a14482fb54a6758f5054f8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194). New cstate: current_term: 1 leader_uuid: "7c1b52895a14482fb54a6758f5054f8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:30.775291 19277 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_8189c34636b04c0881a90a5311af243e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:30.781056 19277 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8189c34636b04c0881a90a5311af243e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:30.785126 19275 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_746abdd6da964b3592615357002e76bd" } new_table_name: "default.Table_52f43ad5551841a987dae01eb56ed549" modify_external_catalogs: true
I20260925 13:56:30.792987 19955 ts_tablet_manager.cc:1929] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:30.793231 19271 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 89bd21e08943403489e63c7c1895393a
I20260925 13:56:30.796693 19271 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:39288 (request call id 311) took 1113 ms. Trace:
I20260925 13:56:30.794763 19533 tablet_service.cc:1558] Processing DeleteTablet for tablet 89bd21e08943403489e63c7c1895393a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:29 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:30.802984 19955 log.cc:1199] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/436b8d46e1d244deb8bb9dc128ee6279
I20260925 13:56:30.803742 19955 ts_tablet_manager.cc:1950] T 436b8d46e1d244deb8bb9dc128ee6279 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:30.805717 19533 tablet_service.cc:1511] Processing CreateTablet for tablet ad3444ffde77425ebe3f39effa98d21d (DEFAULT_TABLE table=default.Table_b2092cda68f44cd4a3a79e3360f2e7ff [id=dd903bd2dfd5479ba56c3cfd8bb0520b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:30.797014 19271 rpcz_store.cc:276] 0925 13:56:29.683216 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:29.683877 (+ 661us) service_pool.cc:224] Handling call
0925 13:56:29.696933 (+ 13056us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b5400040454. stack: 00007fdf0a30b323 00007fdf0a30b220 00007fdf09eaf574 0000000000516964 00000000005168f9 00007fdf114095c0 00007fdf114a86fa 00007fdf13219d8c 00007fdf13319876 00007fdf10ec8b34 00007fdf10ec8ac2 00007fdf10ec89f1 00007fdf10ec896c 00007fdf10ec7c22 00007fdf0f16e09c 00007fdf0f16d4d6
0925 13:56:29.697973 (+ 1040us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 13:56:29.698602 (+ 629us) catalog_manager.cc:2854] Modifying in-memory table state
0925 13:56:29.698737 (+ 135us) catalog_manager.cc:2863] Locking tablets
0925 13:56:30.756929 (+1058192us) catalog_manager.cc:2877] Removing table and tablets from system table
0925 13:56:30.792916 (+ 35987us) catalog_manager.cc:2895] Removing table from by-name map
0925 13:56:30.793082 (+ 166us) catalog_manager.cc:2911] Committing in-memory state
0925 13:56:30.793161 (+ 79us) catalog_manager.cc:2915] Aborting table tasks
0925 13:56:30.796633 (+ 3472us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:30.758650 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:30.759022 (+ 372us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:30.759044 (+ 22us) write_op.cc:435] Acquired schema lock
0925 13:56:30.759056 (+ 12us) tablet.cc:662] Decoding operations
0925 13:56:30.759354 (+ 298us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:30.759426 (+ 72us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:30.759463 (+ 37us) tablet.cc:685] Acquiring locks for 2 operations
0925 13:56:30.759702 (+ 239us) tablet.cc:701] Row locks acquired
0925 13:56:30.759713 (+ 11us) write_op.cc:260] PREPARE: finished
0925 13:56:30.759803 (+ 90us) write_op.cc:270] Start()
0925 13:56:30.759871 (+ 68us) write_op.cc:276] Timestamp: P: 1790344590759771 usec, L: 0
0925 13:56:30.759885 (+ 14us) op_driver.cc:348] REPLICATION: starting
0925 13:56:30.760278 (+ 393us) log.cc:844] Serialized 734 byte log entry
0925 13:56:30.768897 (+ 8619us) op_driver.cc:464] REPLICATION: finished
0925 13:56:30.773727 (+ 4830us) write_op.cc:301] APPLY: starting
0925 13:56:30.773775 (+ 48us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:30.773868 (+ 93us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:30.773954 (+ 86us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:30.774294 (+ 340us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:30.774450 (+ 156us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 13:56:30.774490 (+ 40us) write_op.cc:312] APPLY: finished
0925 13:56:30.774648 (+ 158us) log.cc:844] Serialized 29 byte log entry
0925 13:56:30.775179 (+ 531us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:30.775246 (+ 67us) write_op.cc:454] Released schema lock
0925 13:56:30.775289 (+ 43us) write_op.cc:341] FINISH: Updating metrics
0925 13:56:30.775507 (+ 218us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:30.775530 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":28452224,"child_traces":[["op",{"apply.queue_time_us":4690,"apply.run_cpu_time_us":1360,"apply.run_wall_time_us":1770,"num_ops":2,"prepare.queue_time_us":97,"prepare.run_cpu_time_us":1857,"prepare.run_wall_time_us":1857,"raft.queue_time_us":877,"raft.run_cpu_time_us":685,"raft.run_wall_time_us":12199,"replication_time_us":8941}]]}
I20260925 13:56:30.829010 19280 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 7d55b56fa80143248f4ccc14898381b6
I20260925 13:56:30.830765 19280 catalog_manager.cc:6934] ReplaceTablet: tablet 7d55b56fa80143248f4ccc14898381b6 deleted and replaced by tablet e1c96465b34540ee865688f13a282699
I20260925 13:56:30.831957 19280 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_5d37f80782a44056aa57b74959fd9c78"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:30.833505 19280 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5d37f80782a44056aa57b74959fd9c78 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 13:56:30.835827 19267 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 69934b9a49354c92b5b13e182247ef77 (table default.Table_49155edad5db40549fa76e9195967efc [id=50c3843b240b47e1bfd1bfd07e3d4875], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 69934b9a49354c92b5b13e182247ef77 consensus state has no leader
I20260925 13:56:30.836674 19279 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 8e5f54ef08054e7fb231c361615e2ab2
I20260925 13:56:30.838213 19279 catalog_manager.cc:6934] ReplaceTablet: tablet 8e5f54ef08054e7fb231c361615e2ab2 deleted and replaced by tablet 0b1e2dc2abc249528de6b9ff6fad4275
I20260925 13:56:30.843945 19520 tablet_service.cc:1558] Processing DeleteTablet for tablet 7d55b56fa80143248f4ccc14898381b6 with delete_type TABLET_DATA_DELETED (replaced by tablet e1c96465b34540ee865688f13a282699) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:30.845119 19532 tablet_service.cc:1558] Processing DeleteTablet for tablet 8e5f54ef08054e7fb231c361615e2ab2 with delete_type TABLET_DATA_DELETED (replaced by tablet 0b1e2dc2abc249528de6b9ff6fad4275) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:30.848724 20723 raft_consensus.cc:493] T e2e44b2ca25c4bbcb91288424b3e18b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:30.849180 20723 raft_consensus.cc:515] T e2e44b2ca25c4bbcb91288424b3e18b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:30.851042 20723 leader_election.cc:290] T e2e44b2ca25c4bbcb91288424b3e18b9 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:30.730139 20804 raft_consensus.cc:493] T 1ce5d15e67644247b95960f444acc621 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:30.860230 19532 tablet_service.cc:1511] Processing CreateTablet for tablet addc93fc8e634c49ba29985c14dff0e0 (DEFAULT_TABLE table=default.Table_9ed8ad948c7742f0a0e75c2a7a368a9b [id=780d973508d7484eb5e52e548fc349ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:30.863942 19520 tablet_service.cc:1511] Processing CreateTablet for tablet 2541861530df4aae97215288da05df75 (DEFAULT_TABLE table=default.Table_25c2cde2aaca4b76b5d5595a803e37a0 [id=af19f26dd63f4f23adddf7ef35705d37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:30.867136 19524 tablet_service.cc:1511] Processing CreateTablet for tablet f753f6e45ade49ffa3128d5f4e09584a (DEFAULT_TABLE table=default.Table_871b30a592064a0abf86c3339a7d5847 [id=6da3861c3bdd4da18111be622813e45a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:30.870134 19817 tablet_service.cc:1511] Processing CreateTablet for tablet addc93fc8e634c49ba29985c14dff0e0 (DEFAULT_TABLE table=default.Table_9ed8ad948c7742f0a0e75c2a7a368a9b [id=780d973508d7484eb5e52e548fc349ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:30.871129 19817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet addc93fc8e634c49ba29985c14dff0e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:30.796224 19666 tablet_service.cc:1558] Processing DeleteTablet for tablet 89bd21e08943403489e63c7c1895393a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:29 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:30.805717 19669 tablet_service.cc:1511] Processing CreateTablet for tablet ad3444ffde77425ebe3f39effa98d21d (DEFAULT_TABLE table=default.Table_b2092cda68f44cd4a3a79e3360f2e7ff [id=dd903bd2dfd5479ba56c3cfd8bb0520b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:30.872439 19272 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet d2303a934d974e37ab6482d563492871
I20260925 13:56:30.872881 19669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad3444ffde77425ebe3f39effa98d21d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:30.810657 19674 tablet_service.cc:1511] Processing CreateTablet for tablet addc93fc8e634c49ba29985c14dff0e0 (DEFAULT_TABLE table=default.Table_9ed8ad948c7742f0a0e75c2a7a368a9b [id=780d973508d7484eb5e52e548fc349ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:30.875936 19674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet addc93fc8e634c49ba29985c14dff0e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:30.878207 19670 tablet_service.cc:1558] Processing DeleteTablet for tablet d2303a934d974e37ab6482d563492871 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:29 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:30.878814 19531 tablet_service.cc:1558] Processing DeleteTablet for tablet d2303a934d974e37ab6482d563492871 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:29 UTC) from {username='slave'} at 127.0.0.1:40018
W20260925 13:56:30.879215 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 89bd21e08943403489e63c7c1895393a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 89bd21e08943403489e63c7c1895393a already in progress: creating tablet
W20260925 13:56:30.880525 19264 catalog_manager.cc:5030] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet d2303a934d974e37ab6482d563492871 because the tablet was not found. No further retry: Not found: Tablet not found: d2303a934d974e37ab6482d563492871
I20260925 13:56:30.748353 20856 consensus_queue.cc:1048] T 82972fa4868d4c59b3b9bb1240300938 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.073s
I20260925 13:56:30.844275 19671 tablet_service.cc:1558] Processing DeleteTablet for tablet 7d55b56fa80143248f4ccc14898381b6 with delete_type TABLET_DATA_DELETED (replaced by tablet e1c96465b34540ee865688f13a282699) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:30.844703 19668 tablet_service.cc:1558] Processing DeleteTablet for tablet 8e5f54ef08054e7fb231c361615e2ab2 with delete_type TABLET_DATA_DELETED (replaced by tablet 0b1e2dc2abc249528de6b9ff6fad4275) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:30.856741 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2e44b2ca25c4bbcb91288424b3e18b9" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:30.862378 19673 tablet_service.cc:1511] Processing CreateTablet for tablet 2541861530df4aae97215288da05df75 (DEFAULT_TABLE table=default.Table_25c2cde2aaca4b76b5d5595a803e37a0 [id=af19f26dd63f4f23adddf7ef35705d37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:30.892400 19673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2541861530df4aae97215288da05df75. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:30.893527 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 7d55b56fa80143248f4ccc14898381b6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7d55b56fa80143248f4ccc14898381b6 already in progress: creating tablet
W20260925 13:56:30.894210 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 8e5f54ef08054e7fb231c361615e2ab2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8e5f54ef08054e7fb231c361615e2ab2 already in progress: creating tablet
I20260925 13:56:30.894425 19819 tablet_service.cc:1558] Processing DeleteTablet for tablet 7d55b56fa80143248f4ccc14898381b6 with delete_type TABLET_DATA_DELETED (replaced by tablet e1c96465b34540ee865688f13a282699) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:30.865460 19675 tablet_service.cc:1511] Processing CreateTablet for tablet f753f6e45ade49ffa3128d5f4e09584a (DEFAULT_TABLE table=default.Table_871b30a592064a0abf86c3339a7d5847 [id=6da3861c3bdd4da18111be622813e45a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:30.896170 19675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f753f6e45ade49ffa3128d5f4e09584a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:30.898813 19942 consensus_queue.cc:260] T e2e44b2ca25c4bbcb91288424b3e18b9 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:30.903836 19939 tablet_bootstrap.cc:492] T 000aad39953c4688b543c7d882a08ec8 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:30.903968 19533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad3444ffde77425ebe3f39effa98d21d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:30.907977 19532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet addc93fc8e634c49ba29985c14dff0e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:30.908488 19524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f753f6e45ade49ffa3128d5f4e09584a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:30.908955 19690 raft_consensus.cc:2468] T e2e44b2ca25c4bbcb91288424b3e18b9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:30.910010 19942 ts_tablet_manager.cc:1434] T e2e44b2ca25c4bbcb91288424b3e18b9 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.172s user 0.004s sys 0.000s
I20260925 13:56:30.910799 19942 tablet_bootstrap.cc:492] T f3b46e4fac694084b06b7086bcc596b9 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:30.912031 19488 leader_election.cc:304] T e2e44b2ca25c4bbcb91288424b3e18b9 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3; no voters:
I20260925 13:56:30.912922 19520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2541861530df4aae97215288da05df75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:30.914178 20723 raft_consensus.cc:493] T e9dbcd85dc8647c68c120ba3fcfe1790 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:30.914582 20723 raft_consensus.cc:515] T e9dbcd85dc8647c68c120ba3fcfe1790 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:30.917006 20723 leader_election.cc:290] T e9dbcd85dc8647c68c120ba3fcfe1790 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:30.917470 20723 raft_consensus.cc:2804] T e2e44b2ca25c4bbcb91288424b3e18b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:30.917395 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9dbcd85dc8647c68c120ba3fcfe1790" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:30.917732 20723 raft_consensus.cc:493] T e2e44b2ca25c4bbcb91288424b3e18b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:30.918040 20723 raft_consensus.cc:3060] T e2e44b2ca25c4bbcb91288424b3e18b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
W20260925 13:56:30.918708 19488 leader_election.cc:343] T e9dbcd85dc8647c68c120ba3fcfe1790 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:30.919667 20644 raft_consensus.cc:2243] T 3e6c8b9bbf4549b3a8685f886973ed9c P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:30.920221 20644 raft_consensus.cc:2272] T 3e6c8b9bbf4549b3a8685f886973ed9c P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:30.920054 19809 tablet_service.cc:1511] Processing CreateTablet for tablet ad3444ffde77425ebe3f39effa98d21d (DEFAULT_TABLE table=default.Table_b2092cda68f44cd4a3a79e3360f2e7ff [id=dd903bd2dfd5479ba56c3cfd8bb0520b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:30.921094 19809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad3444ffde77425ebe3f39effa98d21d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:30.923504 19802 tablet_service.cc:1558] Processing DeleteTablet for tablet 89bd21e08943403489e63c7c1895393a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:29 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:30.923451 19816 tablet_service.cc:1511] Processing CreateTablet for tablet 2541861530df4aae97215288da05df75 (DEFAULT_TABLE table=default.Table_25c2cde2aaca4b76b5d5595a803e37a0 [id=af19f26dd63f4f23adddf7ef35705d37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:30.924253 19939 tablet_bootstrap.cc:654] T 000aad39953c4688b543c7d882a08ec8 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:30.924477 19816 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2541861530df4aae97215288da05df75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:30.857136 20804 raft_consensus.cc:515] T 1ce5d15e67644247b95960f444acc621 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:30.928934 20804 leader_election.cc:290] T 1ce5d15e67644247b95960f444acc621 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:30.930544 19531 tablet_service.cc:1511] Processing CreateTablet for tablet d2303a934d974e37ab6482d563492871 (DEFAULT_TABLE table=default.Table_66af2a1d5acd40a4a7ed59932097b68e [id=4f3908c1491e433daf3a7938e02d3f73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:30.931430 19531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2303a934d974e37ab6482d563492871. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:30.931864 19819 tablet_service.cc:1511] Processing CreateTablet for tablet f753f6e45ade49ffa3128d5f4e09584a (DEFAULT_TABLE table=default.Table_871b30a592064a0abf86c3339a7d5847 [id=6da3861c3bdd4da18111be622813e45a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:30.932684 19819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f753f6e45ade49ffa3128d5f4e09584a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:30.936378 19490 leader_election.cc:343] T e2e44b2ca25c4bbcb91288424b3e18b9 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: e2e44b2ca25c4bbcb91288424b3e18b9
I20260925 13:56:30.922228 20644 ts_tablet_manager.cc:1916] T 3e6c8b9bbf4549b3a8685f886973ed9c P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:30.940323 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 197) took 6629 ms. Trace:
I20260925 13:56:30.946781 19673 tablet_service.cc:1511] Processing CreateTablet for tablet d2303a934d974e37ab6482d563492871 (DEFAULT_TABLE table=default.Table_66af2a1d5acd40a4a7ed59932097b68e [id=4f3908c1491e433daf3a7938e02d3f73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:30.947722 19939 tablet_bootstrap.cc:492] T 000aad39953c4688b543c7d882a08ec8 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:30.948134 19939 ts_tablet_manager.cc:1403] T 000aad39953c4688b543c7d882a08ec8 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.044s user 0.004s sys 0.009s
I20260925 13:56:30.947867 19673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2303a934d974e37ab6482d563492871. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:30.950567 19939 raft_consensus.cc:359] T 000aad39953c4688b543c7d882a08ec8 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:30.951134 19939 raft_consensus.cc:385] T 000aad39953c4688b543c7d882a08ec8 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:30.951472 19939 raft_consensus.cc:740] T 000aad39953c4688b543c7d882a08ec8 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:30.940611 19955 rpcz_store.cc:276] 0925 13:56:24.311176 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:24.343634 (+ 32458us) service_pool.cc:224] Handling call
0925 13:56:30.734792 (+6391158us) ts_tablet_manager.cc:1151] Deleting tablet 436b8d46e1d244deb8bb9dc128ee6279
0925 13:56:30.734831 (+ 39us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:30.745066 (+ 10235us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:30.792880 (+ 47814us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:30.940270 (+147390us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6373966}
I20260925 13:56:30.952808 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet 436b8d46e1d244deb8bb9dc128ee6279 (table default.Table_079723ff92144ba293733e54ab25d782 [id=4b0bfb9726854123864f8fe743a16a33]) successfully deleted
I20260925 13:56:30.952509 19939 consensus_queue.cc:260] T 000aad39953c4688b543c7d882a08ec8 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:30.947152 19773 leader_election.cc:304] T 5f8a96ae6eec4f15a221f9aad537c094 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:30.953070 19272 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_b8c44d75312d4cf48bedf12f8f6a948f" } new_table_name: "default.Table_563c733029714c03aa60643edfb85dbc" modify_external_catalogs: true
W20260925 13:56:30.953825 19773 leader_election.cc:343] T 82c1002d2bcb4b11839e2f4a93b899ad P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:30.954349 19773 leader_election.cc:343] T 5f8a96ae6eec4f15a221f9aad537c094 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:30.954993 19939 ts_tablet_manager.cc:1434] T 000aad39953c4688b543c7d882a08ec8 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
W20260925 13:56:30.954995 19773 leader_election.cc:343] T 82c1002d2bcb4b11839e2f4a93b899ad P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:30.956060 19939 tablet_bootstrap.cc:492] T 82972fa4868d4c59b3b9bb1240300938 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:30.935232 19805 tablet_service.cc:1558] Processing DeleteTablet for tablet 8e5f54ef08054e7fb231c361615e2ab2 with delete_type TABLET_DATA_DELETED (replaced by tablet 0b1e2dc2abc249528de6b9ff6fad4275) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:30.961896 19939 tablet_bootstrap.cc:654] T 82972fa4868d4c59b3b9bb1240300938 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:30.967962 19279 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_86e65c13d8024eba9016537e6f430289"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:30.969671 19279 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_86e65c13d8024eba9016537e6f430289 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:30.963069 19833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9dbcd85dc8647c68c120ba3fcfe1790" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:30.971702 19833 raft_consensus.cc:2468] T e9dbcd85dc8647c68c120ba3fcfe1790 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:30.971887 20831 raft_consensus.cc:493] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:30.960537 20804 raft_consensus.cc:2804] T 5f8a96ae6eec4f15a221f9aad537c094 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:30.972263 20831 raft_consensus.cc:515] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:30.974095 20831 leader_election.cc:290] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
W20260925 13:56:30.975924 19488 consensus_peers.cc:597] T 31049bccd170436893512a7e9ef435cc P 8f578869bc084a5d8a9a69f143954eb3 -> Peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Couldn't send request to peer 7c1b52895a14482fb54a6758f5054f8f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:30.978457 19490 leader_election.cc:304] T e9dbcd85dc8647c68c120ba3fcfe1790 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:30.979369 20868 raft_consensus.cc:2804] T e9dbcd85dc8647c68c120ba3fcfe1790 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:30.979795 20868 raft_consensus.cc:493] T e9dbcd85dc8647c68c120ba3fcfe1790 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:30.980168 20868 raft_consensus.cc:3060] T e9dbcd85dc8647c68c120ba3fcfe1790 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:30.982955 20794 raft_consensus.cc:493] T 000aad39953c4688b543c7d882a08ec8 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:30.983706 20794 raft_consensus.cc:515] T 000aad39953c4688b543c7d882a08ec8 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:30.895499 19271 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_f3eb4354192a4817a78558d8a6e64673"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:30.986195 20794 leader_election.cc:290] T 000aad39953c4688b543c7d882a08ec8 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
W20260925 13:56:30.986645 19271 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f3eb4354192a4817a78558d8a6e64673 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:30.990262 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "000aad39953c4688b543c7d882a08ec8" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:30.995014 19488 leader_election.cc:343] T 000aad39953c4688b543c7d882a08ec8 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:31.003132 19278 catalog_manager.cc:5838] T 82972fa4868d4c59b3b9bb1240300938 P 7c1b52895a14482fb54a6758f5054f8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194). New cstate: current_term: 1 leader_uuid: "7c1b52895a14482fb54a6758f5054f8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:31.004468 19278 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 82972fa4868d4c59b3b9bb1240300938 (table default.Table_d3d2076fbb6241a68f04e83462a34182 [id=4232cecee9104120ac2521eb250bbd0d]). Expected version 1 got 0
I20260925 13:56:31.004601 19833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "000aad39953c4688b543c7d882a08ec8" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:31.005678 19833 raft_consensus.cc:2468] T 000aad39953c4688b543c7d882a08ec8 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:31.005175 19278 catalog_manager.cc:5838] T 1d641f09429546d683f2884c02152f73 P 7c1b52895a14482fb54a6758f5054f8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194). New cstate: current_term: 1 leader_uuid: "7c1b52895a14482fb54a6758f5054f8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:31.007720 19274 catalog_manager.cc:6391] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: default.Table_dd8a1797a728429697e35fd628338f05 [id=a6d1215ba3be4fca903af9b17566b4a8] alter complete (version 1)
I20260925 13:56:31.007673 19490 leader_election.cc:304] T 000aad39953c4688b543c7d882a08ec8 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:31.008544 20794 raft_consensus.cc:2804] T 000aad39953c4688b543c7d882a08ec8 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:31.008936 20794 raft_consensus.cc:493] T 000aad39953c4688b543c7d882a08ec8 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:31.009318 20794 raft_consensus.cc:3060] T 000aad39953c4688b543c7d882a08ec8 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:31.010277 19276 catalog_manager.cc:6391] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: default.Table_01f0109a149148dfaf1974cee0c3e42b [id=f1353f53d28748c8a8a4a2c6259581bf] alter complete (version 3)
W20260925 13:56:31.011694 19276 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.195:57799 (request call id 41) took 978 ms (client timeout 1000 ms). Trace:
W20260925 13:56:31.012064 19276 rpcz_store.cc:269] 0925 13:56:30.033518 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:30.033609 (+ 91us) service_pool.cc:224] Handling call
0925 13:56:31.011622 (+978013us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:30.815089 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:30.816038 (+ 949us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:30.816058 (+ 20us) write_op.cc:435] Acquired schema lock
0925 13:56:30.816070 (+ 12us) tablet.cc:662] Decoding operations
0925 13:56:30.816284 (+ 214us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:30.816326 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:30.816346 (+ 20us) tablet.cc:685] Acquiring locks for 1 operations
0925 13:56:30.816472 (+ 126us) tablet.cc:701] Row locks acquired
0925 13:56:30.816486 (+ 14us) write_op.cc:260] PREPARE: finished
0925 13:56:30.816578 (+ 92us) write_op.cc:270] Start()
0925 13:56:30.816646 (+ 68us) write_op.cc:276] Timestamp: P: 1790344590816537 usec, L: 0
0925 13:56:30.816661 (+ 15us) op_driver.cc:348] REPLICATION: starting
0925 13:56:30.817039 (+ 378us) log.cc:844] Serialized 356 byte log entry
0925 13:56:30.868498 (+ 51459us) op_driver.cc:464] REPLICATION: finished
0925 13:56:30.869174 (+ 676us) write_op.cc:301] APPLY: starting
0925 13:56:30.869227 (+ 53us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:30.869320 (+ 93us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:30.869407 (+ 87us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:30.869588 (+ 181us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:30.869704 (+ 116us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 13:56:30.869763 (+ 59us) write_op.cc:312] APPLY: finished
0925 13:56:30.869932 (+ 169us) log.cc:844] Serialized 23 byte log entry
0925 13:56:30.870091 (+ 159us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:30.870149 (+ 58us) write_op.cc:454] Released schema lock
0925 13:56:30.870183 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0925 13:56:30.870399 (+ 216us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:30.870422 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":542,"apply.run_cpu_time_us":1192,"apply.run_wall_time_us":1191,"num_ops":1,"prepare.queue_time_us":22235,"prepare.run_cpu_time_us":1991,"prepare.run_wall_time_us":2114,"replication_time_us":51763}]]}
I20260925 13:56:31.018412 19938 tablet_bootstrap.cc:654] T 505a24ebe7014f80b3c184329cc74a9a P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:31.023439 19955 tablet_replica.cc:333] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:31.024230 20804 raft_consensus.cc:697] T 5f8a96ae6eec4f15a221f9aad537c094 P f790493ba7ad403993b678f2df2b55be [term 1 LEADER]: Becoming Leader. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Running, Role: LEADER
I20260925 13:56:31.027331 20794 raft_consensus.cc:515] T 000aad39953c4688b543c7d882a08ec8 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:31.037154 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "000aad39953c4688b543c7d882a08ec8" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
W20260925 13:56:31.038941 19488 leader_election.cc:343] T 000aad39953c4688b543c7d882a08ec8 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:31.043855 20882 log.cc:927] Time spent T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728: Append to log took a long time: real 0.098s user 0.000s sys 0.000s
I20260925 13:56:31.053124 20804 consensus_queue.cc:237] T 5f8a96ae6eec4f15a221f9aad537c094 P f790493ba7ad403993b678f2df2b55be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:31.058917 20697 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: connection negotiation to 127.18.182.254:40825 for RPC RequestConsensusVote timed out after 0.525s (ON_OUTBOUND_QUEUE)
I20260925 13:56:31.066390 19833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "000aad39953c4688b543c7d882a08ec8" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:31.067153 19833 raft_consensus.cc:3060] T 000aad39953c4688b543c7d882a08ec8 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:30.955305 19773 leader_election.cc:304] T 82c1002d2bcb4b11839e2f4a93b899ad P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:31.072176 20891 raft_consensus.cc:2749] T 82c1002d2bcb4b11839e2f4a93b899ad P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:56:31.074071 19773 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.194:38641, user_credentials={real_user=slave}} blocked reactor thread for 119095us
I20260925 13:56:31.075485 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce5d15e67644247b95960f444acc621" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:31.078109 19833 raft_consensus.cc:2468] T 000aad39953c4688b543c7d882a08ec8 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:31.078610 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce5d15e67644247b95960f444acc621" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:31.079233 20794 leader_election.cc:290] T 000aad39953c4688b543c7d882a08ec8 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:31.081483 19939 tablet_bootstrap.cc:492] T 82972fa4868d4c59b3b9bb1240300938 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:31.082017 19939 ts_tablet_manager.cc:1403] T 82972fa4868d4c59b3b9bb1240300938 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.126s user 0.054s sys 0.069s
I20260925 13:56:31.085314 19939 raft_consensus.cc:359] T 82972fa4868d4c59b3b9bb1240300938 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:31.086197 19939 raft_consensus.cc:385] T 82972fa4868d4c59b3b9bb1240300938 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:31.086681 19939 raft_consensus.cc:740] T 82972fa4868d4c59b3b9bb1240300938 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:31.087319 19939 consensus_queue.cc:260] T 82972fa4868d4c59b3b9bb1240300938 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:31.088766 19939 ts_tablet_manager.cc:1434] T 82972fa4868d4c59b3b9bb1240300938 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260925 13:56:31.089574 19939 tablet_bootstrap.cc:492] T f3b46e4fac694084b06b7086bcc596b9 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
W20260925 13:56:31.092937 20863 sasl_common.cc:323] Time spent creating new SASL kudu client: real 0.504s user 0.002s sys 0.000s
I20260925 13:56:31.093796 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d01a791445149e7bee9493e1eec5c6b" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:31.096639 19939 tablet_bootstrap.cc:654] T f3b46e4fac694084b06b7086bcc596b9 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:31.098119 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d01a791445149e7bee9493e1eec5c6b" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:31.098688 19555 raft_consensus.cc:2468] T 4d01a791445149e7bee9493e1eec5c6b P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 0.
W20260925 13:56:31.103690 20697 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.254:40825, user_credentials={real_user=slave}} blocked reactor thread for 45394.5us
I20260925 13:56:31.106766 19214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fc3953a2ba984a658630c423ec9b9728" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: false dest_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" is_pre_election: true
I20260925 13:56:31.110481 20697 leader_election.cc:304] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc3953a2ba984a658630c423ec9b9728; no voters: 7dcc50a8531144a69ab0cc00df1f0df4, bcc92728d49643efab5528d77c3d7828
I20260925 13:56:31.125180 20723 raft_consensus.cc:515] T e2e44b2ca25c4bbcb91288424b3e18b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:31.127566 20723 leader_election.cc:290] T e2e44b2ca25c4bbcb91288424b3e18b9 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:31.128376 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2e44b2ca25c4bbcb91288424b3e18b9" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:31.159508 19690 raft_consensus.cc:3060] T e2e44b2ca25c4bbcb91288424b3e18b9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:31.166955 19690 raft_consensus.cc:2468] T e2e44b2ca25c4bbcb91288424b3e18b9 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:31.113749 19942 tablet_bootstrap.cc:654] T f3b46e4fac694084b06b7086bcc596b9 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:31.157661 19955 raft_consensus.cc:2243] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:31.171162 19805 tablet_service.cc:1511] Processing CreateTablet for tablet d2303a934d974e37ab6482d563492871 (DEFAULT_TABLE table=default.Table_66af2a1d5acd40a4a7ed59932097b68e [id=4f3908c1491e433daf3a7938e02d3f73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:31.172205 19805 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2303a934d974e37ab6482d563492871. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:31.172433 19955 raft_consensus.cc:2272] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:31.174162 19955 ts_tablet_manager.cc:1916] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:31.161494 19490 leader_election.cc:304] T 000aad39953c4688b543c7d882a08ec8 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:31.175706 20794 raft_consensus.cc:2804] T 000aad39953c4688b543c7d882a08ec8 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:31.176129 20794 raft_consensus.cc:697] T 000aad39953c4688b543c7d882a08ec8 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Running, Role: LEADER
I20260925 13:56:31.176764 19939 tablet_bootstrap.cc:492] T f3b46e4fac694084b06b7086bcc596b9 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:31.177196 19939 ts_tablet_manager.cc:1403] T f3b46e4fac694084b06b7086bcc596b9 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.088s user 0.017s sys 0.035s
I20260925 13:56:31.176893 20794 consensus_queue.cc:237] T 000aad39953c4688b543c7d882a08ec8 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:31.180121 19939 raft_consensus.cc:359] T f3b46e4fac694084b06b7086bcc596b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:31.180816 19939 raft_consensus.cc:385] T f3b46e4fac694084b06b7086bcc596b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:31.181092 19939 raft_consensus.cc:740] T f3b46e4fac694084b06b7086bcc596b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:31.181834 19817 tablet_service.cc:1558] Processing DeleteTablet for tablet d2303a934d974e37ab6482d563492871 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:29 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:31.181762 19939 consensus_queue.cc:260] T f3b46e4fac694084b06b7086bcc596b9 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:31.022732 20868 raft_consensus.cc:515] T e9dbcd85dc8647c68c120ba3fcfe1790 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:31.168110 19488 leader_election.cc:304] T e2e44b2ca25c4bbcb91288424b3e18b9 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3; no voters:
I20260925 13:56:31.187702 20795 raft_consensus.cc:2804] T e2e44b2ca25c4bbcb91288424b3e18b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election won for term 1
W20260925 13:56:31.188367 19490 leader_election.cc:343] T e2e44b2ca25c4bbcb91288424b3e18b9 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: e2e44b2ca25c4bbcb91288424b3e18b9
I20260925 13:56:31.188982 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9dbcd85dc8647c68c120ba3fcfe1790" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:31.189944 19939 ts_tablet_manager.cc:1434] T f3b46e4fac694084b06b7086bcc596b9 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.012s user 0.006s sys 0.001s
W20260925 13:56:31.191044 19488 leader_election.cc:343] T e9dbcd85dc8647c68c120ba3fcfe1790 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:31.193007 20868 leader_election.cc:290] T e9dbcd85dc8647c68c120ba3fcfe1790 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:31.194756 19830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9dbcd85dc8647c68c120ba3fcfe1790" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:31.195305 19830 raft_consensus.cc:3060] T e9dbcd85dc8647c68c120ba3fcfe1790 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:31.201781 19830 raft_consensus.cc:2468] T e9dbcd85dc8647c68c120ba3fcfe1790 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:31.204370 19490 leader_election.cc:304] T e9dbcd85dc8647c68c120ba3fcfe1790 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:31.205143 20892 raft_consensus.cc:493] T f3b46e4fac694084b06b7086bcc596b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:31.205605 20892 raft_consensus.cc:515] T f3b46e4fac694084b06b7086bcc596b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:31.208530 20892 leader_election.cc:290] T f3b46e4fac694084b06b7086bcc596b9 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:31.209213 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b46e4fac694084b06b7086bcc596b9" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:31.210604 20892 raft_consensus.cc:2804] T e9dbcd85dc8647c68c120ba3fcfe1790 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:31.210845 19830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b46e4fac694084b06b7086bcc596b9" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:31.211458 19830 raft_consensus.cc:2468] T f3b46e4fac694084b06b7086bcc596b9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:31.212545 19490 leader_election.cc:304] T f3b46e4fac694084b06b7086bcc596b9 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters:
W20260925 13:56:31.221359 19488 leader_election.cc:343] T f3b46e4fac694084b06b7086bcc596b9 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:31.221740 19942 tablet_bootstrap.cc:492] T f3b46e4fac694084b06b7086bcc596b9 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:31.222162 19942 ts_tablet_manager.cc:1403] T f3b46e4fac694084b06b7086bcc596b9 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.312s user 0.012s sys 0.000s
I20260925 13:56:31.225936 20892 raft_consensus.cc:697] T e9dbcd85dc8647c68c120ba3fcfe1790 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Running, Role: LEADER
I20260925 13:56:31.228073 20723 raft_consensus.cc:2804] T f3b46e4fac694084b06b7086bcc596b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:31.228403 20723 raft_consensus.cc:493] T f3b46e4fac694084b06b7086bcc596b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:31.228698 20723 raft_consensus.cc:3060] T f3b46e4fac694084b06b7086bcc596b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:31.224593 19942 raft_consensus.cc:359] T f3b46e4fac694084b06b7086bcc596b9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:31.246210 19942 raft_consensus.cc:385] T f3b46e4fac694084b06b7086bcc596b9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:31.246872 19942 raft_consensus.cc:740] T f3b46e4fac694084b06b7086bcc596b9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:31.248940 19938 tablet_bootstrap.cc:492] T 505a24ebe7014f80b3c184329cc74a9a P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:31.248175 19942 consensus_queue.cc:260] T f3b46e4fac694084b06b7086bcc596b9 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:31.249357 19938 ts_tablet_manager.cc:1403] T 505a24ebe7014f80b3c184329cc74a9a P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.537s user 0.004s sys 0.006s
I20260925 13:56:31.252740 19942 ts_tablet_manager.cc:1434] T f3b46e4fac694084b06b7086bcc596b9 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.030s user 0.006s sys 0.001s
I20260925 13:56:31.254294 19942 tablet_bootstrap.cc:492] T 000aad39953c4688b543c7d882a08ec8 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:31.226730 20892 consensus_queue.cc:237] T e9dbcd85dc8647c68c120ba3fcfe1790 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:31.262629 20795 raft_consensus.cc:697] T e2e44b2ca25c4bbcb91288424b3e18b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Running, Role: LEADER
I20260925 13:56:31.263516 20795 consensus_queue.cc:237] T e2e44b2ca25c4bbcb91288424b3e18b9 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:31.266655 19942 tablet_bootstrap.cc:654] T 000aad39953c4688b543c7d882a08ec8 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:31.274602 20868 consensus_queue.cc:1048] T 31049bccd170436893512a7e9ef435cc P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:31.281481 19555 raft_consensus.cc:3060] T 0fdb45b31f93466c8bed1b848c6cdf15 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:31.290040 19939 tablet_bootstrap.cc:492] T eff33a56361946edb556ac611d2299eb P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:31.294842 19939 tablet_bootstrap.cc:654] T eff33a56361946edb556ac611d2299eb P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:31.219616 19955 ts_tablet_manager.cc:1929] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:56:31.295773 19955 log.cc:1199] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/8dc5ea164d6749b0926c63d6bda9dcc5
I20260925 13:56:31.298002 20644 ts_tablet_manager.cc:1929] T 3e6c8b9bbf4549b3a8685f886973ed9c P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:56:31.308336 20748 raft_consensus.cc:493] T f3b46e4fac694084b06b7086bcc596b9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:31.309073 20644 log.cc:1199] T 3e6c8b9bbf4549b3a8685f886973ed9c P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/3e6c8b9bbf4549b3a8685f886973ed9c
I20260925 13:56:31.308835 20748 raft_consensus.cc:515] T f3b46e4fac694084b06b7086bcc596b9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:31.310756 20748 leader_election.cc:290] T f3b46e4fac694084b06b7086bcc596b9 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:31.312213 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b46e4fac694084b06b7086bcc596b9" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:31.315927 20644 ts_tablet_manager.cc:1950] T 3e6c8b9bbf4549b3a8685f886973ed9c P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:31.347777 19280 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_8a9f077513e94c6d985f3c86954c3ccd" } modify_external_catalogs: true
I20260925 13:56:31.348317 19280 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_8a9f077513e94c6d985f3c86954c3ccd" } modify_external_catalogs: true
I20260925 13:56:31.350569 19942 tablet_bootstrap.cc:492] T 000aad39953c4688b543c7d882a08ec8 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
W20260925 13:56:31.352584 19275 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b45c85b8ff7c4b8f8cc8bab6fe20780b consensus state has no leader
I20260925 13:56:31.357162 19277 master_service.cc:960] ReplaceTablet: received request to replace tablet 000aad39953c4688b543c7d882a08ec8 from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:31.367053 19942 ts_tablet_manager.cc:1403] T 000aad39953c4688b543c7d882a08ec8 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.113s user 0.005s sys 0.007s
W20260925 13:56:31.318933 19773 leader_election.cc:343] T 1ce5d15e67644247b95960f444acc621 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:31.367559 19773 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.193:38269, user_credentials={real_user=slave}} blocked reactor thread for 61400.9us
I20260925 13:56:31.368186 19773 leader_election.cc:304] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:31.368901 20804 raft_consensus.cc:2804] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260925 13:56:31.369011 19773 leader_election.cc:343] T 1ce5d15e67644247b95960f444acc621 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:31.353698 19275 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_29a7a24743f647108b2fa4629b620df5" } modify_external_catalogs: true
I20260925 13:56:31.369343 20804 raft_consensus.cc:493] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:31.369814 20804 raft_consensus.cc:3060] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:31.387599 20804 raft_consensus.cc:515] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:31.398639 20804 leader_election.cc:290] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:31.399111 19275 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_29a7a24743f647108b2fa4629b620df5" } modify_external_catalogs: true
I20260925 13:56:31.389684 19942 raft_consensus.cc:359] T 000aad39953c4688b543c7d882a08ec8 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:31.400368 19942 raft_consensus.cc:385] T 000aad39953c4688b543c7d882a08ec8 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:31.396390 20644 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 225) took 2754 ms. Trace:
I20260925 13:56:31.402889 20644 rpcz_store.cc:276] 0925 13:56:28.642403 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:28.642538 (+ 135us) service_pool.cc:224] Handling call
0925 13:56:30.530594 (+1888056us) ts_tablet_manager.cc:1151] Deleting tablet 3e6c8b9bbf4549b3a8685f886973ed9c
0925 13:56:30.530631 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:31.022255 (+491624us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:31.297879 (+275624us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:31.396332 (+ 98453us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":334495,"tablet-delete.queue_time_us":1887473}
I20260925 13:56:31.404390 20644 tablet_replica.cc:333] T 78d8c01c88d24deeae21d647e7f205ba P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:31.404963 19942 raft_consensus.cc:740] T 000aad39953c4688b543c7d882a08ec8 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:31.317214 19939 tablet_bootstrap.cc:492] T eff33a56361946edb556ac611d2299eb P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:31.316349 19549 raft_consensus.cc:3060] T 82972fa4868d4c59b3b9bb1240300938 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:31.406177 19939 ts_tablet_manager.cc:1403] T eff33a56361946edb556ac611d2299eb P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.116s user 0.011s sys 0.002s
I20260925 13:56:31.406021 19942 consensus_queue.cc:260] T 000aad39953c4688b543c7d882a08ec8 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:31.408403 19939 raft_consensus.cc:359] T eff33a56361946edb556ac611d2299eb P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:31.408986 19939 raft_consensus.cc:385] T eff33a56361946edb556ac611d2299eb P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:31.409227 19939 raft_consensus.cc:740] T eff33a56361946edb556ac611d2299eb P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:31.410033 19939 consensus_queue.cc:260] T eff33a56361946edb556ac611d2299eb P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:31.411487 19939 ts_tablet_manager.cc:1434] T eff33a56361946edb556ac611d2299eb P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260925 13:56:31.412319 19939 tablet_bootstrap.cc:492] T b3172d99afee4a4c878e71b645037917 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:31.314335 19955 ts_tablet_manager.cc:1950] T 8dc5ea164d6749b0926c63d6bda9dcc5 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:31.414987 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 198) took 7103 ms. Trace:
I20260925 13:56:31.320884 19938 raft_consensus.cc:359] T 505a24ebe7014f80b3c184329cc74a9a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:31.424193 19938 raft_consensus.cc:385] T 505a24ebe7014f80b3c184329cc74a9a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:31.424506 19938 raft_consensus.cc:740] T 505a24ebe7014f80b3c184329cc74a9a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:31.428380 19955 rpcz_store.cc:276] 0925 13:56:24.311885 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:24.315466 (+ 3581us) service_pool.cc:224] Handling call
0925 13:56:30.960647 (+6645181us) ts_tablet_manager.cc:1151] Deleting tablet 8dc5ea164d6749b0926c63d6bda9dcc5
0925 13:56:30.960690 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:31.206547 (+245857us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:31.219494 (+ 12947us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:31.414938 (+195444us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":125955,"tablet-delete.queue_time_us":6567345}
I20260925 13:56:31.429884 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 203) took 6325 ms. Trace:
I20260925 13:56:31.429886 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet 8dc5ea164d6749b0926c63d6bda9dcc5 (table default.Table_e0f7b807e8884431898298a1c635c784 [id=e49e015b173c48c1b6ef8112f60b33e4]) successfully deleted
I20260925 13:56:31.423234 19942 ts_tablet_manager.cc:1434] T 000aad39953c4688b543c7d882a08ec8 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.044s user 0.003s sys 0.002s
I20260925 13:56:31.435010 19942 tablet_bootstrap.cc:492] T 089dc4dfeb5243788dc03061c8743a81 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:31.369364 19773 leader_election.cc:304] T 1ce5d15e67644247b95960f444acc621 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
W20260925 13:56:31.436961 19773 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.194:38641, user_credentials={real_user=slave}} blocked reactor thread for 67977.3us
W20260925 13:56:31.437577 19773 leader_election.cc:343] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:31.452961 19272 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 120825852c314caf95e67c1cbb95154d consensus state has no leader
I20260925 13:56:31.457777 19942 tablet_bootstrap.cc:654] T 089dc4dfeb5243788dc03061c8743a81 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:31.483549 20748 consensus_queue.cc:1048] T 1d641f09429546d683f2884c02152f73 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 13:56:31.425273 19938 consensus_queue.cc:260] T 505a24ebe7014f80b3c184329cc74a9a P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:31.374183 20723 raft_consensus.cc:515] T f3b46e4fac694084b06b7086bcc596b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:31.491559 19955 rpcz_store.cc:276] 0925 13:56:25.104818 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:25.108206 (+ 3388us) service_pool.cc:224] Handling call
0925 13:56:31.429553 (+6321347us) ts_tablet_manager.cc:1151] Deleting tablet 72183473a5c94629a58c2cd8e1c56f32
0925 13:56:31.429592 (+ 39us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:31.429831 (+ 239us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6319857}
I20260925 13:56:31.492851 20723 leader_election.cc:290] T f3b46e4fac694084b06b7086bcc596b9 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
W20260925 13:56:31.492868 19264 catalog_manager.cc:5043] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): delete failed for tablet 72183473a5c94629a58c2cd8e1c56f32 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 72183473a5c94629a58c2cd8e1c56f32 already in progress: creating tablet
I20260925 13:56:31.493506 19554 raft_consensus.cc:2393] T f3b46e4fac694084b06b7086bcc596b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7c1b52895a14482fb54a6758f5054f8f in current term 1: Already voted for candidate 8f578869bc084a5d8a9a69f143954eb3 in this term.
I20260925 13:56:31.496402 19830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b46e4fac694084b06b7086bcc596b9" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:31.496934 19830 raft_consensus.cc:3060] T f3b46e4fac694084b06b7086bcc596b9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:31.497144 19942 tablet_bootstrap.cc:492] T 089dc4dfeb5243788dc03061c8743a81 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:31.497926 19942 ts_tablet_manager.cc:1403] T 089dc4dfeb5243788dc03061c8743a81 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.063s user 0.007s sys 0.003s
I20260925 13:56:31.500061 19942 raft_consensus.cc:359] T 089dc4dfeb5243788dc03061c8743a81 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:31.500775 19942 raft_consensus.cc:385] T 089dc4dfeb5243788dc03061c8743a81 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:31.501051 19942 raft_consensus.cc:740] T 089dc4dfeb5243788dc03061c8743a81 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:31.501737 19942 consensus_queue.cc:260] T 089dc4dfeb5243788dc03061c8743a81 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:31.504096 19830 raft_consensus.cc:2468] T f3b46e4fac694084b06b7086bcc596b9 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:31.505180 19490 leader_election.cc:304] T f3b46e4fac694084b06b7086bcc596b9 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:31.505986 20723 raft_consensus.cc:2804] T f3b46e4fac694084b06b7086bcc596b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:31.506397 20723 raft_consensus.cc:697] T f3b46e4fac694084b06b7086bcc596b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Running, Role: LEADER
I20260925 13:56:31.507136 20723 consensus_queue.cc:237] T f3b46e4fac694084b06b7086bcc596b9 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:31.508111 19942 ts_tablet_manager.cc:1434] T 089dc4dfeb5243788dc03061c8743a81 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.010s user 0.006s sys 0.001s
I20260925 13:56:31.508898 19942 tablet_bootstrap.cc:492] T 31049bccd170436893512a7e9ef435cc P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:31.417384 19939 tablet_bootstrap.cc:654] T b3172d99afee4a4c878e71b645037917 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
W20260925 13:56:31.456228 19488 consensus_peers.cc:597] T 089dc4dfeb5243788dc03061c8743a81 P 8f578869bc084a5d8a9a69f143954eb3 -> Peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Couldn't send request to peer 7c1b52895a14482fb54a6758f5054f8f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260925 13:56:31.520328 19488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.194:38641, user_credentials={real_user=slave}} blocked reactor thread for 64204.4us
I20260925 13:56:31.521903 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b46e4fac694084b06b7086bcc596b9" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:31.522244 19942 tablet_bootstrap.cc:654] T 31049bccd170436893512a7e9ef435cc P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:31.522396 19690 raft_consensus.cc:3060] T f3b46e4fac694084b06b7086bcc596b9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:31.527849 20644 raft_consensus.cc:2243] T 78d8c01c88d24deeae21d647e7f205ba P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:31.528540 20891 raft_consensus.cc:2749] T 1ce5d15e67644247b95960f444acc621 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:56:31.529321 19773 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.194:38641, user_credentials={real_user=slave}} blocked reactor thread for 89959.4us
I20260925 13:56:31.530134 20644 raft_consensus.cc:2272] T 78d8c01c88d24deeae21d647e7f205ba P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:31.532081 20644 ts_tablet_manager.cc:1916] T 78d8c01c88d24deeae21d647e7f205ba P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:31.532075 19690 raft_consensus.cc:2468] T f3b46e4fac694084b06b7086bcc596b9 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:31.550222 19938 ts_tablet_manager.cc:1434] T 505a24ebe7014f80b3c184329cc74a9a P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.301s user 0.005s sys 0.001s
I20260925 13:56:31.551844 19938 tablet_bootstrap.cc:492] T 11077cdb735840b5921e22f588059149 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:31.552693 19942 tablet_bootstrap.cc:492] T 31049bccd170436893512a7e9ef435cc P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:31.552527 20723 consensus_queue.cc:1048] T 089dc4dfeb5243788dc03061c8743a81 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:31.553066 19942 ts_tablet_manager.cc:1403] T 31049bccd170436893512a7e9ef435cc P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.044s user 0.009s sys 0.000s
I20260925 13:56:31.555578 20856 raft_consensus.cc:493] T 089dc4dfeb5243788dc03061c8743a81 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:31.556027 19942 raft_consensus.cc:359] T 31049bccd170436893512a7e9ef435cc P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:31.555972 20856 raft_consensus.cc:515] T 089dc4dfeb5243788dc03061c8743a81 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:31.557788 20856 leader_election.cc:290] T 089dc4dfeb5243788dc03061c8743a81 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:31.558807 19942 raft_consensus.cc:385] T 31049bccd170436893512a7e9ef435cc P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:31.559078 19942 raft_consensus.cc:740] T 31049bccd170436893512a7e9ef435cc P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:31.559693 19942 consensus_queue.cc:260] T 31049bccd170436893512a7e9ef435cc P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:31.560758 19549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "089dc4dfeb5243788dc03061c8743a81" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:31.563773 19939 tablet_bootstrap.cc:492] T b3172d99afee4a4c878e71b645037917 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:31.564168 19939 ts_tablet_manager.cc:1403] T b3172d99afee4a4c878e71b645037917 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.152s user 0.006s sys 0.003s
I20260925 13:56:31.564690 19942 ts_tablet_manager.cc:1434] T 31049bccd170436893512a7e9ef435cc P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20260925 13:56:31.565529 19942 tablet_bootstrap.cc:492] T 5f8a96ae6eec4f15a221f9aad537c094 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:31.570472 19942 tablet_bootstrap.cc:654] T 5f8a96ae6eec4f15a221f9aad537c094 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:31.595531 19939 raft_consensus.cc:359] T b3172d99afee4a4c878e71b645037917 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:31.597746 19939 raft_consensus.cc:385] T b3172d99afee4a4c878e71b645037917 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:31.598469 19939 raft_consensus.cc:740] T b3172d99afee4a4c878e71b645037917 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:31.600584 19939 consensus_queue.cc:260] T b3172d99afee4a4c878e71b645037917 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:31.610726 19939 ts_tablet_manager.cc:1434] T b3172d99afee4a4c878e71b645037917 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.033s user 0.000s sys 0.004s
I20260925 13:56:31.631632 19955 tablet_replica.cc:333] T e2e44b2ca25c4bbcb91288424b3e18b9 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:31.632225 19939 tablet_bootstrap.cc:492] T c5a9df833271480aa49feb7d9f925459 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:31.632364 19955 raft_consensus.cc:2243] T e2e44b2ca25c4bbcb91288424b3e18b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:31.633008 19955 pending_rounds.cc:70] T e2e44b2ca25c4bbcb91288424b3e18b9 P 8f578869bc084a5d8a9a69f143954eb3: Trying to abort 1 pending ops.
W20260925 13:56:31.634964 19773 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.194:38641, user_credentials={real_user=slave}} blocked reactor thread for 105062us
I20260925 13:56:31.640715 19549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d01a791445149e7bee9493e1eec5c6b" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:31.641270 19549 raft_consensus.cc:3060] T 4d01a791445149e7bee9493e1eec5c6b P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:31.648775 19549 raft_consensus.cc:2468] T 4d01a791445149e7bee9493e1eec5c6b P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 1.
I20260925 13:56:31.633224 19955 pending_rounds.cc:77] T e2e44b2ca25c4bbcb91288424b3e18b9 P 8f578869bc084a5d8a9a69f143954eb3: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251445824999424 op_type: NO_OP noop_request { }
I20260925 13:56:31.649953 19955 raft_consensus.cc:2889] T e2e44b2ca25c4bbcb91288424b3e18b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:56:31.650295 19955 raft_consensus.cc:2272] T e2e44b2ca25c4bbcb91288424b3e18b9 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:31.652542 19955 ts_tablet_manager.cc:1916] T e2e44b2ca25c4bbcb91288424b3e18b9 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:31.655699 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d01a791445149e7bee9493e1eec5c6b" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:31.676569 19939 tablet_bootstrap.cc:654] T c5a9df833271480aa49feb7d9f925459 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:31.688486 19942 tablet_bootstrap.cc:492] T 5f8a96ae6eec4f15a221f9aad537c094 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:31.688961 19942 ts_tablet_manager.cc:1403] T 5f8a96ae6eec4f15a221f9aad537c094 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.124s user 0.006s sys 0.005s
I20260925 13:56:31.691272 19942 raft_consensus.cc:359] T 5f8a96ae6eec4f15a221f9aad537c094 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:31.701401 19939 tablet_bootstrap.cc:492] T c5a9df833271480aa49feb7d9f925459 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:31.701782 19939 ts_tablet_manager.cc:1403] T c5a9df833271480aa49feb7d9f925459 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.070s user 0.001s sys 0.009s
I20260925 13:56:31.704125 19939 raft_consensus.cc:359] T c5a9df833271480aa49feb7d9f925459 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:31.704737 19939 raft_consensus.cc:385] T c5a9df833271480aa49feb7d9f925459 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:31.704969 19939 raft_consensus.cc:740] T c5a9df833271480aa49feb7d9f925459 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:31.705538 19939 consensus_queue.cc:260] T c5a9df833271480aa49feb7d9f925459 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:31.707329 19939 ts_tablet_manager.cc:1434] T c5a9df833271480aa49feb7d9f925459 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260925 13:56:31.708223 19939 tablet_bootstrap.cc:492] T 78d8c01c88d24deeae21d647e7f205ba P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:31.708258 19955 ts_tablet_manager.cc:1929] T e2e44b2ca25c4bbcb91288424b3e18b9 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:31.708583 19955 log.cc:1199] T e2e44b2ca25c4bbcb91288424b3e18b9 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/e2e44b2ca25c4bbcb91288424b3e18b9
I20260925 13:56:31.709218 19955 ts_tablet_manager.cc:1950] T e2e44b2ca25c4bbcb91288424b3e18b9 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:31.709682 19271 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_845bbf47906f4e3fb7c047f2780dd6e3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:31.711020 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 205) took 6290 ms. Trace:
I20260925 13:56:31.711339 19955 rpcz_store.cc:276] 0925 13:56:25.420745 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:25.421084 (+ 339us) service_pool.cc:224] Handling call
0925 13:56:31.520757 (+6099673us) ts_tablet_manager.cc:1151] Deleting tablet e2e44b2ca25c4bbcb91288424b3e18b9
0925 13:56:31.626561 (+105804us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:31.696600 (+ 70039us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:31.708156 (+ 11556us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:31.710963 (+ 2807us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6099114}
W20260925 13:56:31.711315 19271 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_845bbf47906f4e3fb7c047f2780dd6e3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:31.712028 19280 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 9c5ef5b050624127990737acab3a8e9f
I20260925 13:56:31.713315 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet e2e44b2ca25c4bbcb91288424b3e18b9 (table default.Table_261023ab15a845df82deea31f0eac8c8 [id=f03b9baa231c40999b38848e9059553e]) successfully deleted
I20260925 13:56:31.718086 19533 tablet_service.cc:1558] Processing DeleteTablet for tablet 9c5ef5b050624127990737acab3a8e9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:31 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:31.719224 19280 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_ba7902001a1e48df84e25b9942bd98f9" } new_table_name: "default.Table_626a2de34ead48f4a50d72ea44cc25f2" modify_external_catalogs: true
I20260925 13:56:31.723265 19955 tablet_replica.cc:333] T c5a9df833271480aa49feb7d9f925459 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:31.723969 19955 raft_consensus.cc:2243] T c5a9df833271480aa49feb7d9f925459 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:31.724785 19955 raft_consensus.cc:2272] T c5a9df833271480aa49feb7d9f925459 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:31.726708 19955 ts_tablet_manager.cc:1916] T c5a9df833271480aa49feb7d9f925459 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:31.728936 19942 raft_consensus.cc:385] T 5f8a96ae6eec4f15a221f9aad537c094 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:31.729260 19942 raft_consensus.cc:740] T 5f8a96ae6eec4f15a221f9aad537c094 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:31.729846 19942 consensus_queue.cc:260] T 5f8a96ae6eec4f15a221f9aad537c094 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:31.735963 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 3e6c8b9bbf4549b3a8685f886973ed9c (table default.Table_d0dadd9b31b143fd8dd5adcb7f752c5f [id=beee0ac537e44859b0f7c76d6808b0e0]) successfully deleted
I20260925 13:56:31.757828 19673 tablet_service.cc:1558] Processing DeleteTablet for tablet 9c5ef5b050624127990737acab3a8e9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:31 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:31.758091 19942 ts_tablet_manager.cc:1434] T 5f8a96ae6eec4f15a221f9aad537c094 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.069s user 0.003s sys 0.002s
I20260925 13:56:31.758291 20892 raft_consensus.cc:493] T eff33a56361946edb556ac611d2299eb P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260925 13:56:31.758910 19631 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 122995us
I20260925 13:56:31.758718 20892 raft_consensus.cc:515] T eff33a56361946edb556ac611d2299eb P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:31.761162 20892 leader_election.cc:290] T eff33a56361946edb556ac611d2299eb P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:31.762181 19830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff33a56361946edb556ac611d2299eb" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:31.762781 19830 raft_consensus.cc:2468] T eff33a56361946edb556ac611d2299eb P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:31.764621 19490 leader_election.cc:304] T eff33a56361946edb556ac611d2299eb P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:31.765503 20794 raft_consensus.cc:2804] T eff33a56361946edb556ac611d2299eb P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:31.765825 20794 raft_consensus.cc:493] T eff33a56361946edb556ac611d2299eb P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:31.766139 20794 raft_consensus.cc:3060] T eff33a56361946edb556ac611d2299eb P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:31.775689 19939 tablet_bootstrap.cc:654] T 78d8c01c88d24deeae21d647e7f205ba P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:31.775691 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff33a56361946edb556ac611d2299eb" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:31.780601 19942 tablet_bootstrap.cc:492] T eff33a56361946edb556ac611d2299eb P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:31.781450 20794 raft_consensus.cc:515] T eff33a56361946edb556ac611d2299eb P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
W20260925 13:56:31.782785 19280 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d7c8beb65fd64639b22374fe4d3ef400 has no consensus state
I20260925 13:56:31.784010 19280 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_1363fa0f07a54c5e866650fa74b28e0c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:31.785910 19280 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1363fa0f07a54c5e866650fa74b28e0c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:31.786319 19830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff33a56361946edb556ac611d2299eb" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:31.786824 19830 raft_consensus.cc:3060] T eff33a56361946edb556ac611d2299eb P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
W20260925 13:56:31.789734 19488 leader_election.cc:343] T eff33a56361946edb556ac611d2299eb P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:31.794512 19830 raft_consensus.cc:2468] T eff33a56361946edb556ac611d2299eb P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:31.795231 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff33a56361946edb556ac611d2299eb" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:31.795612 19490 leader_election.cc:304] T eff33a56361946edb556ac611d2299eb P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:31.796473 20892 raft_consensus.cc:2804] T eff33a56361946edb556ac611d2299eb P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:31.796892 20892 raft_consensus.cc:697] T eff33a56361946edb556ac611d2299eb P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Running, Role: LEADER
W20260925 13:56:31.799640 19488 leader_election.cc:343] T eff33a56361946edb556ac611d2299eb P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:31.798895 20892 consensus_queue.cc:237] T eff33a56361946edb556ac611d2299eb P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:31.803229 20794 leader_election.cc:290] T eff33a56361946edb556ac611d2299eb P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:31.803887 19275 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 84188f52b54b477393d6a8433cbd1cb9
I20260925 13:56:31.804306 19955 ts_tablet_manager.cc:1929] T c5a9df833271480aa49feb7d9f925459 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:31.804625 19955 log.cc:1199] T c5a9df833271480aa49feb7d9f925459 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/c5a9df833271480aa49feb7d9f925459
I20260925 13:56:31.805806 19533 tablet_service.cc:1558] Processing DeleteTablet for tablet 84188f52b54b477393d6a8433cbd1cb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:31 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:31.807457 19955 ts_tablet_manager.cc:1950] T c5a9df833271480aa49feb7d9f925459 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:31.809399 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 206) took 6352 ms. Trace:
I20260925 13:56:31.809666 19955 rpcz_store.cc:276] 0925 13:56:25.456520 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:25.456656 (+ 136us) service_pool.cc:224] Handling call
0925 13:56:31.723108 (+6266452us) ts_tablet_manager.cc:1151] Deleting tablet c5a9df833271480aa49feb7d9f925459
0925 13:56:31.723162 (+ 54us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:31.733897 (+ 10735us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:31.804193 (+ 70296us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:31.809348 (+ 5155us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6265847}
W20260925 13:56:31.810616 20882 log.cc:927] Time spent T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728: Append to log took a long time: real 0.167s user 0.024s sys 0.005s
I20260925 13:56:31.810950 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet c5a9df833271480aa49feb7d9f925459 (table default.Table_25c2c368343741a3923a23f0a7cb2b62 [id=7d4523c5927040ba846c35a4b02172a7]) successfully deleted
I20260925 13:56:31.812355 19675 tablet_service.cc:1558] Processing DeleteTablet for tablet 84188f52b54b477393d6a8433cbd1cb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:31 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:31.813520 19275 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_b2092cda68f44cd4a3a79e3360f2e7ff" } new_table_name: "default.Table_b8532d5deb4b4b3a96df72497bf3dee1" modify_external_catalogs: true
I20260925 13:56:31.818418 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 209) took 6086 ms. Trace:
I20260925 13:56:31.818713 19955 rpcz_store.cc:276] 0925 13:56:25.732050 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:25.732183 (+ 133us) service_pool.cc:224] Handling call
0925 13:56:31.818184 (+6086001us) ts_tablet_manager.cc:1151] Deleting tablet 0de8f2d2766c4ff197b552a1b5ed3d6b
0925 13:56:31.818227 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:31.818371 (+ 144us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6085449}
W20260925 13:56:31.820279 19264 catalog_manager.cc:5030] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): delete failed for tablet 0de8f2d2766c4ff197b552a1b5ed3d6b because the tablet was not found. No further retry: Not found: Tablet not found: 0de8f2d2766c4ff197b552a1b5ed3d6b
I20260925 13:56:31.820402 19939 tablet_bootstrap.cc:492] T 78d8c01c88d24deeae21d647e7f205ba P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:31.820775 19939 ts_tablet_manager.cc:1403] T 78d8c01c88d24deeae21d647e7f205ba P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.113s user 0.008s sys 0.000s
I20260925 13:56:31.821128 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 208) took 6089 ms. Trace:
I20260925 13:56:31.821439 19955 rpcz_store.cc:276] 0925 13:56:25.731763 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:25.733012 (+ 1249us) service_pool.cc:224] Handling call
0925 13:56:31.820914 (+6087902us) ts_tablet_manager.cc:1151] Deleting tablet c2d333f892f84a258f8a148a90ce3b06
0925 13:56:31.820952 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:31.821081 (+ 129us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6087384}
W20260925 13:56:31.822544 19264 catalog_manager.cc:5030] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): delete failed for tablet c2d333f892f84a258f8a148a90ce3b06 because the tablet was not found. No further retry: Not found: Tablet not found: c2d333f892f84a258f8a148a90ce3b06
I20260925 13:56:31.822829 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 210) took 6090 ms. Trace:
I20260925 13:56:31.823285 19939 raft_consensus.cc:359] T 78d8c01c88d24deeae21d647e7f205ba P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:31.823940 19939 raft_consensus.cc:385] T 78d8c01c88d24deeae21d647e7f205ba P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:31.824180 19939 raft_consensus.cc:740] T 78d8c01c88d24deeae21d647e7f205ba P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:31.823112 19955 rpcz_store.cc:276] 0925 13:56:25.732342 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:25.733622 (+ 1280us) service_pool.cc:224] Handling call
0925 13:56:31.822613 (+6088991us) ts_tablet_manager.cc:1151] Deleting tablet 7ea4fd6817654bff9c0fe8c6e42ed6b9
0925 13:56:31.822647 (+ 34us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:31.822786 (+ 139us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6088475}
W20260925 13:56:31.825649 19264 catalog_manager.cc:5030] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): delete failed for tablet 7ea4fd6817654bff9c0fe8c6e42ed6b9 because the tablet was not found. No further retry: Not found: Tablet not found: 7ea4fd6817654bff9c0fe8c6e42ed6b9
I20260925 13:56:31.825549 19833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b46e4fac694084b06b7086bcc596b9" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:31.826272 19833 raft_consensus.cc:2393] T f3b46e4fac694084b06b7086bcc596b9 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7c1b52895a14482fb54a6758f5054f8f in current term 1: Already voted for candidate 8f578869bc084a5d8a9a69f143954eb3 in this term.
I20260925 13:56:31.828801 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 211) took 6096 ms. Trace:
I20260925 13:56:31.829094 19955 rpcz_store.cc:276] 0925 13:56:25.732617 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:25.734247 (+ 1630us) service_pool.cc:224] Handling call
0925 13:56:31.828579 (+6094332us) ts_tablet_manager.cc:1151] Deleting tablet 1da6eadb1648479889027fa911f4676b
0925 13:56:31.828621 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:31.828754 (+ 133us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6093817}
W20260925 13:56:31.830193 19264 catalog_manager.cc:5030] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): delete failed for tablet 1da6eadb1648479889027fa911f4676b because the tablet was not found. No further retry: Not found: Tablet not found: 1da6eadb1648479889027fa911f4676b
I20260925 13:56:31.831804 19939 consensus_queue.cc:260] T 78d8c01c88d24deeae21d647e7f205ba P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:31.834969 19939 ts_tablet_manager.cc:1434] T 78d8c01c88d24deeae21d647e7f205ba P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.014s user 0.000s sys 0.006s
I20260925 13:56:31.840065 19939 tablet_bootstrap.cc:492] T b23948807b2c4df9bf9a4da5f767e3b5 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:31.833125 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 214) took 6098 ms. Trace:
I20260925 13:56:31.843652 19955 rpcz_store.cc:276] 0925 13:56:25.735039 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:25.756227 (+ 21188us) service_pool.cc:224] Handling call
0925 13:56:31.832921 (+6076694us) ts_tablet_manager.cc:1151] Deleting tablet f514fa52e7524cd3ba07f3f65de532d4
0925 13:56:31.832959 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:31.833083 (+ 124us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6076057}
W20260925 13:56:31.844998 19264 catalog_manager.cc:5030] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): delete failed for tablet f514fa52e7524cd3ba07f3f65de532d4 because the tablet was not found. No further retry: Not found: Tablet not found: f514fa52e7524cd3ba07f3f65de532d4
I20260925 13:56:31.846067 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 215) took 5461 ms. Trace:
W20260925 13:56:31.846477 19267 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet d7c8beb65fd64639b22374fe4d3ef400 (table default.Table_626a2de34ead48f4a50d72ea44cc25f2 [id=f4560686cb314de99329f63b5902971d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet d7c8beb65fd64639b22374fe4d3ef400 has no consensus state
I20260925 13:56:31.846400 19955 rpcz_store.cc:276] 0925 13:56:26.384978 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:26.385125 (+ 147us) service_pool.cc:224] Handling call
0925 13:56:31.845736 (+5460611us) ts_tablet_manager.cc:1151] Deleting tablet 65ad0197267d47158751bc44c539b555
0925 13:56:31.845778 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:31.846017 (+ 239us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5460018}
W20260925 13:56:31.847565 19264 catalog_manager.cc:5043] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): delete failed for tablet 65ad0197267d47158751bc44c539b555 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 65ad0197267d47158751bc44c539b555 already in progress: creating tablet
I20260925 13:56:31.848404 19955 tablet_replica.cc:333] T 18a6d1c4eb7948149e709a380da50d93 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:31.832577 20927 consensus_queue.cc:1048] T 1d641f09429546d683f2884c02152f73 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260925 13:56:31.853390 19939 tablet_bootstrap.cc:654] T b23948807b2c4df9bf9a4da5f767e3b5 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
W20260925 13:56:31.853581 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 9c5ef5b050624127990737acab3a8e9f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9c5ef5b050624127990737acab3a8e9f already in progress: creating tablet
W20260925 13:56:31.855121 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 84188f52b54b477393d6a8433cbd1cb9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 84188f52b54b477393d6a8433cbd1cb9 already in progress: creating tablet
I20260925 13:56:31.855397 19955 raft_consensus.cc:2243] T 18a6d1c4eb7948149e709a380da50d93 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:31.856345 19955 raft_consensus.cc:2272] T 18a6d1c4eb7948149e709a380da50d93 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:31.858063 19955 ts_tablet_manager.cc:1916] T 18a6d1c4eb7948149e709a380da50d93 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:31.862821 19942 tablet_bootstrap.cc:654] T eff33a56361946edb556ac611d2299eb P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
W20260925 13:56:31.863736 19741 heartbeater.cc:412] Failed 1 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 13:56:31.864101 19741 heartbeater.cc:461] Registering TS with master...
W20260925 13:56:31.880357 19629 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.193:38269, user_credentials={real_user=slave}} blocked reactor thread for 66802.9us
I20260925 13:56:31.881467 20925 consensus_queue.cc:1048] T 321c740ba8eb4cbb8974305c5f8702af P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 13:56:31.884639 20719 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.193:60609 (request call id 5) took 938 ms (client timeout 1000 ms). Trace:
I20260925 13:56:31.885188 19938 tablet_bootstrap.cc:654] T 11077cdb735840b5921e22f588059149 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:31.885573 20898 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:56:31.888496 20837 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bcc92728d49643efab5528d77c3d7828. Latest consensus state: current_term: 1 leader_uuid: "bcc92728d49643efab5528d77c3d7828" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } } }
I20260925 13:56:31.889186 20837 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:56:31.890887 19939 tablet_bootstrap.cc:492] T b23948807b2c4df9bf9a4da5f767e3b5 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:31.891390 19939 ts_tablet_manager.cc:1403] T b23948807b2c4df9bf9a4da5f767e3b5 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.052s user 0.008s sys 0.003s
I20260925 13:56:31.891863 20717 ts_manager.cc:194] Registered new tserver with Master: 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:31.893661 19939 raft_consensus.cc:359] T b23948807b2c4df9bf9a4da5f767e3b5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
W20260925 13:56:31.894297 20717 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.194:33741 (request call id 3) took 859 ms (client timeout 1000 ms). Trace:
I20260925 13:56:31.894414 19939 raft_consensus.cc:385] T b23948807b2c4df9bf9a4da5f767e3b5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:31.894752 19939 raft_consensus.cc:740] T b23948807b2c4df9bf9a4da5f767e3b5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:31.895435 19939 consensus_queue.cc:260] T b23948807b2c4df9bf9a4da5f767e3b5 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:31.898440 19939 ts_tablet_manager.cc:1434] T b23948807b2c4df9bf9a4da5f767e3b5 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.007s user 0.001s sys 0.003s
I20260925 13:56:31.902392 19695 raft_consensus.cc:3060] T 31049bccd170436893512a7e9ef435cc P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:31.904711 20925 raft_consensus.cc:493] T 000aad39953c4688b543c7d882a08ec8 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:31.905207 20925 raft_consensus.cc:515] T 000aad39953c4688b543c7d882a08ec8 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:31.886323 20719 rpcz_store.cc:269] 0925 13:56:30.946933 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:30.947454 (+ 521us) service_pool.cc:224] Handling call
0925 13:56:31.881566 (+934112us) spinlock_profiling.cc:243] Waited 847 ms on lock 0x7b5400040454. stack: 00007fa24768b323 00007fa24768b220 00007fa24722f574 0000000000516964 00000000005168f9 00007fa24e7895c0 00007fa24e8286fa 00007fa250599d8c 00007fa25069801e 00007fa24e23cfb4 00007fa24e23cf42 00007fa24e23ce71 00007fa24e23cdec 00007fa24e23c0a2 00007fa24c4ee09c 00007fa24c4ed4d6
0925 13:56:31.884572 (+ 3006us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1947820928}
I20260925 13:56:31.909657 20925 leader_election.cc:290] T 000aad39953c4688b543c7d882a08ec8 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:31.900753 19955 ts_tablet_manager.cc:1929] T 18a6d1c4eb7948149e709a380da50d93 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:56:31.913808 19955 log.cc:1199] T 18a6d1c4eb7948149e709a380da50d93 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/18a6d1c4eb7948149e709a380da50d93
I20260925 13:56:31.914558 19955 ts_tablet_manager.cc:1950] T 18a6d1c4eb7948149e709a380da50d93 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:31.916280 19942 tablet_bootstrap.cc:492] T eff33a56361946edb556ac611d2299eb P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:31.916661 19942 ts_tablet_manager.cc:1403] T eff33a56361946edb556ac611d2299eb P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.136s user 0.004s sys 0.005s
I20260925 13:56:31.917117 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 216) took 5313 ms. Trace:
I20260925 13:56:31.908255 19549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "000aad39953c4688b543c7d882a08ec8" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:31.917450 19955 rpcz_store.cc:276] 0925 13:56:26.603225 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:26.603378 (+ 153us) service_pool.cc:224] Handling call
0925 13:56:31.848271 (+5244893us) ts_tablet_manager.cc:1151] Deleting tablet 18a6d1c4eb7948149e709a380da50d93
0925 13:56:31.848308 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:31.882066 (+ 33758us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:31.900625 (+ 18559us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:31.917049 (+ 16424us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5244327}
I20260925 13:56:31.919528 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet 18a6d1c4eb7948149e709a380da50d93 (table default.Table_e814702f1f874529a98bd1631d3b5c84 [id=6c0ec9465714408da93ff6e47827eb48]) successfully deleted
I20260925 13:56:31.919806 19955 tablet_replica.cc:333] T f5a0bd0e205f4889be93739dfbd2a816 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:31.920548 19955 raft_consensus.cc:2243] T f5a0bd0e205f4889be93739dfbd2a816 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:31.921434 19955 raft_consensus.cc:2272] T f5a0bd0e205f4889be93739dfbd2a816 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:31.925105 19942 raft_consensus.cc:359] T eff33a56361946edb556ac611d2299eb P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:31.925707 19942 raft_consensus.cc:385] T eff33a56361946edb556ac611d2299eb P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:31.925961 19942 raft_consensus.cc:740] T eff33a56361946edb556ac611d2299eb P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:31.926535 19942 consensus_queue.cc:260] T eff33a56361946edb556ac611d2299eb P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:31.927642 19955 ts_tablet_manager.cc:1916] T f5a0bd0e205f4889be93739dfbd2a816 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 13:56:31.894613 20717 rpcz_store.cc:269] 0925 13:56:31.035280 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:31.035799 (+ 519us) service_pool.cc:224] Handling call
0925 13:56:31.886714 (+850915us) spinlock_profiling.cc:243] Waited 832 ms on lock 0x7b5400040454. stack: 00007fa24768b323 00007fa24768b220 00007fa24722f574 0000000000516964 00000000005168f9 00007fa24e7895c0 00007fa24e8286fa 00007fa250599d8c 00007fa25069801e 00007fa24e23cfb4 00007fa24e23cf42 00007fa24e23ce71 00007fa24e23cdec 00007fa24e23c0a2 00007fa24c4ee09c 00007fa24c4ed4d6
0925 13:56:31.894244 (+ 7530us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1913560960}
I20260925 13:56:31.940686 19939 tablet_bootstrap.cc:492] T 7e544c93488541da9ebb588a465b27b7 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:31.953475 19939 tablet_bootstrap.cc:654] T 7e544c93488541da9ebb588a465b27b7 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
W20260925 13:56:31.965582 20718 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.195:41233 (request call id 4) took 1329 ms (client timeout 1000 ms). Trace:
I20260925 13:56:31.966747 19955 ts_tablet_manager.cc:1929] T f5a0bd0e205f4889be93739dfbd2a816 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:31.967088 19955 log.cc:1199] T f5a0bd0e205f4889be93739dfbd2a816 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/f5a0bd0e205f4889be93739dfbd2a816
I20260925 13:56:31.967774 19955 ts_tablet_manager.cc:1950] T f5a0bd0e205f4889be93739dfbd2a816 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
W20260925 13:56:31.966007 20718 rpcz_store.cc:269] 0925 13:56:30.635668 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:30.710618 (+ 74950us) service_pool.cc:224] Handling call
0925 13:56:31.876333 (+1165715us) spinlock_profiling.cc:243] Waited 838 ms on lock 0x7b5400040454. stack: 00007fa24768b323 00007fa24768b220 00007fa24722f574 0000000000516964 00000000005168f9 00007fa24e7895c0 00007fa24e8286fa 00007fa250599d8c 00007fa25069801e 00007fa24e23cfb4 00007fa24e23cf42 00007fa24e23ce71 00007fa24e23cdec 00007fa24e23c0a2 00007fa24c4ee09c 00007fa24c4ed4d6
0925 13:56:31.965504 (+ 89171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1926672640}
I20260925 13:56:31.969616 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 217) took 5003 ms. Trace:
I20260925 13:56:31.969894 19955 rpcz_store.cc:276] 0925 13:56:26.966596 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:26.966707 (+ 111us) service_pool.cc:224] Handling call
0925 13:56:31.919666 (+4952959us) ts_tablet_manager.cc:1151] Deleting tablet f5a0bd0e205f4889be93739dfbd2a816
0925 13:56:31.919705 (+ 39us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:31.960085 (+ 40380us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:31.966644 (+ 6559us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:31.969563 (+ 2919us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4952387}
I20260925 13:56:31.970702 20713 ts_descriptor.cc:143] Processing retry of TS registration from permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" instance_seqno: 1790344566482664
I20260925 13:56:31.971158 20713 ts_manager.cc:194] Re-registered known tserver with Master: 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:31.971097 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet f5a0bd0e205f4889be93739dfbd2a816 (table default.Table_04d053f331f243ce8e98beb7c00d4159 [id=745c9e0d4f2c4ce58c3a201c34f1f1e3]) successfully deleted
I20260925 13:56:31.972919 19271 master_service.cc:960] ReplaceTablet: received request to replace tablet 43692c9ae2dc460d9e32efad41c263c1 from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:31.978873 19939 tablet_bootstrap.cc:492] T 7e544c93488541da9ebb588a465b27b7 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:31.979372 19939 ts_tablet_manager.cc:1403] T 7e544c93488541da9ebb588a465b27b7 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.039s user 0.013s sys 0.000s
I20260925 13:56:31.982089 19939 raft_consensus.cc:359] T 7e544c93488541da9ebb588a465b27b7 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:31.982867 19939 raft_consensus.cc:385] T 7e544c93488541da9ebb588a465b27b7 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:31.983170 19939 raft_consensus.cc:740] T 7e544c93488541da9ebb588a465b27b7 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:31.983989 19939 consensus_queue.cc:260] T 7e544c93488541da9ebb588a465b27b7 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
W20260925 13:56:31.985862 19275 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ad3444ffde77425ebe3f39effa98d21d consensus state has no leader
I20260925 13:56:31.997967 19272 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 82972fa4868d4c59b3b9bb1240300938 (table default.Table_d3d2076fbb6241a68f04e83462a34182 [id=4232cecee9104120ac2521eb250bbd0d]). Expected version 1 got 0
I20260925 13:56:32.000125 19280 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_6ad3c828fd44418fbe6f3b0234f8f6f4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:32.001580 19280 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6ad3c828fd44418fbe6f3b0234f8f6f4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:32.006425 19275 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_640b2a7e12424a0c960c5ee25a876667"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:32.007793 19275 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_640b2a7e12424a0c960c5ee25a876667 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:32.010144 19942 ts_tablet_manager.cc:1434] T eff33a56361946edb556ac611d2299eb P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.093s user 0.006s sys 0.000s
I20260925 13:56:32.011011 19942 tablet_bootstrap.cc:492] T 4d01a791445149e7bee9493e1eec5c6b P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:32.013360 20860 catalog_manager.cc:1308] Loaded cluster ID: 83d15c6349674bc6880b5fd44887fcd6
I20260925 13:56:32.013589 20860 catalog_manager.cc:1601] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728: loading cluster ID for follower catalog manager: success
I20260925 13:56:32.012169 19272 catalog_manager.cc:5838] T 5f8a96ae6eec4f15a221f9aad537c094 P f790493ba7ad403993b678f2df2b55be reported cstate change: term changed from 0 to 1, leader changed from <none> to f790493ba7ad403993b678f2df2b55be (127.18.182.195). New cstate: current_term: 1 leader_uuid: "f790493ba7ad403993b678f2df2b55be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: HEALTHY } } }
I20260925 13:56:32.016300 19272 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 5f8a96ae6eec4f15a221f9aad537c094 (table default.Table_3a06919b658e46f893318b4e1ec8b52f [id=e6f2a872536b4e55b071231020903581]). Expected version 1 got 0
I20260925 13:56:32.019572 19673 tablet_service.cc:1511] Processing CreateTablet for tablet c1c7ae13fb3e407e8c718b6c6aef4b9a (DEFAULT_TABLE table=default.Table_f35518532f584ec5b480332f43e688ad [id=649b05108f9d4378b7b15ae4f648532a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.020538 19673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1c7ae13fb3e407e8c718b6c6aef4b9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:32.023564 19675 tablet_service.cc:1511] Processing CreateTablet for tablet 61eed9c89c5c459eb70ee034b09ecc67 (DEFAULT_TABLE table=default.Table_caeed989c841465a9b384a6641618002 [id=4ef1c53ff94541ab807402fbef936cf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.024477 19675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61eed9c89c5c459eb70ee034b09ecc67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:32.025697 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 218) took 4684 ms. Trace:
I20260925 13:56:32.026026 19955 rpcz_store.cc:276] 0925 13:56:27.341126 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:27.341256 (+ 130us) service_pool.cc:224] Handling call
0925 13:56:31.986310 (+4645054us) ts_tablet_manager.cc:1151] Deleting tablet f01fc02d12c8461f96ef0a93412ac641
0925 13:56:31.986350 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:32.025630 (+ 39280us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4599955}
I20260925 13:56:32.020839 19271 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 43692c9ae2dc460d9e32efad41c263c1
I20260925 13:56:32.030335 19271 catalog_manager.cc:6934] ReplaceTablet: tablet 43692c9ae2dc460d9e32efad41c263c1 deleted and replaced by tablet a426f65001f3420fb5b3672c57ae53d1
I20260925 13:56:32.031647 19271 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_770e2b27b6d4455787f7b0eff2a7221a" } modify_external_catalogs: true
I20260925 13:56:32.032122 19271 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_770e2b27b6d4455787f7b0eff2a7221a" } modify_external_catalogs: true
I20260925 13:56:32.037734 19669 tablet_service.cc:1558] Processing DeleteTablet for tablet 43692c9ae2dc460d9e32efad41c263c1 with delete_type TABLET_DATA_DELETED (replaced by tablet a426f65001f3420fb5b3672c57ae53d1) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:32.025727 19277 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 000aad39953c4688b543c7d882a08ec8
I20260925 13:56:32.039268 19277 catalog_manager.cc:6934] ReplaceTablet: tablet 000aad39953c4688b543c7d882a08ec8 deleted and replaced by tablet 3927adc85b2746d0977a3cb6f84ad68f
I20260925 13:56:32.039765 20761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35262 (request call id 0) took 1475 ms. Trace:
I20260925 13:56:32.040006 19277 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_9529f5b35c21450db26ad3efb6bece7b" } new_table_name: "default.Table_52321f7ed87340e4a745a378171311dd" modify_external_catalogs: true
I20260925 13:56:32.043634 20860 catalog_manager.cc:1623] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728: acquiring CA information for follower catalog manager: success
W20260925 13:56:32.044224 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 43692c9ae2dc460d9e32efad41c263c1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 43692c9ae2dc460d9e32efad41c263c1 already in progress: creating tablet
I20260925 13:56:32.044732 19669 tablet_service.cc:1558] Processing DeleteTablet for tablet 000aad39953c4688b543c7d882a08ec8 with delete_type TABLET_DATA_DELETED (replaced by tablet 3927adc85b2746d0977a3cb6f84ad68f) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:32.045385 20933 tablet_replica.cc:333] T 000aad39953c4688b543c7d882a08ec8 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:32.046192 20933 raft_consensus.cc:2243] T 000aad39953c4688b543c7d882a08ec8 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:32.046622 20933 raft_consensus.cc:2272] T 000aad39953c4688b543c7d882a08ec8 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:32.048511 20933 ts_tablet_manager.cc:1916] T 000aad39953c4688b543c7d882a08ec8 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:32.050733 20860 catalog_manager.cc:1651] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 13:56:32.062372 19275 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_2b6499b3f48241f58bae5603d9f439c8" } modify_external_catalogs: true
I20260925 13:56:32.063030 19275 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_2b6499b3f48241f58bae5603d9f439c8" } modify_external_catalogs: true
I20260925 13:56:32.074321 19832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "089dc4dfeb5243788dc03061c8743a81" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:32.081897 19832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "000aad39953c4688b543c7d882a08ec8" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:32.082579 19832 raft_consensus.cc:2393] T 000aad39953c4688b543c7d882a08ec8 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7c1b52895a14482fb54a6758f5054f8f in current term 1: Already voted for candidate 8f578869bc084a5d8a9a69f143954eb3 in this term.
I20260925 13:56:32.085228 19805 tablet_service.cc:1558] Processing DeleteTablet for tablet 9c5ef5b050624127990737acab3a8e9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:31 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:32.085602 19631 leader_election.cc:304] T f3b46e4fac694084b06b7086bcc596b9 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:32.087064 20927 raft_consensus.cc:2749] T f3b46e4fac694084b06b7086bcc596b9 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:32.087177 19631 leader_election.cc:304] T 089dc4dfeb5243788dc03061c8743a81 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:32.087745 20927 consensus_queue.cc:1048] T 321c740ba8eb4cbb8974305c5f8702af P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:32.088285 20854 raft_consensus.cc:2749] T 089dc4dfeb5243788dc03061c8743a81 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:32.100289 20644 ts_tablet_manager.cc:1929] T 78d8c01c88d24deeae21d647e7f205ba P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:32.101039 19816 tablet_service.cc:1558] Processing DeleteTablet for tablet 84188f52b54b477393d6a8433cbd1cb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:31 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:32.102619 19809 tablet_service.cc:1558] Processing DeleteTablet for tablet 43692c9ae2dc460d9e32efad41c263c1 with delete_type TABLET_DATA_DELETED (replaced by tablet a426f65001f3420fb5b3672c57ae53d1) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:32.103647 19819 tablet_service.cc:1558] Processing DeleteTablet for tablet 000aad39953c4688b543c7d882a08ec8 with delete_type TABLET_DATA_DELETED (replaced by tablet 3927adc85b2746d0977a3cb6f84ad68f) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:32.104171 20644 log.cc:1199] T 78d8c01c88d24deeae21d647e7f205ba P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/78d8c01c88d24deeae21d647e7f205ba
I20260925 13:56:32.104723 19805 tablet_service.cc:1511] Processing CreateTablet for tablet c1c7ae13fb3e407e8c718b6c6aef4b9a (DEFAULT_TABLE table=default.Table_f35518532f584ec5b480332f43e688ad [id=649b05108f9d4378b7b15ae4f648532a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.105908 19805 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1c7ae13fb3e407e8c718b6c6aef4b9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:32.106006 19819 tablet_service.cc:1511] Processing CreateTablet for tablet 61eed9c89c5c459eb70ee034b09ecc67 (DEFAULT_TABLE table=default.Table_caeed989c841465a9b384a6641618002 [id=4ef1c53ff94541ab807402fbef936cf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.106877 19819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61eed9c89c5c459eb70ee034b09ecc67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:32.112010 20644 ts_tablet_manager.cc:1950] T 78d8c01c88d24deeae21d647e7f205ba P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:32.113750 19669 tablet_service.cc:1511] Processing CreateTablet for tablet d7c8beb65fd64639b22374fe4d3ef400 (DEFAULT_TABLE table=default.Table_626a2de34ead48f4a50d72ea44cc25f2 [id=f4560686cb314de99329f63b5902971d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.114602 20644 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 239) took 3087 ms. Trace:
I20260925 13:56:32.119016 19669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7c8beb65fd64639b22374fe4d3ef400. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:32.136687 20644 rpcz_store.cc:276] 0925 13:56:29.027678 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:29.027886 (+ 208us) service_pool.cc:224] Handling call
0925 13:56:31.404111 (+2376225us) ts_tablet_manager.cc:1151] Deleting tablet 78d8c01c88d24deeae21d647e7f205ba
0925 13:56:31.404151 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:31.540291 (+136140us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:32.100181 (+559890us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:32.114547 (+ 14366us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":124168,"tablet-delete.queue_time_us":2375513}
I20260925 13:56:32.122419 20891 raft_consensus.cc:493] T 505a24ebe7014f80b3c184329cc74a9a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:32.144143 20891 raft_consensus.cc:515] T 505a24ebe7014f80b3c184329cc74a9a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:32.145916 20891 leader_election.cc:290] T 505a24ebe7014f80b3c184329cc74a9a P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
W20260925 13:56:32.104946 19773 leader_election.cc:343] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:32.147778 19773 leader_election.cc:304] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:32.148597 20891 raft_consensus.cc:2804] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:32.149341 20891 raft_consensus.cc:697] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be [term 1 LEADER]: Becoming Leader. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Running, Role: LEADER
I20260925 13:56:32.150241 20891 consensus_queue.cc:237] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:32.153177 19674 tablet_service.cc:1511] Processing CreateTablet for tablet 76faddb4346a4690bf22235c71a04b08 (DEFAULT_TABLE table=default.Table_3bc5f887568f417aa8c1269ddf12544a [id=f94d4009c9f64e41a079b87ef9998bea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.154165 19674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76faddb4346a4690bf22235c71a04b08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:32.155054 19631 leader_election.cc:304] T 000aad39953c4688b543c7d882a08ec8 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
W20260925 13:56:32.155628 19631 raft_consensus.cc:2686] T 000aad39953c4688b543c7d882a08ec8 P 7c1b52895a14482fb54a6758f5054f8f: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260925 13:56:32.160192 19271 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 4b36d5f94de04dca98b62e8128602cbb
I20260925 13:56:32.160674 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505a24ebe7014f80b3c184329cc74a9a" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:32.164450 19668 tablet_service.cc:1558] Processing DeleteTablet for tablet 4b36d5f94de04dca98b62e8128602cbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:32 UTC) from {username='slave'} at 127.0.0.1:45948
W20260925 13:56:32.169866 19277 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2fc32bc6d9ca43cb974c06714efe81a5 consensus state has no leader
I20260925 13:56:32.171959 20644 tablet_replica.cc:333] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:32.182837 20856 raft_consensus.cc:493] T eff33a56361946edb556ac611d2299eb P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:32.183300 20856 raft_consensus.cc:515] T eff33a56361946edb556ac611d2299eb P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:32.190343 19280 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_98879cfd535843ff9cbc54221b3cf028" } new_table_name: "default.Table_fd205659579c49559fd2d121b32274d4" modify_external_catalogs: true
I20260925 13:56:32.193080 19809 tablet_service.cc:1511] Processing CreateTablet for tablet d7c8beb65fd64639b22374fe4d3ef400 (DEFAULT_TABLE table=default.Table_626a2de34ead48f4a50d72ea44cc25f2 [id=f4560686cb314de99329f63b5902971d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.194198 19809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7c8beb65fd64639b22374fe4d3ef400. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:32.196177 19277 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_88f8d1b14c3b420a9dd5152c44b91577"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:32.203015 19277 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_88f8d1b14c3b420a9dd5152c44b91577 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 13:56:32.205135 19631 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 50157.5us
I20260925 13:56:32.210245 19271 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_b77c6557f7004edd8970cde538de23a9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:32.211848 19271 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b77c6557f7004edd8970cde538de23a9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:32.213138 19816 tablet_service.cc:1511] Processing CreateTablet for tablet 76faddb4346a4690bf22235c71a04b08 (DEFAULT_TABLE table=default.Table_3bc5f887568f417aa8c1269ddf12544a [id=f94d4009c9f64e41a079b87ef9998bea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.214190 19816 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76faddb4346a4690bf22235c71a04b08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:32.215456 19276 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet 5f8a96ae6eec4f15a221f9aad537c094 (table default.Table_3a06919b658e46f893318b4e1ec8b52f [id=e6f2a872536b4e55b071231020903581]). Expected version 1 got 0
I20260925 13:56:32.216271 20856 leader_election.cc:290] T eff33a56361946edb556ac611d2299eb P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:32.216176 19276 catalog_manager.cc:5838] T 089dc4dfeb5243788dc03061c8743a81 P 8f578869bc084a5d8a9a69f143954eb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "8f578869bc084a5d8a9a69f143954eb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:32.217883 19276 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet e9dbcd85dc8647c68c120ba3fcfe1790 (table default.Table_5bf9b87b5083469fad04eb3a80a4b8d5 [id=443536a0bd4d4d568d52f6ffe3d06ecb]). Expected version 1 got 0
I20260925 13:56:32.219913 19817 tablet_service.cc:1558] Processing DeleteTablet for tablet 4b36d5f94de04dca98b62e8128602cbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:32 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:32.226225 19942 tablet_bootstrap.cc:700] T 4d01a791445149e7bee9493e1eec5c6b P 7c1b52895a14482fb54a6758f5054f8f: Time spent opening tablet: real 0.128s user 0.003s sys 0.000s
I20260925 13:56:32.226840 19942 tablet_bootstrap.cc:654] T 4d01a791445149e7bee9493e1eec5c6b P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
W20260925 13:56:32.248423 19773 leader_election.cc:343] T 505a24ebe7014f80b3c184329cc74a9a P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:32.255801 19938 tablet_bootstrap.cc:492] T 11077cdb735840b5921e22f588059149 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:32.256263 19938 ts_tablet_manager.cc:1403] T 11077cdb735840b5921e22f588059149 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.705s user 0.009s sys 0.011s
I20260925 13:56:32.258490 19938 raft_consensus.cc:359] T 11077cdb735840b5921e22f588059149 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:32.263417 20856 raft_consensus.cc:493] T 5f8a96ae6eec4f15a221f9aad537c094 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:32.263845 20856 raft_consensus.cc:515] T 5f8a96ae6eec4f15a221f9aad537c094 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:32.266214 20856 leader_election.cc:290] T 5f8a96ae6eec4f15a221f9aad537c094 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:32.246907 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 78d8c01c88d24deeae21d647e7f205ba (table default.Table_17642b3f89974e5c97036637fe6d04d3 [id=4ca863b59dfd4410822e15705934b503]) successfully deleted
I20260925 13:56:32.268193 19276 catalog_manager.cc:5838] T 31049bccd170436893512a7e9ef435cc P 8f578869bc084a5d8a9a69f143954eb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "8f578869bc084a5d8a9a69f143954eb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:32.279666 19674 tablet_service.cc:1511] Processing CreateTablet for tablet e1c96465b34540ee865688f13a282699 (DEFAULT_TABLE table=default.Table_6800d8124538458699f660740653e710 [id=6fd743f0cf5a430fb7b37f6d8d12189c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.286558 20933 ts_tablet_manager.cc:1929] T 000aad39953c4688b543c7d882a08ec8 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:32.286978 20933 log.cc:1199] T 000aad39953c4688b543c7d882a08ec8 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/000aad39953c4688b543c7d882a08ec8
I20260925 13:56:32.296370 19673 tablet_service.cc:1511] Processing CreateTablet for tablet f3f0b400bec547afa3d484e54d6817e0 (DEFAULT_TABLE table=default.Table_4a32345f45fd4ca2a6bdce69c6c2073b [id=12202b5dbd364ad89710f39a368f6af7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.299741 20933 ts_tablet_manager.cc:1950] T 000aad39953c4688b543c7d882a08ec8 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:32.304942 19942 tablet_bootstrap.cc:492] T 4d01a791445149e7bee9493e1eec5c6b P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:32.305560 19942 ts_tablet_manager.cc:1403] T 4d01a791445149e7bee9493e1eec5c6b P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.295s user 0.009s sys 0.000s
I20260925 13:56:32.307873 19695 raft_consensus.cc:3060] T 5f8a96ae6eec4f15a221f9aad537c094 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:32.308636 19942 raft_consensus.cc:359] T 4d01a791445149e7bee9493e1eec5c6b P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:32.312745 19277 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_9f3ab8123a784bceb2bebdf1798fb643"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:32.314199 19277 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9f3ab8123a784bceb2bebdf1798fb643 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:32.315717 19275 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 67ff370d64d64e9da3a44747e44df9c8
I20260925 13:56:32.317453 19275 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_3dab1a66e22c4656b3c8eff23a8b5f0e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:32.318804 19275 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3dab1a66e22c4656b3c8eff23a8b5f0e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 13:56:32.319903 19280 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b23948807b2c4df9bf9a4da5f767e3b5 consensus state has no leader
I20260925 13:56:32.321017 19674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1c96465b34540ee865688f13a282699. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:32.328193 19942 raft_consensus.cc:385] T 4d01a791445149e7bee9493e1eec5c6b P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:32.328461 19942 raft_consensus.cc:740] T 4d01a791445149e7bee9493e1eec5c6b P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:32.331599 19942 consensus_queue.cc:260] T 4d01a791445149e7bee9493e1eec5c6b P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:32.334120 19942 ts_tablet_manager.cc:1434] T 4d01a791445149e7bee9493e1eec5c6b P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.028s user 0.004s sys 0.000s
I20260925 13:56:32.302599 20644 raft_consensus.cc:2243] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:32.335170 19830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff33a56361946edb556ac611d2299eb" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:32.339812 19830 raft_consensus.cc:2393] T eff33a56361946edb556ac611d2299eb P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7c1b52895a14482fb54a6758f5054f8f in current term 1: Already voted for candidate 8f578869bc084a5d8a9a69f143954eb3 in this term.
I20260925 13:56:32.338416 19669 tablet_service.cc:1558] Processing DeleteTablet for tablet 67ff370d64d64e9da3a44747e44df9c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:32 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:32.339380 19668 tablet_service.cc:1511] Processing CreateTablet for tablet 79a031b84d834f37b6ee98c50eea46a7 (DEFAULT_TABLE table=default.Table_32a95710635c459294ae80be58eed594 [id=9454c1fb60c246d99d306ebe585cd452]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.339645 19837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f8a96ae6eec4f15a221f9aad537c094" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:32.341018 19942 tablet_bootstrap.cc:492] T e9dbcd85dc8647c68c120ba3fcfe1790 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:32.341250 19668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79a031b84d834f37b6ee98c50eea46a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:32.335824 19819 tablet_service.cc:1558] Processing DeleteTablet for tablet 67ff370d64d64e9da3a44747e44df9c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:32 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:32.345630 19938 raft_consensus.cc:385] T 11077cdb735840b5921e22f588059149 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:32.345979 19938 raft_consensus.cc:740] T 11077cdb735840b5921e22f588059149 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:32.346750 19938 consensus_queue.cc:260] T 11077cdb735840b5921e22f588059149 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:32.356129 19805 tablet_service.cc:1511] Processing CreateTablet for tablet e1c96465b34540ee865688f13a282699 (DEFAULT_TABLE table=default.Table_6800d8124538458699f660740653e710 [id=6fd743f0cf5a430fb7b37f6d8d12189c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.357266 19805 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1c96465b34540ee865688f13a282699. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:32.360944 19942 tablet_bootstrap.cc:654] T e9dbcd85dc8647c68c120ba3fcfe1790 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:32.366622 19669 tablet_service.cc:1511] Processing CreateTablet for tablet 1d6cec4a434e4aa1bfee205273725f7c (DEFAULT_TABLE table=default.Table_ed1744bfe9644572b68ed9a8ccbb8d19 [id=2fd7560327cd490e9da2ea32e80cb8ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.370523 19669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d6cec4a434e4aa1bfee205273725f7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:32.376406 19271 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_7786394fcec84a879a5da3eb2377ca74" } new_table_name: "default.Table_0eb4fb3fbd9c4440afd9c10c8c318c56" modify_external_catalogs: true
I20260925 13:56:32.381268 19942 tablet_bootstrap.cc:492] T e9dbcd85dc8647c68c120ba3fcfe1790 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:32.381901 19942 ts_tablet_manager.cc:1403] T e9dbcd85dc8647c68c120ba3fcfe1790 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.041s user 0.008s sys 0.002s
I20260925 13:56:32.382757 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 000aad39953c4688b543c7d882a08ec8 (table default.Table_3a3e48a0641a43a1a783c6ba6374a708 [id=e64fe44a237f48859a288a87a565ee5e]) successfully deleted
I20260925 13:56:32.383409 19673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3f0b400bec547afa3d484e54d6817e0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:32.385821 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 4b36d5f94de04dca98b62e8128602cbb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4b36d5f94de04dca98b62e8128602cbb already in progress: creating tablet
W20260925 13:56:32.386675 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 67ff370d64d64e9da3a44747e44df9c8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 67ff370d64d64e9da3a44747e44df9c8 already in progress: creating tablet
I20260925 13:56:32.387856 19280 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_88f8d1b14c3b420a9dd5152c44b91577" } modify_external_catalogs: true
I20260925 13:56:32.388322 19280 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_88f8d1b14c3b420a9dd5152c44b91577" } modify_external_catalogs: true
I20260925 13:56:32.385198 19942 raft_consensus.cc:359] T e9dbcd85dc8647c68c120ba3fcfe1790 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:32.411711 19942 raft_consensus.cc:385] T e9dbcd85dc8647c68c120ba3fcfe1790 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:32.415159 19942 raft_consensus.cc:740] T e9dbcd85dc8647c68c120ba3fcfe1790 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:32.415285 19675 tablet_service.cc:1511] Processing CreateTablet for tablet 0b1e2dc2abc249528de6b9ff6fad4275 (DEFAULT_TABLE table=default.Table_7046f5be3ed44b3e94bd4d14e34c25ee [id=d7aaf1f0836e45f7a6d666f4f430c6f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.417007 19675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b1e2dc2abc249528de6b9ff6fad4275. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:32.416796 19942 consensus_queue.cc:260] T e9dbcd85dc8647c68c120ba3fcfe1790 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:32.422956 20795 raft_consensus.cc:493] T 78d8c01c88d24deeae21d647e7f205ba P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:32.423854 19939 ts_tablet_manager.cc:1434] T 7e544c93488541da9ebb588a465b27b7 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.444s user 0.006s sys 0.000s
I20260925 13:56:32.424521 20795 raft_consensus.cc:515] T 78d8c01c88d24deeae21d647e7f205ba P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:32.428383 20795 leader_election.cc:290] T 78d8c01c88d24deeae21d647e7f205ba P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:32.429476 19939 tablet_bootstrap.cc:492] T 1ce5d15e67644247b95960f444acc621 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:32.429876 20795 raft_consensus.cc:493] T 5a9b3dfaa7474611a1824d63eb906b43 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:32.431818 19955 tablet_replica.cc:333] T eb794cef052b43da831531455e2b5f15 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:32.430692 20795 raft_consensus.cc:515] T 5a9b3dfaa7474611a1824d63eb906b43 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:32.437333 20795 leader_election.cc:290] T 5a9b3dfaa7474611a1824d63eb906b43 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:32.441278 19809 tablet_service.cc:1511] Processing CreateTablet for tablet f3f0b400bec547afa3d484e54d6817e0 (DEFAULT_TABLE table=default.Table_4a32345f45fd4ca2a6bdce69c6c2073b [id=12202b5dbd364ad89710f39a368f6af7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.442114 19942 ts_tablet_manager.cc:1434] T e9dbcd85dc8647c68c120ba3fcfe1790 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.060s user 0.006s sys 0.000s
I20260925 13:56:32.442451 19809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3f0b400bec547afa3d484e54d6817e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:32.442986 19942 tablet_bootstrap.cc:492] T c5a9df833271480aa49feb7d9f925459 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:32.442783 19671 tablet_service.cc:1511] Processing CreateTablet for tablet aea3670acb654810a05eda9ddd52e79b (DEFAULT_TABLE table=default.Table_95a1130b07a34c82a6a6d7bb1c8245c8 [id=f4beec04dea942f98007b1f4cf33c9ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.448055 20891 consensus_queue.cc:1048] T 5f8a96ae6eec4f15a221f9aad537c094 P f790493ba7ad403993b678f2df2b55be [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:32.448930 20644 pending_rounds.cc:70] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be: Trying to abort 1 pending ops.
I20260925 13:56:32.449342 20644 pending_rounds.cc:77] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251449456738304 op_type: NO_OP noop_request { }
I20260925 13:56:32.449757 20644 raft_consensus.cc:2889] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:56:32.450078 20644 raft_consensus.cc:2272] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:32.451210 19666 tablet_service.cc:1511] Processing CreateTablet for tablet 2fc32bc6d9ca43cb974c06714efe81a5 (DEFAULT_TABLE table=default.Table_52321f7ed87340e4a745a378171311dd [id=f50cb5b15bd94b2383244f024f5271e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.452174 20644 ts_tablet_manager.cc:1916] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:32.456905 19938 ts_tablet_manager.cc:1434] T 11077cdb735840b5921e22f588059149 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.200s user 0.005s sys 0.000s
I20260925 13:56:32.457782 19938 tablet_bootstrap.cc:492] T 37947005aaf14f27a6e5d1f30c67250c P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:32.459596 19671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aea3670acb654810a05eda9ddd52e79b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:32.468552 19773 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.194:38641, user_credentials={real_user=slave}} blocked reactor thread for 129289us
I20260925 13:56:32.478893 19279 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 82972fa4868d4c59b3b9bb1240300938 (table default.Table_d3d2076fbb6241a68f04e83462a34182 [id=4232cecee9104120ac2521eb250bbd0d]). Expected version 1 got 0
I20260925 13:56:32.479504 19942 tablet_bootstrap.cc:654] T c5a9df833271480aa49feb7d9f925459 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:32.480825 19279 catalog_manager.cc:5838] T 321c740ba8eb4cbb8974305c5f8702af P 7c1b52895a14482fb54a6758f5054f8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194). New cstate: current_term: 1 leader_uuid: "7c1b52895a14482fb54a6758f5054f8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:32.482182 19279 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet f3b46e4fac694084b06b7086bcc596b9 (table default.Table_2efda975ab0a464eaf96f1f9a48eff1c [id=c31f813d79a244afa726df0238376aba]). Expected version 1 got 0
I20260925 13:56:32.482730 19273 catalog_manager.cc:6391] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: default.Table_1b1f36048d2a4a3c86e513e46bcf5471 [id=0c13bfd171f2464b887f15d085439fa3] alter complete (version 1)
W20260925 13:56:32.483307 19273 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.194:34125 (request call id 45) took 3150 ms (client timeout 1000 ms). Trace:
I20260925 13:56:32.486812 19274 catalog_manager.cc:6391] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: default.Table_1da8b04f68ab4682a8ec32ad47854930 [id=b954461264164e969a40af810627fcb2] alter complete (version 1)
W20260925 13:56:32.489171 19274 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.193:37907 (request call id 39) took 2717 ms (client timeout 1000 ms). Trace:
I20260925 13:56:32.495244 19666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fc32bc6d9ca43cb974c06714efe81a5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:32.498906 19272 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.195:57799 (request call id 42) took 1100 ms (client timeout 1000 ms). Trace:
I20260925 13:56:32.500706 19942 tablet_bootstrap.cc:492] T c5a9df833271480aa49feb7d9f925459 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:32.502048 19942 ts_tablet_manager.cc:1403] T c5a9df833271480aa49feb7d9f925459 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.059s user 0.010s sys 0.000s
I20260925 13:56:32.504586 19942 raft_consensus.cc:359] T c5a9df833271480aa49feb7d9f925459 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:32.505177 19942 raft_consensus.cc:385] T c5a9df833271480aa49feb7d9f925459 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:32.505404 19942 raft_consensus.cc:740] T c5a9df833271480aa49feb7d9f925459 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:32.507458 19942 consensus_queue.cc:260] T c5a9df833271480aa49feb7d9f925459 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
W20260925 13:56:32.499241 19272 rpcz_store.cc:269] 0925 13:56:31.398855 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:31.453635 (+ 54780us) service_pool.cc:224] Handling call
0925 13:56:31.464208 (+ 10573us) spinlock_profiling.cc:243] Waited 1.42 ms on lock 0x7b5400040454. stack: 00007fdf0a30b323 00007fdf0a30b220 00007fdf09eaf574 0000000000516964 00000000005168f9 00007fdf114095c0 00007fdf114a86fa 00007fdf13219d8c 00007fdf1331801e 00007fdf10ebcfb4 00007fdf10ebcf42 00007fdf10ebce71 00007fdf10ebcdec 00007fdf10ebc0a2 00007fdf0f16e09c 00007fdf0f16d4d6
0925 13:56:32.498837 (+1034629us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:32.119729 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:32.120065 (+ 336us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:32.120086 (+ 21us) write_op.cc:435] Acquired schema lock
0925 13:56:32.120098 (+ 12us) tablet.cc:662] Decoding operations
0925 13:56:32.120324 (+ 226us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:32.120356 (+ 32us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:32.120378 (+ 22us) tablet.cc:685] Acquiring locks for 1 operations
0925 13:56:32.120510 (+ 132us) tablet.cc:701] Row locks acquired
0925 13:56:32.120521 (+ 11us) write_op.cc:260] PREPARE: finished
0925 13:56:32.120605 (+ 84us) write_op.cc:270] Start()
0925 13:56:32.120671 (+ 66us) write_op.cc:276] Timestamp: P: 1790344592120561 usec, L: 0
0925 13:56:32.120708 (+ 37us) op_driver.cc:348] REPLICATION: starting
0925 13:56:32.121103 (+ 395us) log.cc:844] Serialized 536 byte log entry
0925 13:56:32.167679 (+ 46576us) op_driver.cc:464] REPLICATION: finished
0925 13:56:32.169549 (+ 1870us) write_op.cc:301] APPLY: starting
0925 13:56:32.169591 (+ 42us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:32.169671 (+ 80us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:32.169767 (+ 96us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:32.169905 (+ 138us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:32.170023 (+ 118us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 13:56:32.170068 (+ 45us) write_op.cc:312] APPLY: finished
0925 13:56:32.170224 (+ 156us) log.cc:844] Serialized 23 byte log entry
0925 13:56:32.170327 (+ 103us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:32.170378 (+ 51us) write_op.cc:454] Released schema lock
0925 13:56:32.170405 (+ 27us) write_op.cc:341] FINISH: Updating metrics
0925 13:56:32.170598 (+ 193us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:32.170626 (+ 28us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":11,"spinlock_wait_cycles":3261824,"child_traces":[["op",{"apply.queue_time_us":1739,"apply.run_cpu_time_us":1013,"apply.run_wall_time_us":1013,"num_ops":1,"prepare.queue_time_us":65961,"prepare.run_cpu_time_us":1549,"prepare.run_wall_time_us":1547,"replication_time_us":46887}]]}
I20260925 13:56:32.515210 20961 tablet.cc:1751] T e3cf9676160c4af08f1f62ae952b4c57 P 7c1b52895a14482fb54a6758f5054f8f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:32.516716 19942 ts_tablet_manager.cc:1434] T c5a9df833271480aa49feb7d9f925459 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.014s user 0.005s sys 0.000s
I20260925 13:56:32.517665 19942 tablet_bootstrap.cc:492] T 78d8c01c88d24deeae21d647e7f205ba P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
W20260925 13:56:32.519369 19278 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.194:34125 (request call id 46) took 2001 ms (client timeout 1000 ms). Trace:
I20260925 13:56:32.522614 19277 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_77f9464ff4694e999c6250b01465c6c0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:32.524114 19277 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_77f9464ff4694e999c6250b01465c6c0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:32.530064 19819 tablet_service.cc:1511] Processing CreateTablet for tablet 1d6cec4a434e4aa1bfee205273725f7c (DEFAULT_TABLE table=default.Table_ed1744bfe9644572b68ed9a8ccbb8d19 [id=2fd7560327cd490e9da2ea32e80cb8ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.531355 19802 tablet_service.cc:1511] Processing CreateTablet for tablet aea3670acb654810a05eda9ddd52e79b (DEFAULT_TABLE table=default.Table_95a1130b07a34c82a6a6d7bb1c8245c8 [id=f4beec04dea942f98007b1f4cf33c9ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.530174 19817 tablet_service.cc:1511] Processing CreateTablet for tablet 0b1e2dc2abc249528de6b9ff6fad4275 (DEFAULT_TABLE table=default.Table_7046f5be3ed44b3e94bd4d14e34c25ee [id=d7aaf1f0836e45f7a6d666f4f430c6f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.532538 19816 tablet_service.cc:1511] Processing CreateTablet for tablet 79a031b84d834f37b6ee98c50eea46a7 (DEFAULT_TABLE table=default.Table_32a95710635c459294ae80be58eed594 [id=9454c1fb60c246d99d306ebe585cd452]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 13:56:32.483628 19273 rpcz_store.cc:269] 0925 13:56:29.332916 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:29.336323 (+ 3407us) service_pool.cc:224] Handling call
0925 13:56:32.483248 (+3146925us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:30.819044 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:30.848041 (+ 28997us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:30.848067 (+ 26us) write_op.cc:435] Acquired schema lock
0925 13:56:30.848081 (+ 14us) tablet.cc:662] Decoding operations
0925 13:56:30.848605 (+ 524us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:30.848641 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:30.848665 (+ 24us) tablet.cc:685] Acquiring locks for 1 operations
0925 13:56:30.848811 (+ 146us) tablet.cc:701] Row locks acquired
0925 13:56:30.848825 (+ 14us) write_op.cc:260] PREPARE: finished
0925 13:56:30.849102 (+ 277us) write_op.cc:270] Start()
0925 13:56:30.849187 (+ 85us) write_op.cc:276] Timestamp: P: 1790344590848892 usec, L: 0
0925 13:56:30.849205 (+ 18us) op_driver.cc:348] REPLICATION: starting
0925 13:56:30.867268 (+ 18063us) log.cc:844] Serialized 536 byte log entry
0925 13:56:30.877623 (+ 10355us) spinlock_profiling.cc:243] Waited 7.86 ms on lock 0x7b58000605d0. stack: 00007fdf0a30b323 00007fdf0a30b220 00007fdf09eaf574 0000000000516964 00000000005168f9 00007fdf114095c0 00007fdf113f996c 00007fdf11400283 00007fdf11400209 00007fdf11400199 00007fdf11400161 00007fdf113ff32d 00000000005bd4e4 00000000005bd319 00007fdf0a33d3e5 00007fdf0a33f751
0925 13:56:30.991056 (+113433us) op_driver.cc:464] REPLICATION: finished
0925 13:56:30.994273 (+ 3217us) write_op.cc:301] APPLY: starting
0925 13:56:30.994337 (+ 64us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:30.994434 (+ 97us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:30.994452 (+ 18us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:30.994624 (+ 172us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:30.994762 (+ 138us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 13:56:30.994820 (+ 58us) write_op.cc:312] APPLY: finished
0925 13:56:30.995011 (+ 191us) log.cc:844] Serialized 23 byte log entry
0925 13:56:30.995152 (+ 141us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:30.995213 (+ 61us) write_op.cc:454] Released schema lock
0925 13:56:30.995244 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0925 13:56:30.995479 (+ 235us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:30.995504 (+ 25us) write_op.cc:454] Released schema lock
Related trace 'op':
0925 13:56:31.583733 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:31.584108 (+ 375us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:31.584131 (+ 23us) write_op.cc:435] Acquired schema lock
0925 13:56:31.584144 (+ 13us) tablet.cc:662] Decoding operations
0925 13:56:31.584382 (+ 238us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:31.584416 (+ 34us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:31.584437 (+ 21us) tablet.cc:685] Acquiring locks for 1 operations
0925 13:56:31.584575 (+ 138us) tablet.cc:701] Row locks acquired
0925 13:56:31.584588 (+ 13us) write_op.cc:260] PREPARE: finished
0925 13:56:31.584687 (+ 99us) write_op.cc:270] Start()
0925 13:56:31.584776 (+ 89us) write_op.cc:276] Timestamp: P: 1790344591584649 usec, L: 0
0925 13:56:31.584789 (+ 13us) op_driver.cc:348] REPLICATION: starting
0925 13:56:31.585218 (+ 429us) log.cc:844] Serialized 356 byte log entry
0925 13:56:31.612624 (+ 27406us) op_driver.cc:464] REPLICATION: finished
0925 13:56:31.615043 (+ 2419us) write_op.cc:301] APPLY: starting
0925 13:56:31.615103 (+ 60us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:31.615208 (+ 105us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:31.615315 (+ 107us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:31.620046 (+ 4731us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:31.620166 (+ 120us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 13:56:31.620210 (+ 44us) write_op.cc:312] APPLY: finished
0925 13:56:31.620358 (+ 148us) log.cc:844] Serialized 23 byte log entry
0925 13:56:31.620480 (+ 122us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:31.620529 (+ 49us) write_op.cc:454] Released schema lock
0925 13:56:31.620556 (+ 27us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":3070,"apply.run_cpu_time_us":1176,"apply.run_wall_time_us":1176,"num_ops":1,"prepare.queue_time_us":10547,"prepare.run_cpu_time_us":1954,"prepare.run_wall_time_us":48494,"raft.queue_time_us":815,"raft.run_cpu_time_us":1582,"raft.run_wall_time_us":9374,"replication_time_us":141757,"spinlock_wait_cycles":18068352}],["op",{"apply.queue_time_us":2276,"apply.run_cpu_time_us":1138,"apply.run_wall_time_us":5799,"num_ops":1,"prepare.queue_time_us":581735,"prepare.run_cpu_time_us":1672,"prepare.run_wall_time_us":2517,"replication_time_us":27751,"thread_start_us":17710,"threads_started":1}]]}
W20260925 13:56:32.489568 19274 rpcz_store.cc:269] 0925 13:56:29.771386 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:29.771503 (+ 117us) service_pool.cc:224] Handling call
0925 13:56:32.489032 (+2717529us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:30.499619 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:30.500022 (+ 403us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:30.500044 (+ 22us) write_op.cc:435] Acquired schema lock
0925 13:56:30.500057 (+ 13us) tablet.cc:662] Decoding operations
0925 13:56:30.500284 (+ 227us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:30.500317 (+ 33us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:30.500339 (+ 22us) tablet.cc:685] Acquiring locks for 1 operations
0925 13:56:30.500478 (+ 139us) tablet.cc:701] Row locks acquired
0925 13:56:30.500491 (+ 13us) write_op.cc:260] PREPARE: finished
0925 13:56:30.500588 (+ 97us) write_op.cc:270] Start()
0925 13:56:30.500664 (+ 76us) write_op.cc:276] Timestamp: P: 1790344590500551 usec, L: 0
0925 13:56:30.500678 (+ 14us) op_driver.cc:348] REPLICATION: starting
0925 13:56:30.501193 (+ 515us) log.cc:844] Serialized 356 byte log entry
0925 13:56:30.632006 (+130813us) op_driver.cc:464] REPLICATION: finished
0925 13:56:30.635338 (+ 3332us) write_op.cc:301] APPLY: starting
0925 13:56:30.635410 (+ 72us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:30.635493 (+ 83us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:30.635575 (+ 82us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:30.635721 (+ 146us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:30.635834 (+ 113us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 13:56:30.635869 (+ 35us) write_op.cc:312] APPLY: finished
0925 13:56:30.636049 (+ 180us) log.cc:844] Serialized 23 byte log entry
0925 13:56:30.636195 (+ 146us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:30.636246 (+ 51us) write_op.cc:454] Released schema lock
0925 13:56:30.636278 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0925 13:56:30.636475 (+ 197us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:30.636495 (+ 20us) write_op.cc:454] Released schema lock
Related trace 'op':
0925 13:56:31.586346 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:31.586679 (+ 333us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:31.586698 (+ 19us) write_op.cc:435] Acquired schema lock
0925 13:56:31.586710 (+ 12us) tablet.cc:662] Decoding operations
0925 13:56:31.586937 (+ 227us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:31.586968 (+ 31us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:31.586987 (+ 19us) tablet.cc:685] Acquiring locks for 1 operations
0925 13:56:31.587111 (+ 124us) tablet.cc:701] Row locks acquired
0925 13:56:31.587124 (+ 13us) write_op.cc:260] PREPARE: finished
0925 13:56:31.587213 (+ 89us) write_op.cc:270] Start()
0925 13:56:31.587280 (+ 67us) write_op.cc:276] Timestamp: P: 1790344591587176 usec, L: 0
0925 13:56:31.587295 (+ 15us) op_driver.cc:348] REPLICATION: starting
0925 13:56:31.587698 (+ 403us) log.cc:844] Serialized 356 byte log entry
0925 13:56:31.630723 (+ 43025us) op_driver.cc:464] REPLICATION: finished
0925 13:56:31.630905 (+ 182us) write_op.cc:301] APPLY: starting
0925 13:56:31.630952 (+ 47us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:31.631029 (+ 77us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:31.631213 (+ 184us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:31.631379 (+ 166us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:31.631522 (+ 143us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 13:56:31.631582 (+ 60us) write_op.cc:312] APPLY: finished
0925 13:56:31.631718 (+ 136us) log.cc:844] Serialized 23 byte log entry
0925 13:56:31.632734 (+ 1016us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:31.632784 (+ 50us) write_op.cc:454] Released schema lock
0925 13:56:31.632810 (+ 26us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":8,"child_traces":[["op",{"apply.queue_time_us":3182,"apply.run_cpu_time_us":1123,"apply.run_wall_time_us":1120,"num_ops":1,"prepare.queue_time_us":593311,"prepare.run_cpu_time_us":1750,"prepare.run_wall_time_us":1798,"replication_time_us":131240}],["op",{"apply.queue_time_us":67,"apply.run_cpu_time_us":1115,"apply.run_wall_time_us":2160,"num_ops":1,"prepare.queue_time_us":576295,"prepare.run_cpu_time_us":1529,"prepare.run_wall_time_us":2747,"replication_time_us":43355}]]}
I20260925 13:56:32.546943 19273 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_871b30a592064a0abf86c3339a7d5847" } modify_external_catalogs: true
I20260925 13:56:32.549731 19273 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_871b30a592064a0abf86c3339a7d5847" } modify_external_catalogs: true
W20260925 13:56:32.519744 19278 rpcz_store.cc:269] 0925 13:56:30.494024 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:30.591089 (+ 97065us) service_pool.cc:224] Handling call
0925 13:56:32.495313 (+1904224us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:31.589191 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:31.589546 (+ 355us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:31.589566 (+ 20us) write_op.cc:435] Acquired schema lock
0925 13:56:31.589579 (+ 13us) tablet.cc:662] Decoding operations
0925 13:56:31.589843 (+ 264us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:31.589872 (+ 29us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:31.589893 (+ 21us) tablet.cc:685] Acquiring locks for 2 operations
0925 13:56:31.590109 (+ 216us) tablet.cc:701] Row locks acquired
0925 13:56:31.590121 (+ 12us) write_op.cc:260] PREPARE: finished
0925 13:56:31.590202 (+ 81us) write_op.cc:270] Start()
0925 13:56:31.590266 (+ 64us) write_op.cc:276] Timestamp: P: 1790344591590169 usec, L: 0
0925 13:56:31.590281 (+ 15us) op_driver.cc:348] REPLICATION: starting
0925 13:56:31.590658 (+ 377us) log.cc:844] Serialized 919 byte log entry
0925 13:56:31.637323 (+ 46665us) op_driver.cc:464] REPLICATION: finished
0925 13:56:31.637556 (+ 233us) write_op.cc:301] APPLY: starting
0925 13:56:31.637648 (+ 92us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:31.637764 (+ 116us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:31.637853 (+ 89us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:31.638106 (+ 253us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:31.638229 (+ 123us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 13:56:31.638267 (+ 38us) write_op.cc:312] APPLY: finished
0925 13:56:31.638413 (+ 146us) log.cc:844] Serialized 29 byte log entry
0925 13:56:31.638956 (+ 543us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:31.639036 (+ 80us) write_op.cc:454] Released schema lock
0925 13:56:31.639077 (+ 41us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":3043,"reported_tablets":9,"child_traces":[["op",{"apply.queue_time_us":91,"apply.run_cpu_time_us":1350,"apply.run_wall_time_us":5935,"num_ops":2,"prepare.queue_time_us":568845,"prepare.run_cpu_time_us":1752,"prepare.run_wall_time_us":39152,"raft.queue_time_us":79,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":167,"replication_time_us":46971}]]}
I20260925 13:56:32.553274 19942 tablet_bootstrap.cc:654] T 78d8c01c88d24deeae21d647e7f205ba P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:32.564527 19939 tablet_bootstrap.cc:654] T 1ce5d15e67644247b95960f444acc621 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:32.572033 19819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d6cec4a434e4aa1bfee205273725f7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:32.590610 19942 tablet_bootstrap.cc:492] T 78d8c01c88d24deeae21d647e7f205ba P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:32.590999 19942 ts_tablet_manager.cc:1403] T 78d8c01c88d24deeae21d647e7f205ba P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.073s user 0.008s sys 0.000s
I20260925 13:56:32.593055 19278 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_416e65530d694bc6b6f507558740dd0b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:56:32.593427 19942 raft_consensus.cc:359] T 78d8c01c88d24deeae21d647e7f205ba P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:32.594028 19942 raft_consensus.cc:385] T 78d8c01c88d24deeae21d647e7f205ba P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:32.594290 19942 raft_consensus.cc:740] T 78d8c01c88d24deeae21d647e7f205ba P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
W20260925 13:56:32.594708 19278 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_416e65530d694bc6b6f507558740dd0b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:32.594946 19942 consensus_queue.cc:260] T 78d8c01c88d24deeae21d647e7f205ba P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:32.596385 19531 tablet_service.cc:1558] Processing DeleteTablet for tablet 43692c9ae2dc460d9e32efad41c263c1 with delete_type TABLET_DATA_DELETED (replaced by tablet a426f65001f3420fb5b3672c57ae53d1) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:32.597604 19532 tablet_service.cc:1511] Processing CreateTablet for tablet 61eed9c89c5c459eb70ee034b09ecc67 (DEFAULT_TABLE table=default.Table_caeed989c841465a9b384a6641618002 [id=4ef1c53ff94541ab807402fbef936cf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.597772 19531 tablet_service.cc:1511] Processing CreateTablet for tablet 76faddb4346a4690bf22235c71a04b08 (DEFAULT_TABLE table=default.Table_3bc5f887568f417aa8c1269ddf12544a [id=f94d4009c9f64e41a079b87ef9998bea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.598771 19532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61eed9c89c5c459eb70ee034b09ecc67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:32.599756 19524 tablet_service.cc:1558] Processing DeleteTablet for tablet 000aad39953c4688b543c7d882a08ec8 with delete_type TABLET_DATA_DELETED (replaced by tablet 3927adc85b2746d0977a3cb6f84ad68f) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:32.600874 19520 tablet_service.cc:1511] Processing CreateTablet for tablet d7c8beb65fd64639b22374fe4d3ef400 (DEFAULT_TABLE table=default.Table_626a2de34ead48f4a50d72ea44cc25f2 [id=f4560686cb314de99329f63b5902971d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.601835 19520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7c8beb65fd64639b22374fe4d3ef400. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:32.602862 19522 tablet_service.cc:1558] Processing DeleteTablet for tablet 4b36d5f94de04dca98b62e8128602cbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:32 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:32.603739 19802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aea3670acb654810a05eda9ddd52e79b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:32.605695 19942 ts_tablet_manager.cc:1434] T 78d8c01c88d24deeae21d647e7f205ba P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.014s user 0.006s sys 0.000s
I20260925 13:56:32.607793 19817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b1e2dc2abc249528de6b9ff6fad4275. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:32.607888 19942 tablet_bootstrap.cc:492] T b3172d99afee4a4c878e71b645037917 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:32.609200 19518 tablet_service.cc:1558] Processing DeleteTablet for tablet 67ff370d64d64e9da3a44747e44df9c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:32 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:32.610496 19525 tablet_service.cc:1511] Processing CreateTablet for tablet 79a031b84d834f37b6ee98c50eea46a7 (DEFAULT_TABLE table=default.Table_32a95710635c459294ae80be58eed594 [id=9454c1fb60c246d99d306ebe585cd452]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.610569 19518 tablet_service.cc:1511] Processing CreateTablet for tablet 0b1e2dc2abc249528de6b9ff6fad4275 (DEFAULT_TABLE table=default.Table_7046f5be3ed44b3e94bd4d14e34c25ee [id=d7aaf1f0836e45f7a6d666f4f430c6f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.611768 19525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79a031b84d834f37b6ee98c50eea46a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:32.612315 19529 tablet_service.cc:1511] Processing CreateTablet for tablet 2fc32bc6d9ca43cb974c06714efe81a5 (DEFAULT_TABLE table=default.Table_52321f7ed87340e4a745a378171311dd [id=f50cb5b15bd94b2383244f024f5271e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.613497 19529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fc32bc6d9ca43cb974c06714efe81a5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:32.614060 19264 catalog_manager.cc:5043] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): delete failed for tablet f01fc02d12c8461f96ef0a93412ac641 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f01fc02d12c8461f96ef0a93412ac641 already in progress: creating tablet
I20260925 13:56:32.615839 19521 tablet_service.cc:1511] Processing CreateTablet for tablet aea3670acb654810a05eda9ddd52e79b (DEFAULT_TABLE table=default.Table_95a1130b07a34c82a6a6d7bb1c8245c8 [id=f4beec04dea942f98007b1f4cf33c9ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.616812 19521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aea3670acb654810a05eda9ddd52e79b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:32.619760 19523 tablet_service.cc:1558] Processing DeleteTablet for tablet 8dc5ea164d6749b0926c63d6bda9dcc5 with delete_type TABLET_DATA_DELETED (replaced by tablet d0caac86b259433e9c490aa71ac9d329) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:32.620642 19519 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d01a791445149e7bee9493e1eec5c6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:21 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:32.622016 19526 tablet_service.cc:1511] Processing CreateTablet for tablet 1d6cec4a434e4aa1bfee205273725f7c (DEFAULT_TABLE table=default.Table_ed1744bfe9644572b68ed9a8ccbb8d19 [id=2fd7560327cd490e9da2ea32e80cb8ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.623157 19526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d6cec4a434e4aa1bfee205273725f7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:32.624008 19522 tablet_service.cc:1511] Processing CreateTablet for tablet f3f0b400bec547afa3d484e54d6817e0 (DEFAULT_TABLE table=default.Table_4a32345f45fd4ca2a6bdce69c6c2073b [id=12202b5dbd364ad89710f39a368f6af7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.625387 19524 tablet_service.cc:1511] Processing CreateTablet for tablet e1c96465b34540ee865688f13a282699 (DEFAULT_TABLE table=default.Table_6800d8124538458699f660740653e710 [id=6fd743f0cf5a430fb7b37f6d8d12189c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.629559 19816 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79a031b84d834f37b6ee98c50eea46a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:32.596243 19533 tablet_service.cc:1511] Processing CreateTablet for tablet c1c7ae13fb3e407e8c718b6c6aef4b9a (DEFAULT_TABLE table=default.Table_f35518532f584ec5b480332f43e688ad [id=649b05108f9d4378b7b15ae4f648532a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.639655 20891 raft_consensus.cc:493] T 11077cdb735840b5921e22f588059149 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:32.640166 20891 raft_consensus.cc:515] T 11077cdb735840b5921e22f588059149 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:32.642230 20891 leader_election.cc:290] T 11077cdb735840b5921e22f588059149 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:32.040050 20761 rpcz_store.cc:276] 0925 13:56:30.564765 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:30.564891 (+ 126us) service_pool.cc:224] Handling call
0925 13:56:30.655018 (+ 90127us) raft_consensus.cc:1471] Updating replica for 353 ops
0925 13:56:30.706238 (+ 51220us) raft_consensus.cc:1534] Early marking committed up to index 5
0925 13:56:30.726661 (+ 20423us) raft_consensus.cc:1539] Triggering prepare for 353 ops
0925 13:56:30.727045 (+ 384us) write_op.cc:270] Start()
0925 13:56:30.727153 (+ 108us) write_op.cc:276] Timestamp: P: 1790344568978292 usec, L: 0
0925 13:56:31.041279 (+314126us) write_op.cc:270] Start()
0925 13:56:31.041403 (+ 124us) write_op.cc:276] Timestamp: P: 1790344568982673 usec, L: 0
0925 13:56:31.045020 (+ 3617us) write_op.cc:270] Start()
0925 13:56:31.045169 (+ 149us) write_op.cc:276] Timestamp: P: 1790344568987329 usec, L: 0
0925 13:56:31.047976 (+ 2807us) write_op.cc:270] Start()
0925 13:56:31.058088 (+ 10112us) write_op.cc:276] Timestamp: P: 1790344569016651 usec, L: 0
0925 13:56:31.094803 (+ 36715us) write_op.cc:270] Start()
0925 13:56:31.094920 (+ 117us) write_op.cc:276] Timestamp: P: 1790344569018977 usec, L: 0
0925 13:56:31.096737 (+ 1817us) write_op.cc:270] Start()
0925 13:56:31.096847 (+ 110us) write_op.cc:276] Timestamp: P: 1790344569059198 usec, L: 0
0925 13:56:31.098966 (+ 2119us) write_op.cc:270] Start()
0925 13:56:31.099104 (+ 138us) write_op.cc:276] Timestamp: P: 1790344569068375 usec, L: 0
0925 13:56:31.099629 (+ 525us) write_op.cc:270] Start()
0925 13:56:31.106221 (+ 6592us) write_op.cc:276] Timestamp: P: 1790344569076035 usec, L: 0
0925 13:56:31.106858 (+ 637us) write_op.cc:270] Start()
0925 13:56:31.106942 (+ 84us) write_op.cc:276] Timestamp: P: 1790344569120846 usec, L: 0
0925 13:56:31.121983 (+ 15041us) write_op.cc:270] Start()
0925 13:56:31.122103 (+ 120us) write_op.cc:276] Timestamp: P: 1790344569220263 usec, L: 0
0925 13:56:31.122730 (+ 627us) write_op.cc:270] Start()
0925 13:56:31.122860 (+ 130us) write_op.cc:276] Timestamp: P: 1790344569269588 usec, L: 0
0925 13:56:31.123441 (+ 581us) write_op.cc:270] Start()
0925 13:56:31.123576 (+ 135us) write_op.cc:276] Timestamp: P: 1790344569282615 usec, L: 0
0925 13:56:31.124178 (+ 602us) write_op.cc:270] Start()
0925 13:56:31.124277 (+ 99us) write_op.cc:276] Timestamp: P: 1790344569292017 usec, L: 0
0925 13:56:31.124850 (+ 573us) write_op.cc:270] Start()
0925 13:56:31.124947 (+ 97us) write_op.cc:276] Timestamp: P: 1790344569382408 usec, L: 0
0925 13:56:31.125522 (+ 575us) write_op.cc:270] Start()
0925 13:56:31.125618 (+ 96us) write_op.cc:276] Timestamp: P: 1790344569422938 usec, L: 0
0925 13:56:31.126164 (+ 546us) write_op.cc:270] Start()
0925 13:56:31.126259 (+ 95us) write_op.cc:276] Timestamp: P: 1790344569446716 usec, L: 0
0925 13:56:31.126864 (+ 605us) write_op.cc:270] Start()
0925 13:56:31.126960 (+ 96us) write_op.cc:276] Timestamp: P: 1790344569536236 usec, L: 0
0925 13:56:31.150324 (+ 23364us) write_op.cc:270] Start()
0925 13:56:31.150882 (+ 558us) write_op.cc:276] Timestamp: P: 1790344569541025 usec, L: 0
0925 13:56:31.155104 (+ 4222us) write_op.cc:270] Start()
0925 13:56:31.155771 (+ 667us) write_op.cc:276] Timestamp: P: 1790344569636109 usec, L: 0
0925 13:56:31.172384 (+ 16613us) write_op.cc:270] Start()
0925 13:56:31.172487 (+ 103us) write_op.cc:276] Timestamp: P: 1790344569657315 usec, L: 0
0925 13:56:31.173040 (+ 553us) write_op.cc:270] Start()
0925 13:56:31.173126 (+ 86us) write_op.cc:276] Timestamp: P: 1790344569678433 usec, L: 0
0925 13:56:31.173672 (+ 546us) write_op.cc:270] Start()
0925 13:56:31.173751 (+ 79us) write_op.cc:276] Timestamp: P: 1790344569716557 usec, L: 0
0925 13:56:31.174237 (+ 486us) write_op.cc:270] Start()
0925 13:56:31.174320 (+ 83us) write_op.cc:276] Timestamp: P: 1790344569742243 usec, L: 0
0925 13:56:31.175260 (+ 940us) write_op.cc:270] Start()
0925 13:56:31.183020 (+ 7760us) write_op.cc:276] Timestamp: P: 1790344569762012 usec, L: 0
0925 13:56:31.183787 (+ 767us) write_op.cc:270] Start()
0925 13:56:31.183891 (+ 104us) write_op.cc:276] Timestamp: P: 1790344569814933 usec, L: 0
0925 13:56:31.184392 (+ 501us) write_op.cc:270] Start()
0925 13:56:31.184484 (+ 92us) write_op.cc:276] Timestamp: P: 1790344569894398 usec, L: 0
0925 13:56:31.184988 (+ 504us) write_op.cc:270] Start()
0925 13:56:31.185064 (+ 76us) write_op.cc:276] Timestamp: P: 1790344569898110 usec, L: 0
0925 13:56:31.185490 (+ 426us) write_op.cc:270] Start()
0925 13:56:31.185563 (+ 73us) write_op.cc:276] Timestamp: P: 1790344569994421 usec, L: 0
0925 13:56:31.186052 (+ 489us) write_op.cc:270] Start()
0925 13:56:31.186136 (+ 84us) write_op.cc:276] Timestamp: P: 1790344569998905 usec, L: 0
0925 13:56:31.186666 (+ 530us) write_op.cc:270] Start()
0925 13:56:31.186758 (+ 92us) write_op.cc:276] Timestamp: P: 1790344570074262 usec, L: 0
0925 13:56:31.187335 (+ 577us) write_op.cc:270] Start()
0925 13:56:31.187444 (+ 109us) write_op.cc:276] Timestamp: P: 1790344570119549 usec, L: 0
0925 13:56:31.190351 (+ 2907us) write_op.cc:270] Start()
0925 13:56:31.190452 (+ 101us) write_op.cc:276] Timestamp: P: 1790344570180159 usec, L: 0
0925 13:56:31.192171 (+ 1719us) write_op.cc:270] Start()
0925 13:56:31.192272 (+ 101us) write_op.cc:276] Timestamp: P: 1790344570208710 usec, L: 0
0925 13:56:31.192800 (+ 528us) write_op.cc:270] Start()
0925 13:56:31.192880 (+ 80us) write_op.cc:276] Timestamp: P: 1790344570237382 usec, L: 0
0925 13:56:31.193353 (+ 473us) write_op.cc:270] Start()
0925 13:56:31.193438 (+ 85us) write_op.cc:276] Timestamp: P: 1790344570278193 usec, L: 0
0925 13:56:31.193940 (+ 502us) write_op.cc:270] Start()
0925 13:56:31.194029 (+ 89us) write_op.cc:276] Timestamp: P: 1790344570344857 usec, L: 0
0925 13:56:31.194477 (+ 448us) write_op.cc:270] Start()
0925 13:56:31.194556 (+ 79us) write_op.cc:276] Timestamp: P: 1790344570496170 usec, L: 0
0925 13:56:31.195957 (+ 1401us) write_op.cc:270] Start()
0925 13:56:31.196032 (+ 75us) write_op.cc:276] Timestamp: P: 1790344570530935 usec, L: 0
0925 13:56:31.196472 (+ 440us) write_op.cc:270] Start()
0925 13:56:31.196565 (+ 93us) write_op.cc:276] Timestamp: P: 1790344570554941 usec, L: 0
0925 13:56:31.197098 (+ 533us) write_op.cc:270] Start()
0925 13:56:31.197173 (+ 75us) write_op.cc:276] Timestamp: P: 1790344570585392 usec, L: 0
0925 13:56:31.198536 (+ 1363us) write_op.cc:270] Start()
0925 13:56:31.198614 (+ 78us) write_op.cc:276] Timestamp: P: 1790344570604051 usec, L: 0
0925 13:56:31.199069 (+ 455us) write_op.cc:270] Start()
0925 13:56:31.199142 (+ 73us) write_op.cc:276] Timestamp: P: 1790344570609535 usec, L: 0
0925 13:56:31.199621 (+ 479us) write_op.cc:270] Start()
0925 13:56:31.199707 (+ 86us) write_op.cc:276] Timestamp: P: 1790344570615632 usec, L: 0
0925 13:56:31.200314 (+ 607us) write_op.cc:270] Start()
0925 13:56:31.200418 (+ 104us) write_op.cc:276] Timestamp: P: 1790344570622410 usec, L: 0
0925 13:56:31.200900 (+ 482us) write_op.cc:270] Start()
0925 13:56:31.200978 (+ 78us) write_op.cc:276] Timestamp: P: 1790344570734460 usec, L: 0
0925 13:56:31.201490 (+ 512us) write_op.cc:270] Start()
0925 13:56:31.201598 (+ 108us) write_op.cc:276] Timestamp: P: 1790344570764499 usec, L: 0
0925 13:56:31.202175 (+ 577us) write_op.cc:270] Start()
0925 13:56:31.202270 (+ 95us) write_op.cc:276] Timestamp: P: 1790344570774968 usec, L: 0
0925 13:56:31.202908 (+ 638us) write_op.cc:270] Start()
0925 13:56:31.203039 (+ 131us) write_op.cc:276] Timestamp: P: 1790344570830257 usec, L: 0
0925 13:56:31.203633 (+ 594us) write_op.cc:270] Start()
0925 13:56:31.203739 (+ 106us) write_op.cc:276] Timestamp: P: 1790344570910820 usec, L: 0
0925 13:56:31.204238 (+ 499us) write_op.cc:270] Start()
0925 13:56:31.204336 (+ 98us) write_op.cc:276] Timestamp: P: 1790344570982300 usec, L: 0
0925 13:56:31.204818 (+ 482us) write_op.cc:270] Start()
0925 13:56:31.205013 (+ 195us) write_op.cc:276] Timestamp: P: 1790344571067960 usec, L: 0
0925 13:56:31.205558 (+ 545us) write_op.cc:270] Start()
0925 13:56:31.205650 (+ 92us) write_op.cc:276] Timestamp: P: 1790344571129385 usec, L: 0
0925 13:56:31.206140 (+ 490us) write_op.cc:270] Start()
0925 13:56:31.206215 (+ 75us) write_op.cc:276] Timestamp: P: 1790344571156089 usec, L: 0
0925 13:56:31.206679 (+ 464us) write_op.cc:270] Start()
0925 13:56:31.206754 (+ 75us) write_op.cc:276] Timestamp: P: 1790344571157910 usec, L: 0
0925 13:56:31.207266 (+ 512us) write_op.cc:270] Start()
0925 13:56:31.207386 (+ 120us) write_op.cc:276] Timestamp: P: 1790344571172684 usec, L: 0
0925 13:56:31.207827 (+ 441us) write_op.cc:270] Start()
0925 13:56:31.207900 (+ 73us) write_op.cc:276] Timestamp: P: 1790344571178115 usec, L: 0
0925 13:56:31.208332 (+ 432us) write_op.cc:270] Start()
0925 13:56:31.208405 (+ 73us) write_op.cc:276] Timestamp: P: 1790344571190131 usec, L: 0
0925 13:56:31.208830 (+ 425us) write_op.cc:270] Start()
0925 13:56:31.208899 (+ 69us) write_op.cc:276] Timestamp: P: 1790344571193855 usec, L: 0
0925 13:56:31.209353 (+ 454us) write_op.cc:270] Start()
0925 13:56:31.209425 (+ 72us) write_op.cc:276] Timestamp: P: 1790344571209106 usec, L: 0
0925 13:56:31.209832 (+ 407us) write_op.cc:270] Start()
0925 13:56:31.209970 (+ 138us) write_op.cc:276] Timestamp: P: 1790344571289343 usec, L: 0
0925 13:56:31.210399 (+ 429us) write_op.cc:270] Start()
0925 13:56:31.210475 (+ 76us) write_op.cc:276] Timestamp: P: 1790344571324507 usec, L: 0
0925 13:56:31.210898 (+ 423us) write_op.cc:270] Start()
0925 13:56:31.210974 (+ 76us) write_op.cc:276] Timestamp: P: 1790344571414472 usec, L: 0
0925 13:56:31.211427 (+ 453us) write_op.cc:270] Start()
0925 13:56:31.211525 (+ 98us) write_op.cc:276] Timestamp: P: 1790344571416997 usec, L: 0
0925 13:56:31.212138 (+ 613us) write_op.cc:270] Start()
0925 13:56:31.212224 (+ 86us) write_op.cc:276] Timestamp: P: 1790344571464777 usec, L: 0
0925 13:56:31.212909 (+ 685us) write_op.cc:270] Start()
0925 13:56:31.213002 (+ 93us) write_op.cc:276] Timestamp: P: 1790344571476004 usec, L: 0
0925 13:56:31.213496 (+ 494us) write_op.cc:270] Start()
0925 13:56:31.213594 (+ 98us) write_op.cc:276] Timestamp: P: 1790344571510084 usec, L: 0
0925 13:56:31.214149 (+ 555us) write_op.cc:270] Start()
0925 13:56:31.214238 (+ 89us) write_op.cc:276] Timestamp: P: 1790344571515478 usec, L: 0
0925 13:56:31.214763 (+ 525us) write_op.cc:270] Start()
0925 13:56:31.214960 (+ 197us) write_op.cc:276] Timestamp: P: 1790344571579246 usec, L: 0
0925 13:56:31.215493 (+ 533us) write_op.cc:270] Start()
0925 13:56:31.215597 (+ 104us) write_op.cc:276] Timestamp: P: 1790344571594743 usec, L: 0
0925 13:56:31.216176 (+ 579us) write_op.cc:270] Start()
0925 13:56:31.216286 (+ 110us) write_op.cc:276] Timestamp: P: 1790344571643839 usec, L: 0
0925 13:56:31.216851 (+ 565us) write_op.cc:270] Start()
0925 13:56:31.216948 (+ 97us) write_op.cc:276] Timestamp: P: 1790344571657936 usec, L: 0
0925 13:56:31.217473 (+ 525us) write_op.cc:270] Start()
0925 13:56:31.217541 (+ 68us) write_op.cc:276] Timestamp: P: 1790344571789119 usec, L: 0
0925 13:56:31.217942 (+ 401us) write_op.cc:270] Start()
0925 13:56:31.218008 (+ 66us) write_op.cc:276] Timestamp: P: 1790344571810255 usec, L: 0
0925 13:56:31.218417 (+ 409us) write_op.cc:270] Start()
0925 13:56:31.218498 (+ 81us) write_op.cc:276] Timestamp: P: 1790344571815897 usec, L: 0
0925 13:56:31.218907 (+ 409us) write_op.cc:270] Start()
0925 13:56:31.218978 (+ 71us) write_op.cc:276] Timestamp: P: 1790344571906420 usec, L: 0
0925 13:56:31.219393 (+ 415us) write_op.cc:270] Start()
0925 13:56:31.219467 (+ 74us) write_op.cc:276] Timestamp: P: 1790344571936691 usec, L: 0
0925 13:56:31.219869 (+ 402us) write_op.cc:270] Start()
0925 13:56:31.219948 (+ 79us) write_op.cc:276] Timestamp: P: 1790344571938604 usec, L: 0
0925 13:56:31.220331 (+ 383us) write_op.cc:270] Start()
0925 13:56:31.220404 (+ 73us) write_op.cc:276] Timestamp: P: 1790344572198246 usec, L: 0
0925 13:56:31.220831 (+ 427us) write_op.cc:270] Start()
0925 13:56:31.220909 (+ 78us) write_op.cc:276] Timestamp: P: 1790344572279456 usec, L: 0
0925 13:56:31.221342 (+ 433us) write_op.cc:270] Start()
0925 13:56:31.221414 (+ 72us) write_op.cc:276] Timestamp: P: 1790344572419439 usec, L: 0
0925 13:56:31.222015 (+ 601us) write_op.cc:270] Start()
0925 13:56:31.222131 (+ 116us) write_op.cc:276] Timestamp: P: 1790344572540486 usec, L: 0
0925 13:56:31.222736 (+ 605us) write_op.cc:270] Start()
0925 13:56:31.222878 (+ 142us) write_op.cc:276] Timestamp: P: 1790344572656184 usec, L: 0
0925 13:56:31.223944 (+ 1066us) write_op.cc:270] Start()
0925 13:56:31.224068 (+ 124us) write_op.cc:276] Timestamp: P: 1790344572657832 usec, L: 0
0925 13:56:31.224662 (+ 594us) write_op.cc:270] Start()
0925 13:56:31.224729 (+ 67us) write_op.cc:276] Timestamp: P: 1790344572661128 usec, L: 0
0925 13:56:31.225276 (+ 547us) write_op.cc:270] Start()
0925 13:56:31.225369 (+ 93us) write_op.cc:276] Timestamp: P: 1790344572666569 usec, L: 0
0925 13:56:31.225926 (+ 557us) write_op.cc:270] Start()
0925 13:56:31.226050 (+ 124us) write_op.cc:276] Timestamp: P: 1790344572668556 usec, L: 0
0925 13:56:31.226565 (+ 515us) write_op.cc:270] Start()
0925 13:56:31.226638 (+ 73us) write_op.cc:276] Timestamp: P: 1790344572670756 usec, L: 0
0925 13:56:31.227060 (+ 422us) write_op.cc:270] Start()
0925 13:56:31.227131 (+ 71us) write_op.cc:276] Timestamp: P: 1790344572672381 usec, L: 0
0925 13:56:31.227569 (+ 438us) write_op.cc:270] Start()
0925 13:56:31.227642 (+ 73us) write_op.cc:276] Timestamp: P: 1790344572674053 usec, L: 0
0925 13:56:31.228097 (+ 455us) write_op.cc:270] Start()
0925 13:56:31.228185 (+ 88us) write_op.cc:276] Timestamp: P: 1790344572836904 usec, L: 0
0925 13:56:31.228772 (+ 587us) write_op.cc:270] Start()
0925 13:56:31.229631 (+ 859us) write_op.cc:276] Timestamp: P: 1790344572879648 usec, L: 0
0925 13:56:31.230094 (+ 463us) write_op.cc:270] Start()
0925 13:56:31.230170 (+ 76us) write_op.cc:276] Timestamp: P: 1790344572908957 usec, L: 0
0925 13:56:31.230591 (+ 421us) write_op.cc:270] Start()
0925 13:56:31.230684 (+ 93us) write_op.cc:276] Timestamp: P: 1790344572960592 usec, L: 0
0925 13:56:31.231088 (+ 404us) write_op.cc:270] Start()
0925 13:56:31.231162 (+ 74us) write_op.cc:276] Timestamp: P: 1790344572970356 usec, L: 0
0925 13:56:31.231792 (+ 630us) write_op.cc:270] Start()
0925 13:56:31.231890 (+ 98us) write_op.cc:276] Timestamp: P: 1790344573020716 usec, L: 0
0925 13:56:31.232504 (+ 614us) write_op.cc:270] Start()
0925 13:56:31.232579 (+ 75us) write_op.cc:276] Timestamp: P: 1790344573060804 usec, L: 0
0925 13:56:31.233039 (+ 460us) write_op.cc:270] Start()
0925 13:56:31.233119 (+ 80us) write_op.cc:276] Timestamp: P: 1790344573132410 usec, L: 0
0925 13:56:31.233647 (+ 528us) write_op.cc:270] Start()
0925 13:56:31.233963 (+ 316us) write_op.cc:276] Timestamp: P: 1790344573161336 usec, L: 0
0925 13:56:31.234730 (+ 767us) write_op.cc:270] Start()
0925 13:56:31.234830 (+ 100us) write_op.cc:276] Timestamp: P: 1790344573206711 usec, L: 0
0925 13:56:31.235407 (+ 577us) write_op.cc:270] Start()
0925 13:56:31.235507 (+ 100us) write_op.cc:276] Timestamp: P: 1790344573316118 usec, L: 0
0925 13:56:31.236080 (+ 573us) write_op.cc:270] Start()
0925 13:56:31.236186 (+ 106us) write_op.cc:276] Timestamp: P: 1790344573317998 usec, L: 0
0925 13:56:31.236756 (+ 570us) write_op.cc:270] Start()
0925 13:56:31.236854 (+ 98us) write_op.cc:276] Timestamp: P: 1790344573369315 usec, L: 0
0925 13:56:31.237414 (+ 560us) write_op.cc:270] Start()
0925 13:56:31.237501 (+ 87us) write_op.cc:276] Timestamp: P: 1790344573386672 usec, L: 0
0925 13:56:31.238084 (+ 583us) write_op.cc:270] Start()
0925 13:56:31.238158 (+ 74us) write_op.cc:276] Timestamp: P: 1790344573412537 usec, L: 0
0925 13:56:31.238563 (+ 405us) write_op.cc:270] Start()
0925 13:56:31.238634 (+ 71us) write_op.cc:276] Timestamp: P: 1790344573481448 usec, L: 0
0925 13:56:31.239073 (+ 439us) write_op.cc:270] Start()
0925 13:56:31.239142 (+ 69us) write_op.cc:276] Timestamp: P: 1790344573525460 usec, L: 0
0925 13:56:31.239586 (+ 444us) write_op.cc:270] Start()
0925 13:56:31.239667 (+ 81us) write_op.cc:276] Timestamp: P: 1790344573553407 usec, L: 0
0925 13:56:31.240091 (+ 424us) write_op.cc:270] Start()
0925 13:56:31.240354 (+ 263us) write_op.cc:276] Timestamp: P: 1790344573583512 usec, L: 0
0925 13:56:31.240963 (+ 609us) write_op.cc:270] Start()
0925 13:56:31.241076 (+ 113us) write_op.cc:276] Timestamp: P: 1790344573604665 usec, L: 0
0925 13:56:31.241621 (+ 545us) write_op.cc:270] Start()
0925 13:56:31.241737 (+ 116us) write_op.cc:276] Timestamp: P: 1790344573626141 usec, L: 0
0925 13:56:31.242268 (+ 531us) write_op.cc:270] Start()
0925 13:56:31.242381 (+ 113us) write_op.cc:276] Timestamp: P: 1790344573676909 usec, L: 0
0925 13:56:31.242899 (+ 518us) write_op.cc:270] Start()
0925 13:56:31.242974 (+ 75us) write_op.cc:276] Timestamp: P: 1790344573708992 usec, L: 0
0925 13:56:31.243578 (+ 604us) write_op.cc:270] Start()
0925 13:56:31.243661 (+ 83us) write_op.cc:276] Timestamp: P: 1790344573739255 usec, L: 0
0925 13:56:31.244102 (+ 441us) write_op.cc:270] Start()
0925 13:56:31.244189 (+ 87us) write_op.cc:276] Timestamp: P: 1790344573761385 usec, L: 0
0925 13:56:31.244946 (+ 757us) write_op.cc:270] Start()
0925 13:56:31.245059 (+ 113us) write_op.cc:276] Timestamp: P: 1790344573787880 usec, L: 0
0925 13:56:31.245785 (+ 726us) write_op.cc:270] Start()
0925 13:56:31.245865 (+ 80us) write_op.cc:276] Timestamp: P: 1790344573817669 usec, L: 0
0925 13:56:31.246313 (+ 448us) write_op.cc:270] Start()
0925 13:56:31.246387 (+ 74us) write_op.cc:276] Timestamp: P: 1790344573828284 usec, L: 0
0925 13:56:31.246801 (+ 414us) write_op.cc:270] Start()
0925 13:56:31.246867 (+ 66us) write_op.cc:276] Timestamp: P: 1790344573903096 usec, L: 0
0925 13:56:31.247303 (+ 436us) write_op.cc:270] Start()
0925 13:56:31.247413 (+ 110us) write_op.cc:276] Timestamp: P: 1790344573981415 usec, L: 0
0925 13:56:31.247940 (+ 527us) write_op.cc:270] Start()
0925 13:56:31.248026 (+ 86us) write_op.cc:276] Timestamp: P: 1790344573986168 usec, L: 0
0925 13:56:31.248444 (+ 418us) write_op.cc:270] Start()
0925 13:56:31.248501 (+ 57us) write_op.cc:276] Timestamp: P: 1790344573989291 usec, L: 0
0925 13:56:31.248874 (+ 373us) write_op.cc:270] Start()
0925 13:56:31.248932 (+ 58us) write_op.cc:276] Timestamp: P: 1790344574059008 usec, L: 0
0925 13:56:31.249320 (+ 388us) write_op.cc:270] Start()
0925 13:56:31.249405 (+ 85us) write_op.cc:276] Timestamp: P: 1790344574066936 usec, L: 0
0925 13:56:31.249926 (+ 521us) write_op.cc:270] Start()
0925 13:56:31.249984 (+ 58us) write_op.cc:276] Timestamp: P: 1790344574105618 usec, L: 0
0925 13:56:31.250350 (+ 366us) write_op.cc:270] Start()
0925 13:56:31.250417 (+ 67us) write_op.cc:276] Timestamp: P: 1790344574127480 usec, L: 0
0925 13:56:31.250799 (+ 382us) write_op.cc:270] Start()
0925 13:56:31.250862 (+ 63us) write_op.cc:276] Timestamp: P: 1790344574171216 usec, L: 0
0925 13:56:31.251254 (+ 392us) write_op.cc:270] Start()
0925 13:56:31.251316 (+ 62us) write_op.cc:276] Timestamp: P: 1790344574269834 usec, L: 0
0925 13:56:31.251755 (+ 439us) write_op.cc:270] Start()
0925 13:56:31.251818 (+ 63us) write_op.cc:276] Timestamp: P: 1790344574349105 usec, L: 0
0925 13:56:31.252382 (+ 564us) write_op.cc:270] Start()
0925 13:56:31.258190 (+ 5808us) write_op.cc:276] Timestamp: P: 1790344574399764 usec, L: 0
0925 13:56:31.259151 (+ 961us) write_op.cc:270] Start()
0925 13:56:31.259233 (+ 82us) write_op.cc:276] Timestamp: P: 1790344574404640 usec, L: 0
0925 13:56:31.273029 (+ 13796us) write_op.cc:270] Start()
0925 13:56:31.273167 (+ 138us) write_op.cc:276] Timestamp: P: 1790344574410596 usec, L: 0
0925 13:56:31.273598 (+ 431us) write_op.cc:270] Start()
0925 13:56:31.273672 (+ 74us) write_op.cc:276] Timestamp: P: 1790344574436608 usec, L: 0
0925 13:56:31.274041 (+ 369us) write_op.cc:270] Start()
0925 13:56:31.274106 (+ 65us) write_op.cc:276] Timestamp: P: 1790344574469661 usec, L: 0
0925 13:56:31.274478 (+ 372us) write_op.cc:270] Start()
0925 13:56:31.274545 (+ 67us) write_op.cc:276] Timestamp: P: 1790344574515435 usec, L: 0
0925 13:56:31.274986 (+ 441us) write_op.cc:270] Start()
0925 13:56:31.275048 (+ 62us) write_op.cc:276] Timestamp: P: 1790344574572252 usec, L: 0
0925 13:56:31.275463 (+ 415us) write_op.cc:270] Start()
0925 13:56:31.275528 (+ 65us) write_op.cc:276] Timestamp: P: 1790344574591677 usec, L: 0
0925 13:56:31.275921 (+ 393us) write_op.cc:270] Start()
0925 13:56:31.276007 (+ 86us) write_op.cc:276] Timestamp: P: 1790344574677485 usec, L: 0
0925 13:56:31.276463 (+ 456us) write_op.cc:270] Start()
0925 13:56:31.276533 (+ 70us) write_op.cc:276] Timestamp: P: 1790344574683341 usec, L: 0
0925 13:56:31.276933 (+ 400us) write_op.cc:270] Start()
0925 13:56:31.276994 (+ 61us) write_op.cc:276] Timestamp: P: 1790344574751823 usec, L: 0
0925 13:56:31.277359 (+ 365us) write_op.cc:270] Start()
0925 13:56:31.277430 (+ 71us) write_op.cc:276] Timestamp: P: 1790344574818942 usec, L: 0
0925 13:56:31.277819 (+ 389us) write_op.cc:270] Start()
0925 13:56:31.277893 (+ 74us) write_op.cc:276] Timestamp: P: 1790344574872648 usec, L: 0
0925 13:56:31.278228 (+ 335us) write_op.cc:270] Start()
0925 13:56:31.278291 (+ 63us) write_op.cc:276] Timestamp: P: 1790344574994677 usec, L: 0
0925 13:56:31.278659 (+ 368us) write_op.cc:270] Start()
0925 13:56:31.278724 (+ 65us) write_op.cc:276] Timestamp: P: 1790344575036325 usec, L: 0
0925 13:56:31.279054 (+ 330us) write_op.cc:270] Start()
0925 13:56:31.279109 (+ 55us) write_op.cc:276] Timestamp: P: 1790344575092141 usec, L: 0
0925 13:56:31.313134 (+ 34025us) write_op.cc:270] Start()
0925 13:56:31.313224 (+ 90us) write_op.cc:276] Timestamp: P: 1790344575189616 usec, L: 0
0925 13:56:31.313741 (+ 517us) write_op.cc:270] Start()
0925 13:56:31.313808 (+ 67us) write_op.cc:276] Timestamp: P: 1790344575290213 usec, L: 0
0925 13:56:31.314238 (+ 430us) write_op.cc:270] Start()
0925 13:56:31.314307 (+ 69us) write_op.cc:276] Timestamp: P: 1790344575415997 usec, L: 0
0925 13:56:31.314685 (+ 378us) write_op.cc:270] Start()
0925 13:56:31.314780 (+ 95us) write_op.cc:276] Timestamp: P: 1790344575419928 usec, L: 0
0925 13:56:31.317408 (+ 2628us) write_op.cc:270] Start()
0925 13:56:31.317487 (+ 79us) write_op.cc:276] Timestamp: P: 1790344575537031 usec, L: 0
0925 13:56:31.317904 (+ 417us) write_op.cc:270] Start()
0925 13:56:31.317976 (+ 72us) write_op.cc:276] Timestamp: P: 1790344575922286 usec, L: 0
0925 13:56:31.318332 (+ 356us) write_op.cc:270] Start()
0925 13:56:31.318389 (+ 57us) write_op.cc:276] Timestamp: P: 1790344575943040 usec, L: 0
0925 13:56:31.330412 (+ 12023us) write_op.cc:270] Start()
0925 13:56:31.330487 (+ 75us) write_op.cc:276] Timestamp: P: 1790344575977106 usec, L: 0
0925 13:56:31.331000 (+ 513us) write_op.cc:270] Start()
0925 13:56:31.331110 (+ 110us) write_op.cc:276] Timestamp: P: 1790344575988791 usec, L: 0
0925 13:56:31.331879 (+ 769us) write_op.cc:270] Start()
0925 13:56:31.331942 (+ 63us) write_op.cc:276] Timestamp: P: 1790344576033747 usec, L: 0
0925 13:56:31.332342 (+ 400us) write_op.cc:270] Start()
0925 13:56:31.332409 (+ 67us) write_op.cc:276] Timestamp: P: 1790344576059268 usec, L: 0
0925 13:56:31.332764 (+ 355us) write_op.cc:270] Start()
0925 13:56:31.332829 (+ 65us) write_op.cc:276] Timestamp: P: 1790344576110613 usec, L: 0
0925 13:56:31.333228 (+ 399us) write_op.cc:270] Start()
0925 13:56:31.333289 (+ 61us) write_op.cc:276] Timestamp: P: 1790344576129048 usec, L: 0
0925 13:56:31.333649 (+ 360us) write_op.cc:270] Start()
0925 13:56:31.333708 (+ 59us) write_op.cc:276] Timestamp: P: 1790344576190361 usec, L: 0
0925 13:56:31.334090 (+ 382us) write_op.cc:270] Start()
0925 13:56:31.334150 (+ 60us) write_op.cc:276] Timestamp: P: 1790344576212284 usec, L: 0
0925 13:56:31.334505 (+ 355us) write_op.cc:270] Start()
0925 13:56:31.334560 (+ 55us) write_op.cc:276] Timestamp: P: 1790344576289216 usec, L: 0
0925 13:56:31.334900 (+ 340us) write_op.cc:270] Start()
0925 13:56:31.334956 (+ 56us) write_op.cc:276] Timestamp: P: 1790344576310345 usec, L: 0
0925 13:56:31.335297 (+ 341us) write_op.cc:270] Start()
0925 13:56:31.340779 (+ 5482us) write_op.cc:276] Timestamp: P: 1790344576362523 usec, L: 0
0925 13:56:31.341219 (+ 440us) write_op.cc:270] Start()
0925 13:56:31.341283 (+ 64us) write_op.cc:276] Timestamp: P: 1790344576474564 usec, L: 0
0925 13:56:31.341660 (+ 377us) write_op.cc:270] Start()
0925 13:56:31.341740 (+ 80us) write_op.cc:276] Timestamp: P: 1790344576498798 usec, L: 0
0925 13:56:31.342135 (+ 395us) write_op.cc:270] Start()
0925 13:56:31.342198 (+ 63us) write_op.cc:276] Timestamp: P: 1790344576545674 usec, L: 0
0925 13:56:31.342555 (+ 357us) write_op.cc:270] Start()
0925 13:56:31.342615 (+ 60us) write_op.cc:276] Timestamp: P: 1790344576572247 usec, L: 0
0925 13:56:31.343105 (+ 490us) write_op.cc:270] Start()
0925 13:56:31.343171 (+ 66us) write_op.cc:276] Timestamp: P: 1790344576585130 usec, L: 0
0925 13:56:31.344874 (+ 1703us) write_op.cc:270] Start()
0925 13:56:31.344938 (+ 64us) write_op.cc:276] Timestamp: P: 1790344576620946 usec, L: 0
0925 13:56:31.345363 (+ 425us) write_op.cc:270] Start()
0925 13:56:31.345425 (+ 62us) write_op.cc:276] Timestamp: P: 1790344576667504 usec, L: 0
0925 13:56:31.346294 (+ 869us) write_op.cc:270] Start()
0925 13:56:31.346359 (+ 65us) write_op.cc:276] Timestamp: P: 1790344576737287 usec, L: 0
0925 13:56:31.349791 (+ 3432us) write_op.cc:270] Start()
0925 13:56:31.349864 (+ 73us) write_op.cc:276] Timestamp: P: 1790344576787212 usec, L: 0
0925 13:56:31.350305 (+ 441us) write_op.cc:270] Start()
0925 13:56:31.350376 (+ 71us) write_op.cc:276] Timestamp: P: 1790344576798034 usec, L: 0
0925 13:56:31.350763 (+ 387us) write_op.cc:270] Start()
0925 13:56:31.350826 (+ 63us) write_op.cc:276] Timestamp: P: 1790344576840667 usec, L: 0
0925 13:56:31.351214 (+ 388us) write_op.cc:270] Start()
0925 13:56:31.351286 (+ 72us) write_op.cc:276] Timestamp: P: 1790344576843228 usec, L: 0
0925 13:56:31.351747 (+ 461us) write_op.cc:270] Start()
0925 13:56:31.351815 (+ 68us) write_op.cc:276] Timestamp: P: 1790344576857592 usec, L: 0
0925 13:56:31.352228 (+ 413us) write_op.cc:270] Start()
0925 13:56:31.352295 (+ 67us) write_op.cc:276] Timestamp: P: 1790344576860260 usec, L: 0
0925 13:56:31.352705 (+ 410us) write_op.cc:270] Start()
0925 13:56:31.352772 (+ 67us) write_op.cc:276] Timestamp: P: 1790344576901937 usec, L: 0
0925 13:56:31.353168 (+ 396us) write_op.cc:270] Start()
0925 13:56:31.353233 (+ 65us) write_op.cc:276] Timestamp: P: 1790344577201099 usec, L: 0
0925 13:56:31.353617 (+ 384us) write_op.cc:270] Start()
0925 13:56:31.353687 (+ 70us) write_op.cc:276] Timestamp: P: 1790344577208071 usec, L: 0
0925 13:56:31.364239 (+ 10552us) write_op.cc:270] Start()
0925 13:56:31.364351 (+ 112us) write_op.cc:276] Timestamp: P: 1790344577216709 usec, L: 0
0925 13:56:31.364919 (+ 568us) write_op.cc:270] Start()
0925 13:56:31.365008 (+ 89us) write_op.cc:276] Timestamp: P: 1790344577221758 usec, L: 0
0925 13:56:31.365424 (+ 416us) write_op.cc:270] Start()
0925 13:56:31.365487 (+ 63us) write_op.cc:276] Timestamp: P: 1790344577223356 usec, L: 0
0925 13:56:31.365840 (+ 353us) write_op.cc:270] Start()
0925 13:56:31.365900 (+ 60us) write_op.cc:276] Timestamp: P: 1790344577227034 usec, L: 0
0925 13:56:31.366268 (+ 368us) write_op.cc:270] Start()
0925 13:56:31.366329 (+ 61us) write_op.cc:276] Timestamp: P: 1790344577512673 usec, L: 0
0925 13:56:31.366692 (+ 363us) write_op.cc:270] Start()
0925 13:56:31.366765 (+ 73us) write_op.cc:276] Timestamp: P: 1790344577514453 usec, L: 0
0925 13:56:31.367133 (+ 368us) write_op.cc:270] Start()
0925 13:56:31.367202 (+ 69us) write_op.cc:276] Timestamp: P: 1790344577642025 usec, L: 0
0925 13:56:31.367601 (+ 399us) write_op.cc:270] Start()
0925 13:56:31.367662 (+ 61us) write_op.cc:276] Timestamp: P: 1790344577691631 usec, L: 0
0925 13:56:31.368012 (+ 350us) write_op.cc:270] Start()
0925 13:56:31.368069 (+ 57us) write_op.cc:276] Timestamp: P: 1790344577750131 usec, L: 0
0925 13:56:31.368426 (+ 357us) write_op.cc:270] Start()
0925 13:56:31.368484 (+ 58us) write_op.cc:276] Timestamp: P: 1790344577758360 usec, L: 0
0925 13:56:31.368819 (+ 335us) write_op.cc:270] Start()
0925 13:56:31.368881 (+ 62us) write_op.cc:276] Timestamp: P: 1790344577787129 usec, L: 0
0925 13:56:31.369261 (+ 380us) write_op.cc:270] Start()
0925 13:56:31.369321 (+ 60us) write_op.cc:276] Timestamp: P: 1790344577809039 usec, L: 0
0925 13:56:31.369709 (+ 388us) write_op.cc:270] Start()
0925 13:56:31.372585 (+ 2876us) write_op.cc:276] Timestamp: P: 1790344577911826 usec, L: 0
0925 13:56:31.373190 (+ 605us) write_op.cc:270] Start()
0925 13:56:31.373282 (+ 92us) write_op.cc:276] Timestamp: P: 1790344577913792 usec, L: 0
0925 13:56:31.373767 (+ 485us) write_op.cc:270] Start()
0925 13:56:31.373851 (+ 84us) write_op.cc:276] Timestamp: P: 1790344578031050 usec, L: 0
0925 13:56:31.374367 (+ 516us) write_op.cc:270] Start()
0925 13:56:31.374451 (+ 84us) write_op.cc:276] Timestamp: P: 1790344578036709 usec, L: 0
0925 13:56:31.374934 (+ 483us) write_op.cc:270] Start()
0925 13:56:31.375027 (+ 93us) write_op.cc:276] Timestamp: P: 1790344578081489 usec, L: 0
0925 13:56:31.375576 (+ 549us) write_op.cc:270] Start()
0925 13:56:31.375658 (+ 82us) write_op.cc:276] Timestamp: P: 17903445
I20260925 13:56:32.654877 20959 raft_consensus.cc:493] T b23948807b2c4df9bf9a4da5f767e3b5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:32.655339 20959 raft_consensus.cc:515] T b23948807b2c4df9bf9a4da5f767e3b5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:32.657387 20959 leader_election.cc:290] T b23948807b2c4df9bf9a4da5f767e3b5 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:32.657787 20959 raft_consensus.cc:493] T b3172d99afee4a4c878e71b645037917 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:32.658141 20959 raft_consensus.cc:515] T b3172d99afee4a4c878e71b645037917 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:32.663686 19271 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fa8330217e794edf9e26a368d8b34685 consensus state has no leader
I20260925 13:56:32.665202 19519 tablet_service.cc:1558] Processing DeleteTablet for tablet e2e44b2ca25c4bbcb91288424b3e18b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:24 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:32.665939 19942 tablet_bootstrap.cc:654] T b3172d99afee4a4c878e71b645037917 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:32.666062 19523 tablet_service.cc:1558] Processing DeleteTablet for tablet f5a0bd0e205f4889be93739dfbd2a816 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:26 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:32.667150 20891 raft_consensus.cc:493] T c5a9df833271480aa49feb7d9f925459 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:32.671247 20891 raft_consensus.cc:515] T c5a9df833271480aa49feb7d9f925459 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:32.692200 19942 tablet_bootstrap.cc:492] T b3172d99afee4a4c878e71b645037917 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:32.692641 19942 ts_tablet_manager.cc:1403] T b3172d99afee4a4c878e71b645037917 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.085s user 0.009s sys 0.000s
I20260925 13:56:32.696624 20856 raft_consensus.cc:493] T 78d8c01c88d24deeae21d647e7f205ba P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:32.697079 20856 raft_consensus.cc:515] T 78d8c01c88d24deeae21d647e7f205ba P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:32.699079 20856 leader_election.cc:290] T 78d8c01c88d24deeae21d647e7f205ba P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:32.701545 20831 raft_consensus.cc:493] T b3172d99afee4a4c878e71b645037917 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:32.701975 20831 raft_consensus.cc:515] T b3172d99afee4a4c878e71b645037917 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:32.703791 20831 leader_election.cc:290] T b3172d99afee4a4c878e71b645037917 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:32.712550 19939 tablet_bootstrap.cc:492] T 1ce5d15e67644247b95960f444acc621 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:32.713126 19939 ts_tablet_manager.cc:1403] T 1ce5d15e67644247b95960f444acc621 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.284s user 0.009s sys 0.007s
I20260925 13:56:32.695128 19942 raft_consensus.cc:359] T b3172d99afee4a4c878e71b645037917 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:32.720328 19942 raft_consensus.cc:385] T b3172d99afee4a4c878e71b645037917 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:32.721338 19939 raft_consensus.cc:359] T 1ce5d15e67644247b95960f444acc621 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:32.722227 19942 raft_consensus.cc:740] T b3172d99afee4a4c878e71b645037917 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:32.723764 19939 raft_consensus.cc:385] T 1ce5d15e67644247b95960f444acc621 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:32.723585 19942 consensus_queue.cc:260] T b3172d99afee4a4c878e71b645037917 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:32.725158 19939 raft_consensus.cc:740] T 1ce5d15e67644247b95960f444acc621 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:32.725419 19531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76faddb4346a4690bf22235c71a04b08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:32.726645 19530 tablet_service.cc:1558] Processing DeleteTablet for tablet 18a6d1c4eb7948149e709a380da50d93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:25 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:32.728163 19523 tablet_service.cc:1558] Processing DeleteTablet for tablet 908df914cfec490cb7058dbe7b501c97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:22 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:32.730446 19939 consensus_queue.cc:260] T 1ce5d15e67644247b95960f444acc621 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:32.732021 19939 ts_tablet_manager.cc:1434] T 1ce5d15e67644247b95960f444acc621 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.019s user 0.006s sys 0.000s
I20260925 13:56:32.734434 19942 ts_tablet_manager.cc:1434] T b3172d99afee4a4c878e71b645037917 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.042s user 0.006s sys 0.000s
I20260925 13:56:32.735608 19524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1c96465b34540ee865688f13a282699. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:32.735553 20963 tablet.cc:1751] T e3cf9676160c4af08f1f62ae952b4c57 P f790493ba7ad403993b678f2df2b55be: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:32.738652 19533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1c7ae13fb3e407e8c718b6c6aef4b9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:32.741101 20927 raft_consensus.cc:493] T b3172d99afee4a4c878e71b645037917 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:32.741794 20927 raft_consensus.cc:515] T b3172d99afee4a4c878e71b645037917 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:32.743587 19522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3f0b400bec547afa3d484e54d6817e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:32.744437 20927 leader_election.cc:290] T b3172d99afee4a4c878e71b645037917 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:32.745549 19942 tablet_bootstrap.cc:492] T 7e544c93488541da9ebb588a465b27b7 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:32.747648 19518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b1e2dc2abc249528de6b9ff6fad4275. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:32.747777 19264 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.18.182.194:38641, user_credentials={real_user=slave}} blocked reactor thread for 122761us
W20260925 13:56:32.748238 19276 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.193:37907 (request call id 40) took 1719 ms (client timeout 1000 ms). Trace:
I20260925 13:56:32.760488 19669 tablet_service.cc:1511] Processing CreateTablet for tablet 940fe6cf272e4264aad6e2930f2cf54c (DEFAULT_TABLE table=default.Table_d57c934ba0b84027b1830f751a466939 [id=dd0a159f8fc445a3809eb8ec76acc34a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.763141 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11077cdb735840b5921e22f588059149" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:32.764348 19942 tablet_bootstrap.cc:654] T 7e544c93488541da9ebb588a465b27b7 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:32.773427 20959 leader_election.cc:290] T b3172d99afee4a4c878e71b645037917 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:32.774818 19163 master-stress-test.cc:579] Killing test thread 1/10
I20260925 13:56:32.779855 20891 leader_election.cc:290] T c5a9df833271480aa49feb7d9f925459 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:32.785104 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3172d99afee4a4c878e71b645037917" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:32.786916 19523 tablet_service.cc:1511] Processing CreateTablet for tablet 940fe6cf272e4264aad6e2930f2cf54c (DEFAULT_TABLE table=default.Table_d57c934ba0b84027b1830f751a466939 [id=dd0a159f8fc445a3809eb8ec76acc34a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.787741 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505a24ebe7014f80b3c184329cc74a9a" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:32.790331 19549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11077cdb735840b5921e22f588059149" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:32.791617 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3172d99afee4a4c878e71b645037917" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:32.794967 19554 raft_consensus.cc:2468] T b3172d99afee4a4c878e71b645037917 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 0.
I20260925 13:56:32.799852 19523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 940fe6cf272e4264aad6e2930f2cf54c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:32.748646 19276 rpcz_store.cc:269] 0925 13:56:31.028588 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:31.028712 (+ 124us) service_pool.cc:224] Handling call
0925 13:56:32.748174 (+1719462us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:32.362156 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:32.362531 (+ 375us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:32.362556 (+ 25us) write_op.cc:435] Acquired schema lock
0925 13:56:32.362570 (+ 14us) tablet.cc:662] Decoding operations
0925 13:56:32.362877 (+ 307us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:32.362914 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:32.362934 (+ 20us) tablet.cc:685] Acquiring locks for 2 operations
0925 13:56:32.363176 (+ 242us) tablet.cc:701] Row locks acquired
0925 13:56:32.363189 (+ 13us) write_op.cc:260] PREPARE: finished
0925 13:56:32.363285 (+ 96us) write_op.cc:270] Start()
0925 13:56:32.363386 (+ 101us) write_op.cc:276] Timestamp: P: 1790344592363247 usec, L: 0
0925 13:56:32.363401 (+ 15us) op_driver.cc:348] REPLICATION: starting
0925 13:56:32.460817 (+ 97416us) log.cc:844] Serialized 919 byte log entry
0925 13:56:32.462406 (+ 1589us) spinlock_profiling.cc:243] Waited 96.9 ms on lock 0x7b5400040454. stack: 00007fdf0a30b323 00007fdf0a30b220 00007fdf09eaf574 0000000000516964 00000000005168f9 00007fdf11408aa1 00007fdf1149b5e8 00007fdf1192b741 00007fdf1192af18 00007fdf1192fbb1 00007fdf1192fb69 00007fdf1192faf9 00007fdf1192fac1 00007fdf1192eced 00000000005bd4e4 00000000005bd319
0925 13:56:32.469329 (+ 6923us) op_driver.cc:464] REPLICATION: finished
0925 13:56:32.469807 (+ 478us) write_op.cc:301] APPLY: starting
0925 13:56:32.469870 (+ 63us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:32.469961 (+ 91us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:32.469979 (+ 18us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:32.470242 (+ 263us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:32.470384 (+ 142us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 13:56:32.470441 (+ 57us) write_op.cc:312] APPLY: finished
0925 13:56:32.470605 (+ 164us) log.cc:844] Serialized 29 byte log entry
0925 13:56:32.470727 (+ 122us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:32.470806 (+ 79us) write_op.cc:454] Released schema lock
0925 13:56:32.470835 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0925 13:56:32.483470 (+ 12635us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:32.483493 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":23,"child_traces":[["op",{"apply.queue_time_us":337,"apply.run_cpu_time_us":1211,"apply.run_wall_time_us":1211,"num_ops":2,"prepare.queue_time_us":91239,"prepare.run_cpu_time_us":2642,"prepare.run_wall_time_us":121299,"raft.queue_time_us":95,"raft.run_cpu_time_us":675,"raft.run_wall_time_us":675,"replication_time_us":105846,"spinlock_wait_cycles":222847872}]]}
W20260925 13:56:32.792621 19490 leader_election.cc:343] T 78d8c01c88d24deeae21d647e7f205ba P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 78d8c01c88d24deeae21d647e7f205ba
W20260925 13:56:32.818503 19490 leader_election.cc:343] T 5a9b3dfaa7474611a1824d63eb906b43 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 5a9b3dfaa7474611a1824d63eb906b43
I20260925 13:56:32.818812 19695 raft_consensus.cc:2468] T b3172d99afee4a4c878e71b645037917 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 0.
W20260925 13:56:32.836779 19773 leader_election.cc:343] T 11077cdb735840b5921e22f588059149 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:32.827838 19690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a9df833271480aa49feb7d9f925459" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:32.837636 19690 raft_consensus.cc:2468] T c5a9df833271480aa49feb7d9f925459 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 0.
I20260925 13:56:32.835496 19671 tablet_service.cc:1511] Processing CreateTablet for tablet f49a9c7c11394c14bf8b09f671ea1854 (DEFAULT_TABLE table=default.Table_2d803b07af1d41a789c69141ec7f60f7 [id=39197c367fc848be9008667322367da5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.813305 19668 tablet_service.cc:1511] Processing CreateTablet for tablet 4234e52546b948f3ac2542431f43771a (DEFAULT_TABLE table=default.Table_b54d79ef34824edd919b01a1cdd16f8d [id=4b046318cc3b4bd1885e8c2a122aefdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.850976 19274 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39288:
table { table_name: "default.Table_1363fa0f07a54c5e866650fa74b28e0c" } new_table_name: "default.Table_ddc133aff70b4778a2e87490f469da02" modify_external_catalogs: true
I20260925 13:56:32.852386 19275 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_408aac5f0ca04822a4e768d814b70669"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:32.853806 19275 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_408aac5f0ca04822a4e768d814b70669 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:32.857240 19271 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39288:
name: "default.Table_b1ea6241873b405d80c47c0d7cdb85b7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:56:32.858541 19271 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b1ea6241873b405d80c47c0d7cdb85b7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:56:32.819028 20959 consensus_queue.cc:1048] T 000aad39953c4688b543c7d882a08ec8 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:32.823352 19530 tablet_service.cc:1511] Processing CreateTablet for tablet 4234e52546b948f3ac2542431f43771a (DEFAULT_TABLE table=default.Table_b54d79ef34824edd919b01a1cdd16f8d [id=4b046318cc3b4bd1885e8c2a122aefdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.863943 19530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4234e52546b948f3ac2542431f43771a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:32.838806 20794 raft_consensus.cc:493] T 7e544c93488541da9ebb588a465b27b7 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:32.867571 20794 raft_consensus.cc:515] T 7e544c93488541da9ebb588a465b27b7 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:32.869397 20794 leader_election.cc:290] T 7e544c93488541da9ebb588a465b27b7 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:32.856021 19529 tablet_service.cc:1511] Processing CreateTablet for tablet f49a9c7c11394c14bf8b09f671ea1854 (DEFAULT_TABLE table=default.Table_2d803b07af1d41a789c69141ec7f60f7 [id=39197c367fc848be9008667322367da5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.870325 19529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f49a9c7c11394c14bf8b09f671ea1854. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:32.872162 19549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff33a56361946edb556ac611d2299eb" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:32.873594 19549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f8a96ae6eec4f15a221f9aad537c094" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:32.874215 19549 raft_consensus.cc:2393] T 5f8a96ae6eec4f15a221f9aad537c094 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7c1b52895a14482fb54a6758f5054f8f in current term 1: Already voted for candidate f790493ba7ad403993b678f2df2b55be in this term.
I20260925 13:56:32.882982 19939 tablet_bootstrap.cc:492] T 82c1002d2bcb4b11839e2f4a93b899ad P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:32.886387 19666 tablet_service.cc:1511] Processing CreateTablet for tablet f5791a011b104ec59721b6ed8ce773f1 (DEFAULT_TABLE table=default.Table_86568b68e691429192052565e933af79 [id=3a35752b668747c8bc1485701e8e7282]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 13:56:32.886937 19773 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.194:38641, user_credentials={real_user=slave}} blocked reactor thread for 50196.2us
W20260925 13:56:32.887979 19488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.194:38641, user_credentials={real_user=slave}} blocked reactor thread for 498927us
I20260925 13:56:32.890518 19692 raft_consensus.cc:3060] T e9dbcd85dc8647c68c120ba3fcfe1790 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:32.892647 19689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23948807b2c4df9bf9a4da5f767e3b5" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:32.893508 19694 raft_consensus.cc:3060] T 089dc4dfeb5243788dc03061c8743a81 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:32.897380 19669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 940fe6cf272e4264aad6e2930f2cf54c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:32.900285 19671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f49a9c7c11394c14bf8b09f671ea1854. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:32.906116 20927 raft_consensus.cc:493] T 4d01a791445149e7bee9493e1eec5c6b P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:32.906527 20927 raft_consensus.cc:515] T 4d01a791445149e7bee9493e1eec5c6b P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:32.907930 20644 ts_tablet_manager.cc:1929] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:32.908301 20644 log.cc:1199] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/4d01a791445149e7bee9493e1eec5c6b
I20260925 13:56:32.908318 20927 leader_election.cc:290] T 4d01a791445149e7bee9493e1eec5c6b P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:32.908999 20644 ts_tablet_manager.cc:1950] T 4d01a791445149e7bee9493e1eec5c6b P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:32.913718 19519 tablet_service.cc:1511] Processing CreateTablet for tablet f5791a011b104ec59721b6ed8ce773f1 (DEFAULT_TABLE table=default.Table_86568b68e691429192052565e933af79 [id=3a35752b668747c8bc1485701e8e7282]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.916983 19938 tablet_bootstrap.cc:654] T 37947005aaf14f27a6e5d1f30c67250c P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:32.914498 19939 tablet_bootstrap.cc:654] T 82c1002d2bcb4b11839e2f4a93b899ad P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
W20260925 13:56:32.946533 19267 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet d7c8beb65fd64639b22374fe4d3ef400 (table default.Table_626a2de34ead48f4a50d72ea44cc25f2 [id=f4560686cb314de99329f63b5902971d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet d7c8beb65fd64639b22374fe4d3ef400 consensus state has no leader
I20260925 13:56:32.950784 19693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3172d99afee4a4c878e71b645037917" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:32.951341 19693 raft_consensus.cc:2468] T b3172d99afee4a4c878e71b645037917 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:32.954667 19832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3172d99afee4a4c878e71b645037917" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:32.954989 19942 tablet_bootstrap.cc:492] T 7e544c93488541da9ebb588a465b27b7 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:32.955219 19832 raft_consensus.cc:2468] T b3172d99afee4a4c878e71b645037917 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:32.955492 19942 ts_tablet_manager.cc:1403] T 7e544c93488541da9ebb588a465b27b7 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.210s user 0.016s sys 0.017s
I20260925 13:56:32.956513 19832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d01a791445149e7bee9493e1eec5c6b" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:32.958050 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d8c01c88d24deeae21d647e7f205ba" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:32.957948 19942 raft_consensus.cc:359] T 7e544c93488541da9ebb588a465b27b7 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:32.958658 19942 raft_consensus.cc:385] T 7e544c93488541da9ebb588a465b27b7 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:32.959056 19942 raft_consensus.cc:740] T 7e544c93488541da9ebb588a465b27b7 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:32.959450 19668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4234e52546b948f3ac2542431f43771a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:32.960029 19666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5791a011b104ec59721b6ed8ce773f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:32.959857 19942 consensus_queue.cc:260] T 7e544c93488541da9ebb588a465b27b7 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:32.963980 19692 raft_consensus.cc:3060] T eff33a56361946edb556ac611d2299eb P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:32.964339 19689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e544c93488541da9ebb588a465b27b7" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:32.964891 19689 raft_consensus.cc:2468] T 7e544c93488541da9ebb588a465b27b7 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:32.965806 19942 ts_tablet_manager.cc:1434] T 7e544c93488541da9ebb588a465b27b7 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20260925 13:56:32.966595 19942 tablet_bootstrap.cc:492] T b23948807b2c4df9bf9a4da5f767e3b5 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:32.966962 20644 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 240) took 3934 ms. Trace:
I20260925 13:56:32.967235 20644 rpcz_store.cc:276] 0925 13:56:29.032871 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:29.061928 (+ 29057us) service_pool.cc:224] Handling call
0925 13:56:32.171818 (+3109890us) ts_tablet_manager.cc:1151] Deleting tablet 4d01a791445149e7bee9493e1eec5c6b
0925 13:56:32.171861 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:32.341931 (+170070us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b54000c5b54. stack: 00007f6c49bb9323 00007f6c49bb9220 00007f6c4975d574 0000000000516964 00000000005168f9 00007f6c50cb6aa1 00007f6c50d55b22 00007f6c5119650c 00007f6c51dffc3a 00007f6c51e30960 00007f6c51e16e6d 00007f6c51e16e19 00007f6c51e16da9 00007f6c51e16d71 00007f6c51e15f9d 00000000005bd4e4
0925 13:56:32.462266 (+120335us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:32.907815 (+445549us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:32.966898 (+ 59083us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":161919,"spinlock_wait_cycles":298517248,"tablet-delete.queue_time_us":3101079}
I20260925 13:56:32.968878 20644 tablet_replica.cc:333] T c5a9df833271480aa49feb7d9f925459 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:32.969626 20644 raft_consensus.cc:2243] T c5a9df833271480aa49feb7d9f925459 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:32.970054 20644 raft_consensus.cc:2272] T c5a9df833271480aa49feb7d9f925459 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:32.972955 19519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5791a011b104ec59721b6ed8ce773f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:32.982045 20955 raft_consensus.cc:493] T 0e8aa12e95dc434a98a426499593d9a5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:32.958643 19695 raft_consensus.cc:2468] T 78d8c01c88d24deeae21d647e7f205ba P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:32.982461 20955 raft_consensus.cc:515] T 0e8aa12e95dc434a98a426499593d9a5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:32.986294 20955 leader_election.cc:290] T 0e8aa12e95dc434a98a426499593d9a5 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:32.988731 19802 tablet_service.cc:1511] Processing CreateTablet for tablet 2fc32bc6d9ca43cb974c06714efe81a5 (DEFAULT_TABLE table=default.Table_52321f7ed87340e4a745a378171311dd [id=f50cb5b15bd94b2383244f024f5271e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:32.989857 19802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fc32bc6d9ca43cb974c06714efe81a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:32.990104 19942 tablet_bootstrap.cc:654] T b23948807b2c4df9bf9a4da5f767e3b5 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:33.006191 19955 raft_consensus.cc:2243] T eb794cef052b43da831531455e2b5f15 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:33.007264 19955 raft_consensus.cc:2272] T eb794cef052b43da831531455e2b5f15 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:33.021574 19819 tablet_service.cc:1511] Processing CreateTablet for tablet 940fe6cf272e4264aad6e2930f2cf54c (DEFAULT_TABLE table=default.Table_d57c934ba0b84027b1830f751a466939 [id=dd0a159f8fc445a3809eb8ec76acc34a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:33.022679 19819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 940fe6cf272e4264aad6e2930f2cf54c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:33.024135 19955 ts_tablet_manager.cc:1916] T eb794cef052b43da831531455e2b5f15 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:33.037714 19817 tablet_service.cc:1511] Processing CreateTablet for tablet 4234e52546b948f3ac2542431f43771a (DEFAULT_TABLE table=default.Table_b54d79ef34824edd919b01a1cdd16f8d [id=4b046318cc3b4bd1885e8c2a122aefdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:33.038699 19817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4234e52546b948f3ac2542431f43771a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:33.047219 19938 tablet_bootstrap.cc:492] T 37947005aaf14f27a6e5d1f30c67250c P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:33.047699 19938 ts_tablet_manager.cc:1403] T 37947005aaf14f27a6e5d1f30c67250c P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.590s user 0.013s sys 0.000s
W20260925 13:56:33.057051 19631 leader_election.cc:343] T 78d8c01c88d24deeae21d647e7f205ba P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 78d8c01c88d24deeae21d647e7f205ba
I20260925 13:56:33.068897 19549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d8c01c88d24deeae21d647e7f205ba" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:33.069698 19549 raft_consensus.cc:2468] T 78d8c01c88d24deeae21d647e7f205ba P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:33.054417 19814 tablet_service.cc:1511] Processing CreateTablet for tablet f49a9c7c11394c14bf8b09f671ea1854 (DEFAULT_TABLE table=default.Table_2d803b07af1d41a789c69141ec7f60f7 [id=39197c367fc848be9008667322367da5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:33.082173 19814 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f49a9c7c11394c14bf8b09f671ea1854. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:33.055768 19818 tablet_service.cc:1511] Processing CreateTablet for tablet f5791a011b104ec59721b6ed8ce773f1 (DEFAULT_TABLE table=default.Table_86568b68e691429192052565e933af79 [id=3a35752b668747c8bc1485701e8e7282]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:33.088542 19818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5791a011b104ec59721b6ed8ce773f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:33.113799 19938 raft_consensus.cc:359] T 37947005aaf14f27a6e5d1f30c67250c P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:33.114544 19938 raft_consensus.cc:385] T 37947005aaf14f27a6e5d1f30c67250c P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:33.114884 19938 raft_consensus.cc:740] T 37947005aaf14f27a6e5d1f30c67250c P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:33.135783 20885 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bcc92728d49643efab5528d77c3d7828" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } } }
I20260925 13:56:33.136545 20885 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828 [sys.catalog]: This master's current role is: LEADER
I20260925 13:56:33.148676 19669 tablet_service.cc:1558] Processing DeleteTablet for tablet 000aad39953c4688b543c7d882a08ec8 with delete_type TABLET_DATA_DELETED (replaced by tablet 3927adc85b2746d0977a3cb6f84ad68f) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:33.148813 19666 tablet_service.cc:1558] Processing DeleteTablet for tablet e2e44b2ca25c4bbcb91288424b3e18b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:24 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:33.145815 19272 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 82972fa4868d4c59b3b9bb1240300938 (table default.Table_d3d2076fbb6241a68f04e83462a34182 [id=4232cecee9104120ac2521eb250bbd0d]). Expected version 1 got 0
I20260925 13:56:33.154829 19272 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 5f8a96ae6eec4f15a221f9aad537c094 (table default.Table_3a06919b658e46f893318b4e1ec8b52f [id=e6f2a872536b4e55b071231020903581]). Expected version 1 got 0
W20260925 13:56:33.155906 19279 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.194:34125 (request call id 47) took 1651 ms (client timeout 1000 ms). Trace:
W20260925 13:56:33.159329 19279 rpcz_store.cc:269] 0925 13:56:31.504562 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:31.616565 (+112003us) service_pool.cc:224] Handling call
0925 13:56:31.634129 (+ 17564us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b5400040454. stack: 00007fdf0a30b323 00007fdf0a30b220 00007fdf09eaf574 0000000000516964 00000000005168f9 00007fdf114095c0 00007fdf114a86fa 00007fdf13219d8c 00007fdf1331801e 00007fdf10ebcfb4 00007fdf10ebcf42 00007fdf10ebce71 00007fdf10ebcdec 00007fdf10ebc0a2 00007fdf0f16e09c 00007fdf0f16d4d6
0925 13:56:33.155841 (+1521712us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:32.599801 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:32.600215 (+ 414us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:32.600238 (+ 23us) write_op.cc:435] Acquired schema lock
0925 13:56:32.600254 (+ 16us) tablet.cc:662] Decoding operations
0925 13:56:32.600458 (+ 204us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:32.600488 (+ 30us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:32.600508 (+ 20us) tablet.cc:685] Acquiring locks for 1 operations
0925 13:56:32.600624 (+ 116us) tablet.cc:701] Row locks acquired
0925 13:56:32.600636 (+ 12us) write_op.cc:260] PREPARE: finished
0925 13:56:32.600731 (+ 95us) write_op.cc:270] Start()
0925 13:56:32.600802 (+ 71us) write_op.cc:276] Timestamp: P: 1790344592600695 usec, L: 0
0925 13:56:32.600819 (+ 17us) op_driver.cc:348] REPLICATION: starting
0925 13:56:32.601205 (+ 386us) log.cc:844] Serialized 536 byte log entry
0925 13:56:33.109705 (+508500us) op_driver.cc:464] REPLICATION: finished
0925 13:56:33.120489 (+ 10784us) write_op.cc:301] APPLY: starting
0925 13:56:33.120539 (+ 50us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:33.120632 (+ 93us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:33.121149 (+ 517us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:33.121286 (+ 137us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:33.121405 (+ 119us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 13:56:33.121446 (+ 41us) write_op.cc:312] APPLY: finished
0925 13:56:33.121586 (+ 140us) log.cc:844] Serialized 23 byte log entry
0925 13:56:33.121712 (+ 126us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:33.121760 (+ 48us) write_op.cc:454] Released schema lock
0925 13:56:33.121788 (+ 28us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":12,"spinlock_wait_cycles":34394880,"child_traces":[["op",{"apply.queue_time_us":2948,"apply.run_cpu_time_us":1475,"apply.run_wall_time_us":2679,"num_ops":1,"prepare.queue_time_us":113282,"prepare.run_cpu_time_us":1576,"prepare.run_wall_time_us":20009,"replication_time_us":508795,"thread_start_us":17256,"threads_started":1}]]}
I20260925 13:56:33.163704 19631 leader_election.cc:304] T b3172d99afee4a4c878e71b645037917 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:33.164420 20927 raft_consensus.cc:2804] T b3172d99afee4a4c878e71b645037917 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:33.164726 20927 raft_consensus.cc:493] T b3172d99afee4a4c878e71b645037917 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:33.165014 20927 raft_consensus.cc:3060] T b3172d99afee4a4c878e71b645037917 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:33.169451 19942 tablet_bootstrap.cc:492] T b23948807b2c4df9bf9a4da5f767e3b5 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
W20260925 13:56:33.171972 19264 catalog_manager.cc:5030] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 000aad39953c4688b543c7d882a08ec8 because the tablet was not found. No further retry: Not found: Tablet not found: 000aad39953c4688b543c7d882a08ec8
I20260925 13:56:33.164403 19272 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 505a24ebe7014f80b3c184329cc74a9a (table default.Table_f071a5a526034801828291eb010c40c0 [id=2e58540132af4ed298c50ddea3540cdf]). Expected version 1 got 0
I20260925 13:56:33.183634 19942 ts_tablet_manager.cc:1403] T b23948807b2c4df9bf9a4da5f767e3b5 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.217s user 0.012s sys 0.035s
I20260925 13:56:33.184055 20856 raft_consensus.cc:493] T e9dbcd85dc8647c68c120ba3fcfe1790 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:33.184510 20856 raft_consensus.cc:515] T e9dbcd85dc8647c68c120ba3fcfe1790 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:33.186436 20856 leader_election.cc:290] T e9dbcd85dc8647c68c120ba3fcfe1790 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:33.171787 20981 tablet_replica.cc:333] T e2e44b2ca25c4bbcb91288424b3e18b9 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:33.192183 20981 raft_consensus.cc:2243] T e2e44b2ca25c4bbcb91288424b3e18b9 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:33.192760 20981 raft_consensus.cc:2272] T e2e44b2ca25c4bbcb91288424b3e18b9 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:33.171226 20927 raft_consensus.cc:515] T b3172d99afee4a4c878e71b645037917 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:33.199970 19631 leader_election.cc:343] T 4d01a791445149e7bee9493e1eec5c6b P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Illegal state: Tablet not RUNNING: STOPPED
I20260925 13:56:33.203015 19276 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 82972fa4868d4c59b3b9bb1240300938 (table default.Table_d3d2076fbb6241a68f04e83462a34182 [id=4232cecee9104120ac2521eb250bbd0d]). Expected version 1 got 0
I20260925 13:56:33.186163 19942 raft_consensus.cc:359] T b23948807b2c4df9bf9a4da5f767e3b5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:33.211391 19942 raft_consensus.cc:385] T b23948807b2c4df9bf9a4da5f767e3b5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:33.213864 19276 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet f3b46e4fac694084b06b7086bcc596b9 (table default.Table_2efda975ab0a464eaf96f1f9a48eff1c [id=c31f813d79a244afa726df0238376aba]). Expected version 1 got 0
I20260925 13:56:33.214470 19942 raft_consensus.cc:740] T b23948807b2c4df9bf9a4da5f767e3b5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:33.219638 20927 leader_election.cc:290] T b3172d99afee4a4c878e71b645037917 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:33.220847 19942 consensus_queue.cc:260] T b23948807b2c4df9bf9a4da5f767e3b5 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:33.226228 19942 ts_tablet_manager.cc:1434] T b23948807b2c4df9bf9a4da5f767e3b5 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.042s user 0.005s sys 0.000s
I20260925 13:56:33.227838 19276 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 5f8a96ae6eec4f15a221f9aad537c094 (table default.Table_3a06919b658e46f893318b4e1ec8b52f [id=e6f2a872536b4e55b071231020903581]). Expected version 1 got 0
I20260925 13:56:33.215302 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 4d01a791445149e7bee9493e1eec5c6b (table default.Table_883c781744824c2a9e6e86917bf2143e [id=4e1702861c25473e94c5ce639f3ab397]) successfully deleted
I20260925 13:56:33.234470 19942 tablet_bootstrap.cc:492] T 82c1002d2bcb4b11839e2f4a93b899ad P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:33.215526 20981 ts_tablet_manager.cc:1916] T e2e44b2ca25c4bbcb91288424b3e18b9 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 13:56:33.231277 19272 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.195:57799 (request call id 43) took 821 ms (client timeout 1000 ms). Trace:
W20260925 13:56:33.251700 19272 rpcz_store.cc:269] 0925 13:56:32.410362 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:32.514952 (+104590us) service_pool.cc:224] Handling call
0925 13:56:32.865714 (+350762us) spinlock_profiling.cc:243] Waited 299 ms on lock 0x7b5400040454. stack: 00007fdf0a30b323 00007fdf0a30b220 00007fdf09eaf574 0000000000516964 00000000005168f9 00007fdf114095c0 00007fdf114a86fa 00007fdf13219d8c 00007fdf1331801e 00007fdf10ebcfb4 00007fdf10ebcf42 00007fdf10ebce71 00007fdf10ebcdec 00007fdf10ebc0a2 00007fdf0f16e09c 00007fdf0f16d4d6
0925 13:56:33.231218 (+365504us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":18,"spinlock_wait_cycles":686776064}
I20260925 13:56:33.255064 19276 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet e9dbcd85dc8647c68c120ba3fcfe1790 (table default.Table_5bf9b87b5083469fad04eb3a80a4b8d5 [id=443536a0bd4d4d568d52f6ffe3d06ecb]). Expected version 1 got 0
I20260925 13:56:33.255908 19276 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet eff33a56361946edb556ac611d2299eb (table default.Table_33e9cd401c9d45d0a1af408bda5e3bfa [id=2c74fb31a35541c4a2dd0b2171ba41ce]). Expected version 1 got 0
I20260925 13:56:33.151751 20644 ts_tablet_manager.cc:1916] T c5a9df833271480aa49feb7d9f925459 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:33.260210 19938 consensus_queue.cc:260] T 37947005aaf14f27a6e5d1f30c67250c P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:33.261544 19938 ts_tablet_manager.cc:1434] T 37947005aaf14f27a6e5d1f30c67250c P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.150s user 0.003s sys 0.000s
I20260925 13:56:33.262348 19938 tablet_bootstrap.cc:492] T 31b5f47966a74ec3b59c7c20479ba702 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
W20260925 13:56:33.278597 19488 leader_election.cc:343] T 5a9b3dfaa7474611a1824d63eb906b43 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Not found: Tablet not found: 5a9b3dfaa7474611a1824d63eb906b43
I20260925 13:56:33.293421 19942 tablet_bootstrap.cc:654] T 82c1002d2bcb4b11839e2f4a93b899ad P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:33.279027 19488 leader_election.cc:304] T 5a9b3dfaa7474611a1824d63eb906b43 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3; no voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:33.250403 19773 leader_election.cc:304] T b3172d99afee4a4c878e71b645037917 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:33.304167 20891 raft_consensus.cc:2804] T b3172d99afee4a4c878e71b645037917 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:33.304517 20891 raft_consensus.cc:493] T b3172d99afee4a4c878e71b645037917 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:33.304816 20891 raft_consensus.cc:3060] T b3172d99afee4a4c878e71b645037917 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:33.311285 20891 raft_consensus.cc:515] T b3172d99afee4a4c878e71b645037917 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:33.313094 20891 leader_election.cc:290] T b3172d99afee4a4c878e71b645037917 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
W20260925 13:56:33.318463 19773 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.194:38641, user_credentials={real_user=slave}} blocked reactor thread for 68241.7us
I20260925 13:56:33.319077 19773 leader_election.cc:304] T c5a9df833271480aa49feb7d9f925459 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
W20260925 13:56:33.334669 19773 raft_consensus.cc:2686] T c5a9df833271480aa49feb7d9f925459 P f790493ba7ad403993b678f2df2b55be: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260925 13:56:33.346100 19274 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 40e326118fad4811b1d3ebd7f632ca92 has no consensus state
I20260925 13:56:33.346282 19273 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet f753f6e45ade49ffa3128d5f4e09584a
I20260925 13:56:33.347925 19669 tablet_service.cc:1558] Processing DeleteTablet for tablet f753f6e45ade49ffa3128d5f4e09584a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:32 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:33.350600 19163 master-stress-test.cc:581] Killed test thread 1/10
I20260925 13:56:33.350834 19163 master-stress-test.cc:579] Killing test thread 2/10
I20260925 13:56:33.351058 19163 master-stress-test.cc:581] Killed test thread 2/10
I20260925 13:56:33.351240 19163 master-stress-test.cc:579] Killing test thread 3/10
I20260925 13:56:33.351495 19163 master-stress-test.cc:581] Killed test thread 3/10
I20260925 13:56:33.351698 19163 master-stress-test.cc:579] Killing test thread 4/10
I20260925 13:56:33.352406 19280 catalog_manager.cc:6128] Not sending DeleteTablet requests; no consensus state for tablet e49ccb9804104f70bdea2d34d78ceb4b
I20260925 13:56:33.352592 19163 master-stress-test.cc:581] Killed test thread 4/10
I20260925 13:56:33.352804 19163 master-stress-test.cc:579] Killing test thread 5/10
I20260925 13:56:33.353005 19163 master-stress-test.cc:581] Killed test thread 5/10
I20260925 13:56:33.353185 19163 master-stress-test.cc:579] Killing test thread 6/10
I20260925 13:56:33.353380 19163 master-stress-test.cc:581] Killed test thread 6/10
I20260925 13:56:33.353540 19163 master-stress-test.cc:579] Killing test thread 7/10
I20260925 13:56:33.353971 19163 master-stress-test.cc:581] Killed test thread 7/10
I20260925 13:56:33.354167 19163 master-stress-test.cc:579] Killing test thread 8/10
I20260925 13:56:33.354424 19163 master-stress-test.cc:581] Killed test thread 8/10
W20260925 13:56:33.354413 19488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.194:38641, user_credentials={real_user=slave}} blocked reactor thread for 75849.4us
I20260925 13:56:33.354600 19163 master-stress-test.cc:579] Killing test thread 9/10
I20260925 13:56:33.355782 19694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3172d99afee4a4c878e71b645037917" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:33.356596 19694 raft_consensus.cc:2393] T b3172d99afee4a4c878e71b645037917 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f790493ba7ad403993b678f2df2b55be in current term 1: Already voted for candidate 7c1b52895a14482fb54a6758f5054f8f in this term.
I20260925 13:56:33.361848 19551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3172d99afee4a4c878e71b645037917" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:33.362030 19547 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3172d99afee4a4c878e71b645037917" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:33.362469 19551 raft_consensus.cc:2468] T b3172d99afee4a4c878e71b645037917 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:33.364588 19550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d01a791445149e7bee9493e1eec5c6b" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:33.365314 19550 raft_consensus.cc:2393] T 4d01a791445149e7bee9493e1eec5c6b P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7c1b52895a14482fb54a6758f5054f8f in current term 1: Already voted for candidate f790493ba7ad403993b678f2df2b55be in this term.
I20260925 13:56:33.366139 19552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9dbcd85dc8647c68c120ba3fcfe1790" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:33.366513 19272 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 5f8a96ae6eec4f15a221f9aad537c094 (table default.Table_3a06919b658e46f893318b4e1ec8b52f [id=e6f2a872536b4e55b071231020903581]). Expected version 1 got 0
I20260925 13:56:33.368997 19669 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d01a791445149e7bee9493e1eec5c6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:21 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:33.371737 19833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23948807b2c4df9bf9a4da5f767e3b5" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:33.372364 19833 raft_consensus.cc:2468] T b23948807b2c4df9bf9a4da5f767e3b5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:33.377851 19938 tablet_bootstrap.cc:654] T 31b5f47966a74ec3b59c7c20479ba702 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:33.378216 20981 ts_tablet_manager.cc:1929] T e2e44b2ca25c4bbcb91288424b3e18b9 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:33.181705 19955 ts_tablet_manager.cc:1929] T eb794cef052b43da831531455e2b5f15 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:33.379096 19955 log.cc:1199] T eb794cef052b43da831531455e2b5f15 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/eb794cef052b43da831531455e2b5f15
W20260925 13:56:33.381240 19488 leader_election.cc:343] T b23948807b2c4df9bf9a4da5f767e3b5 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:33.381912 19488 consensus_peers.cc:597] T 000aad39953c4688b543c7d882a08ec8 P 8f578869bc084a5d8a9a69f143954eb3 -> Peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Couldn't send request to peer 7c1b52895a14482fb54a6758f5054f8f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 000aad39953c4688b543c7d882a08ec8. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:33.367218 19272 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 11077cdb735840b5921e22f588059149 (table default.Table_e4f4f8c05f904aa485bd1b4c5a96d9c8 [id=c219cb8664c64ce5aba0ed135c171937]). Expected version 1 got 0
I20260925 13:56:33.386909 19488 leader_election.cc:304] T b3172d99afee4a4c878e71b645037917 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3; no voters:
I20260925 13:56:33.395624 20981 log.cc:1199] T e2e44b2ca25c4bbcb91288424b3e18b9 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/e2e44b2ca25c4bbcb91288424b3e18b9
I20260925 13:56:33.396373 20981 ts_tablet_manager.cc:1950] T e2e44b2ca25c4bbcb91288424b3e18b9 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:33.399343 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet e2e44b2ca25c4bbcb91288424b3e18b9 (table default.Table_261023ab15a845df82deea31f0eac8c8 [id=f03b9baa231c40999b38848e9059553e]) successfully deleted
I20260925 13:56:33.388392 20794 consensus_queue.cc:1048] T e9dbcd85dc8647c68c120ba3fcfe1790 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260925 13:56:33.395936 20795 consensus_queue.cc:1048] T f3b46e4fac694084b06b7086bcc596b9 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260925 13:56:33.388070 20955 raft_consensus.cc:2749] T 5a9b3dfaa7474611a1824d63eb906b43 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:56:33.428112 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet f753f6e45ade49ffa3128d5f4e09584a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f753f6e45ade49ffa3128d5f4e09584a already in progress: creating tablet
I20260925 13:56:33.429117 20981 tablet_replica.cc:333] T 4d01a791445149e7bee9493e1eec5c6b P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:33.394425 19835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e544c93488541da9ebb588a465b27b7" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:33.431471 19955 ts_tablet_manager.cc:1950] T eb794cef052b43da831531455e2b5f15 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:33.432559 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 219) took 6074 ms. Trace:
I20260925 13:56:33.433039 19955 rpcz_store.cc:276] 0925 13:56:27.357642 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:27.357771 (+ 129us) service_pool.cc:224] Handling call
0925 13:56:32.425063 (+5067292us) ts_tablet_manager.cc:1151] Deleting tablet eb794cef052b43da831531455e2b5f15
0925 13:56:32.431425 (+ 6362us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:33.052175 (+620750us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:33.181257 (+129082us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:33.432452 (+251195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":573157,"tablet-delete.queue_time_us":5033351}
I20260925 13:56:33.436791 20956 raft_consensus.cc:2804] T b3172d99afee4a4c878e71b645037917 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:33.437165 20956 raft_consensus.cc:493] T b3172d99afee4a4c878e71b645037917 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:33.437461 20956 raft_consensus.cc:3060] T b3172d99afee4a4c878e71b645037917 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:33.395387 19836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3172d99afee4a4c878e71b645037917" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:33.411957 19817 tablet_service.cc:1558] Processing DeleteTablet for tablet f753f6e45ade49ffa3128d5f4e09584a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:32 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:33.416440 19830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9dbcd85dc8647c68c120ba3fcfe1790" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:33.422011 20831 raft_consensus.cc:493] T 7e544c93488541da9ebb588a465b27b7 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:33.439769 19830 raft_consensus.cc:2468] T e9dbcd85dc8647c68c120ba3fcfe1790 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
I20260925 13:56:33.439831 20831 raft_consensus.cc:515] T 7e544c93488541da9ebb588a465b27b7 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:33.430073 20981 raft_consensus.cc:2243] T 4d01a791445149e7bee9493e1eec5c6b P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:33.440940 19835 raft_consensus.cc:2468] T 7e544c93488541da9ebb588a465b27b7 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:33.431933 19629 leader_election.cc:304] T eff33a56361946edb556ac611d2299eb P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:33.441361 20981 raft_consensus.cc:2272] T 4d01a791445149e7bee9493e1eec5c6b P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:33.441565 20856 raft_consensus.cc:2749] T eff33a56361946edb556ac611d2299eb P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:33.441767 20831 leader_election.cc:290] T 7e544c93488541da9ebb588a465b27b7 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:33.423727 19840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3172d99afee4a4c878e71b645037917" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:33.442715 19629 leader_election.cc:304] T 5f8a96ae6eec4f15a221f9aad537c094 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:33.443310 20856 raft_consensus.cc:2749] T 5f8a96ae6eec4f15a221f9aad537c094 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:33.443295 20981 ts_tablet_manager.cc:1916] T 4d01a791445149e7bee9493e1eec5c6b P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:33.443988 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet eb794cef052b43da831531455e2b5f15 (table default.Table_6dd81ec6a73d459581359a7c93b23f9c [id=d2bc135460ab4f2fad4740a281bfc6e2]) successfully deleted
I20260925 13:56:33.439849 19836 raft_consensus.cc:2393] T b3172d99afee4a4c878e71b645037917 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f578869bc084a5d8a9a69f143954eb3 in current term 1: Already voted for candidate f790493ba7ad403993b678f2df2b55be in this term.
I20260925 13:56:33.430063 19942 tablet_bootstrap.cc:492] T 82c1002d2bcb4b11839e2f4a93b899ad P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:33.448449 19942 ts_tablet_manager.cc:1403] T 82c1002d2bcb4b11839e2f4a93b899ad P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.214s user 0.015s sys 0.003s
I20260925 13:56:33.450796 19942 raft_consensus.cc:359] T 82c1002d2bcb4b11839e2f4a93b899ad P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:33.451788 19942 raft_consensus.cc:385] T 82c1002d2bcb4b11839e2f4a93b899ad P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:33.452037 19942 raft_consensus.cc:740] T 82c1002d2bcb4b11839e2f4a93b899ad P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:33.454429 20891 consensus_queue.cc:1048] T 5f8a96ae6eec4f15a221f9aad537c094 P f790493ba7ad403993b678f2df2b55be [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 13:56:33.455894 19773 leader_election.cc:343] T 505a24ebe7014f80b3c184329cc74a9a P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:33.456246 19773 leader_election.cc:304] T 505a24ebe7014f80b3c184329cc74a9a P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:33.456393 19523 tablet_service.cc:1558] Processing DeleteTablet for tablet f753f6e45ade49ffa3128d5f4e09584a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:32 UTC) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:33.456934 20891 raft_consensus.cc:2749] T 505a24ebe7014f80b3c184329cc74a9a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:33.461638 19279 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet 82972fa4868d4c59b3b9bb1240300938 (table default.Table_d3d2076fbb6241a68f04e83462a34182 [id=4232cecee9104120ac2521eb250bbd0d]). Expected version 1 got 0
I20260925 13:56:33.462563 19279 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet b3172d99afee4a4c878e71b645037917 (table default.Table_bd52a86b8ff14bdbb06fcf88a60b1e82 [id=203024d67e444f839391c8670d7c7d3b]). Expected version 2 got 0
I20260925 13:56:33.463156 19279 catalog_manager.cc:5838] T f3b46e4fac694084b06b7086bcc596b9 P 8f578869bc084a5d8a9a69f143954eb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "8f578869bc084a5d8a9a69f143954eb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:33.464745 19279 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet f3b46e4fac694084b06b7086bcc596b9 (table default.Table_2efda975ab0a464eaf96f1f9a48eff1c [id=c31f813d79a244afa726df0238376aba]). Expected version 1 got 0
I20260925 13:56:33.466104 19279 catalog_manager.cc:5838] T e9dbcd85dc8647c68c120ba3fcfe1790 P 8f578869bc084a5d8a9a69f143954eb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "8f578869bc084a5d8a9a69f143954eb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: HEALTHY } } }
I20260925 13:56:33.467214 19279 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet e9dbcd85dc8647c68c120ba3fcfe1790 (table default.Table_5bf9b87b5083469fad04eb3a80a4b8d5 [id=443536a0bd4d4d568d52f6ffe3d06ecb]). Expected version 1 got 0
I20260925 13:56:33.468008 19279 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet 7e544c93488541da9ebb588a465b27b7 (table default.Table_4f849dc343f5448eb14a195bb0777d12 [id=f71e790b94b34e3d88f7539e396bc94d]). Expected version 1 got 0
I20260925 13:56:33.468773 19279 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet 5f8a96ae6eec4f15a221f9aad537c094 (table default.Table_3a06919b658e46f893318b4e1ec8b52f [id=e6f2a872536b4e55b071231020903581]). Expected version 1 got 0
I20260925 13:56:33.469774 19279 catalog_manager.cc:5838] T eff33a56361946edb556ac611d2299eb P 8f578869bc084a5d8a9a69f143954eb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "8f578869bc084a5d8a9a69f143954eb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: HEALTHY } } }
I20260925 13:56:33.434777 19955 tablet_replica.cc:333] T 0e8aa12e95dc434a98a426499593d9a5 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:33.470853 19279 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet eff33a56361946edb556ac611d2299eb (table default.Table_33e9cd401c9d45d0a1af408bda5e3bfa [id=2c74fb31a35541c4a2dd0b2171ba41ce]). Expected version 1 got 0
I20260925 13:56:33.473053 19939 tablet_bootstrap.cc:492] T 82c1002d2bcb4b11839e2f4a93b899ad P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:33.473464 19939 ts_tablet_manager.cc:1403] T 82c1002d2bcb4b11839e2f4a93b899ad P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.591s user 0.037s sys 0.058s
I20260925 13:56:33.476076 19939 raft_consensus.cc:359] T 82c1002d2bcb4b11839e2f4a93b899ad P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:33.476655 19939 raft_consensus.cc:385] T 82c1002d2bcb4b11839e2f4a93b899ad P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:33.476897 19939 raft_consensus.cc:740] T 82c1002d2bcb4b11839e2f4a93b899ad P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:33.477568 19939 consensus_queue.cc:260] T 82c1002d2bcb4b11839e2f4a93b899ad P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:33.478922 19939 ts_tablet_manager.cc:1434] T 82c1002d2bcb4b11839e2f4a93b899ad P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
W20260925 13:56:33.481925 19773 leader_election.cc:343] T 11077cdb735840b5921e22f588059149 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:33.482296 19773 leader_election.cc:304] T 11077cdb735840b5921e22f588059149 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:33.483026 20891 raft_consensus.cc:2749] T 11077cdb735840b5921e22f588059149 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:33.485893 20956 raft_consensus.cc:515] T b3172d99afee4a4c878e71b645037917 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:33.487732 19279 catalog_manager.cc:5938] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269) does not have the latest schema for tablet b23948807b2c4df9bf9a4da5f767e3b5 (table default.Table_fd205659579c49559fd2d121b32274d4 [id=0165d602da03423f82104b16da9a20b0]). Expected version 1 got 0
I20260925 13:56:33.490515 19942 consensus_queue.cc:260] T 82c1002d2bcb4b11839e2f4a93b899ad P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:33.503952 19938 tablet_bootstrap.cc:492] T 31b5f47966a74ec3b59c7c20479ba702 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:33.504387 19938 ts_tablet_manager.cc:1403] T 31b5f47966a74ec3b59c7c20479ba702 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.242s user 0.007s sys 0.002s
I20260925 13:56:33.507519 20981 ts_tablet_manager.cc:1929] T 4d01a791445149e7bee9493e1eec5c6b P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:33.507895 20981 log.cc:1199] T 4d01a791445149e7bee9493e1eec5c6b P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/4d01a791445149e7bee9493e1eec5c6b
I20260925 13:56:33.508581 20981 ts_tablet_manager.cc:1950] T 4d01a791445149e7bee9493e1eec5c6b P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:33.508704 19694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e544c93488541da9ebb588a465b27b7" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:33.509477 19694 raft_consensus.cc:2468] T 7e544c93488541da9ebb588a465b27b7 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 0.
I20260925 13:56:33.515453 19938 raft_consensus.cc:359] T 31b5f47966a74ec3b59c7c20479ba702 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:33.516171 19938 raft_consensus.cc:385] T 31b5f47966a74ec3b59c7c20479ba702 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:33.516449 19938 raft_consensus.cc:740] T 31b5f47966a74ec3b59c7c20479ba702 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:33.517207 19938 consensus_queue.cc:260] T 31b5f47966a74ec3b59c7c20479ba702 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:33.519788 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 4d01a791445149e7bee9493e1eec5c6b (table default.Table_883c781744824c2a9e6e86917bf2143e [id=4e1702861c25473e94c5ce639f3ab397]) successfully deleted
I20260925 13:56:33.522192 19629 leader_election.cc:304] T 78d8c01c88d24deeae21d647e7f205ba P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3; no voters: f790493ba7ad403993b678f2df2b55be
I20260925 13:56:33.522234 19631 leader_election.cc:304] T e9dbcd85dc8647c68c120ba3fcfe1790 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:33.524878 20988 raft_consensus.cc:2804] T 78d8c01c88d24deeae21d647e7f205ba P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:33.525221 20988 raft_consensus.cc:493] T 78d8c01c88d24deeae21d647e7f205ba P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:33.525506 20988 raft_consensus.cc:3060] T 78d8c01c88d24deeae21d647e7f205ba P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:33.523259 20856 raft_consensus.cc:2804] T e9dbcd85dc8647c68c120ba3fcfe1790 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 13:56:33.528244 20856 raft_consensus.cc:493] T e9dbcd85dc8647c68c120ba3fcfe1790 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:33.528538 20856 raft_consensus.cc:3060] T e9dbcd85dc8647c68c120ba3fcfe1790 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Advancing to term 2
I20260925 13:56:33.530112 19942 ts_tablet_manager.cc:1434] T 82c1002d2bcb4b11839e2f4a93b899ad P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.081s user 0.006s sys 0.000s
I20260925 13:56:33.530951 19942 tablet_bootstrap.cc:492] T 1ce5d15e67644247b95960f444acc621 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:33.540400 20988 raft_consensus.cc:515] T 78d8c01c88d24deeae21d647e7f205ba P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:33.546355 20988 leader_election.cc:290] T 78d8c01c88d24deeae21d647e7f205ba P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:33.546845 20856 raft_consensus.cc:515] T e9dbcd85dc8647c68c120ba3fcfe1790 P 7c1b52895a14482fb54a6758f5054f8f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:33.561841 20856 leader_election.cc:290] T e9dbcd85dc8647c68c120ba3fcfe1790 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 2 election: Requested vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:33.495822 19939 tablet_bootstrap.cc:492] T 11077cdb735840b5921e22f588059149 P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:33.580430 20956 leader_election.cc:290] T b3172d99afee4a4c878e71b645037917 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
W20260925 13:56:33.563692 19490 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.18.182.252:36181, user_credentials={real_user=slave}} blocked reactor thread for 52039.3us
I20260925 13:56:33.582738 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e544c93488541da9ebb588a465b27b7" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
W20260925 13:56:33.599833 19631 leader_election.cc:343] T 78d8c01c88d24deeae21d647e7f205ba P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 78d8c01c88d24deeae21d647e7f205ba
I20260925 13:56:33.600909 19773 leader_election.cc:304] T 7e544c93488541da9ebb588a465b27b7 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:33.601665 20891 raft_consensus.cc:2804] T 7e544c93488541da9ebb588a465b27b7 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:33.602041 20891 raft_consensus.cc:493] T 7e544c93488541da9ebb588a465b27b7 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:33.602391 20891 raft_consensus.cc:3060] T 7e544c93488541da9ebb588a465b27b7 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:33.605664 19276 master_service.cc:960] ReplaceTablet: received request to replace tablet 3d754e17930549189d65becb98afe2a6 from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:33.610754 20891 raft_consensus.cc:515] T 7e544c93488541da9ebb588a465b27b7 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:33.615763 20891 leader_election.cc:290] T 7e544c93488541da9ebb588a465b27b7 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:33.620522 19938 ts_tablet_manager.cc:1434] T 31b5f47966a74ec3b59c7c20479ba702 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.116s user 0.005s sys 0.000s
I20260925 13:56:33.621413 19938 tablet_bootstrap.cc:492] T f146a9aee3054d7491e74101df0e28e1 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:33.628317 19272 catalog_manager.cc:6391] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: default.Table_4d6d747ea4034b3ab48710e10ff4544f [id=d6e6dd0772bf4901b696e11b10e41a48] alter complete (version 1)
I20260925 13:56:33.630023 20644 ts_tablet_manager.cc:1929] T c5a9df833271480aa49feb7d9f925459 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:33.630383 20644 log.cc:1199] T c5a9df833271480aa49feb7d9f925459 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/c5a9df833271480aa49feb7d9f925459
I20260925 13:56:33.631177 20644 ts_tablet_manager.cc:1950] T c5a9df833271480aa49feb7d9f925459 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:33.632107 19836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9dbcd85dc8647c68c120ba3fcfe1790" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:33.632645 19836 raft_consensus.cc:3060] T e9dbcd85dc8647c68c120ba3fcfe1790 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Advancing to term 2
I20260925 13:56:33.583268 19555 raft_consensus.cc:2468] T 7e544c93488541da9ebb588a465b27b7 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 0.
I20260925 13:56:33.644714 19550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3172d99afee4a4c878e71b645037917" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:33.645463 19939 tablet_bootstrap.cc:654] T 11077cdb735840b5921e22f588059149 P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:33.645474 19550 raft_consensus.cc:2393] T b3172d99afee4a4c878e71b645037917 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f790493ba7ad403993b678f2df2b55be in current term 1: Already voted for candidate 8f578869bc084a5d8a9a69f143954eb3 in this term.
I20260925 13:56:33.677282 19942 tablet_bootstrap.cc:654] T 1ce5d15e67644247b95960f444acc621 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:33.671254 19693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e544c93488541da9ebb588a465b27b7" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:33.681510 19693 raft_consensus.cc:3060] T 7e544c93488541da9ebb588a465b27b7 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:33.682893 19938 tablet_bootstrap.cc:654] T f146a9aee3054d7491e74101df0e28e1 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:33.697139 19693 raft_consensus.cc:2468] T 7e544c93488541da9ebb588a465b27b7 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 1.
I20260925 13:56:33.698107 19773 leader_election.cc:304] T 7e544c93488541da9ebb588a465b27b7 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:33.698789 20891 raft_consensus.cc:2804] T 7e544c93488541da9ebb588a465b27b7 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:33.699210 20891 raft_consensus.cc:697] T 7e544c93488541da9ebb588a465b27b7 P f790493ba7ad403993b678f2df2b55be [term 1 LEADER]: Becoming Leader. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Running, Role: LEADER
I20260925 13:56:33.699940 20891 consensus_queue.cc:237] T 7e544c93488541da9ebb588a465b27b7 P f790493ba7ad403993b678f2df2b55be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:33.724855 19938 tablet_bootstrap.cc:492] T f146a9aee3054d7491e74101df0e28e1 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:33.725356 19938 ts_tablet_manager.cc:1403] T f146a9aee3054d7491e74101df0e28e1 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.104s user 0.009s sys 0.000s
I20260925 13:56:33.728399 19938 raft_consensus.cc:359] T f146a9aee3054d7491e74101df0e28e1 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:33.732604 19942 tablet_bootstrap.cc:492] T 1ce5d15e67644247b95960f444acc621 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:33.732982 19942 ts_tablet_manager.cc:1403] T 1ce5d15e67644247b95960f444acc621 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.202s user 0.011s sys 0.005s
I20260925 13:56:33.735729 19519 tablet_service.cc:1558] Processing DeleteTablet for tablet 78d8c01c88d24deeae21d647e7f205ba with delete_type TABLET_DATA_DELETED (replaced by tablet ab1ba6ef73a046b59ffac150f7c73953) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:33.742955 19938 raft_consensus.cc:385] T f146a9aee3054d7491e74101df0e28e1 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:33.743285 19938 raft_consensus.cc:740] T f146a9aee3054d7491e74101df0e28e1 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:33.752220 19955 raft_consensus.cc:2243] T 0e8aa12e95dc434a98a426499593d9a5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:33.752725 19955 raft_consensus.cc:2272] T 0e8aa12e95dc434a98a426499593d9a5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:33.735116 19942 raft_consensus.cc:359] T 1ce5d15e67644247b95960f444acc621 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:33.753333 19942 raft_consensus.cc:385] T 1ce5d15e67644247b95960f444acc621 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:33.753597 19942 raft_consensus.cc:740] T 1ce5d15e67644247b95960f444acc621 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:33.754185 20891 raft_consensus.cc:493] T 37947005aaf14f27a6e5d1f30c67250c P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:33.754506 19514 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:40014 (request call id 39) took 1160 ms. Trace:
I20260925 13:56:33.754586 20891 raft_consensus.cc:515] T 37947005aaf14f27a6e5d1f30c67250c P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:33.755060 19955 ts_tablet_manager.cc:1916] T 0e8aa12e95dc434a98a426499593d9a5 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:33.754832 19514 rpcz_store.cc:276] 0925 13:56:32.594489 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:32.598522 (+ 4033us) service_pool.cc:224] Handling call
0925 13:56:32.828422 (+229900us) spinlock_profiling.cc:243] Waited 67 ms on lock 0x7b200005c678. stack: 00007f1358052323 00007f1358052220 00007f1357bf6574 0000000000516964 00000000005168f9 00007f135efcad71 00007f135efec83c 00007f135efbdd52 00007f135f6328d2 00007f135f632635 00007f1360251960 00007f135e9a1104 00007f135e9a1092 00007f135e9a0fc1 00007f135e9a0f3c 00007f135e9a01f2
0925 13:56:33.754321 (+925899us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":154124160}
I20260925 13:56:33.753690 19938 consensus_queue.cc:260] T f146a9aee3054d7491e74101df0e28e1 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:33.757711 20891 leader_election.cc:290] T 37947005aaf14f27a6e5d1f30c67250c P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:33.762380 19942 consensus_queue.cc:260] T 1ce5d15e67644247b95960f444acc621 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:33.764230 19942 ts_tablet_manager.cc:1434] T 1ce5d15e67644247b95960f444acc621 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.031s user 0.006s sys 0.001s
I20260925 13:56:33.764523 19938 ts_tablet_manager.cc:1434] T f146a9aee3054d7491e74101df0e28e1 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.039s user 0.006s sys 0.000s
I20260925 13:56:33.765058 19942 tablet_bootstrap.cc:492] T 505a24ebe7014f80b3c184329cc74a9a P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:33.765879 19938 tablet_bootstrap.cc:492] T a218f033a132482a82bd1f60126fa0f9 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:33.773844 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37947005aaf14f27a6e5d1f30c67250c" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:33.775054 19773 leader_election.cc:343] T 37947005aaf14f27a6e5d1f30c67250c P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:33.707237 19629 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.193:38269, user_credentials={real_user=slave}} blocked reactor thread for 185132us
I20260925 13:56:33.776458 19629 leader_election.cc:304] T b3172d99afee4a4c878e71b645037917 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:33.778321 20856 raft_consensus.cc:2749] T b3172d99afee4a4c878e71b645037917 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260925 13:56:33.782852 19833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3172d99afee4a4c878e71b645037917" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:33.789857 19833 raft_consensus.cc:2393] T b3172d99afee4a4c878e71b645037917 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f578869bc084a5d8a9a69f143954eb3 in current term 1: Already voted for candidate f790493ba7ad403993b678f2df2b55be in this term.
I20260925 13:56:33.794615 19942 tablet_bootstrap.cc:654] T 505a24ebe7014f80b3c184329cc74a9a P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:33.796109 20644 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 241) took 4734 ms. Trace:
I20260925 13:56:33.796504 20644 rpcz_store.cc:276] 0925 13:56:29.061846 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:29.106656 (+ 44810us) service_pool.cc:224] Handling call
0925 13:56:32.968744 (+3862088us) ts_tablet_manager.cc:1151] Deleting tablet c5a9df833271480aa49feb7d9f925459
0925 13:56:32.968785 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:33.269875 (+301090us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:33.629334 (+359459us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:33.796043 (+166709us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3861429}
I20260925 13:56:33.797686 20644 tablet_replica.cc:333] T 3aaf29a2310a41d0971eded7ef51fdc9 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:33.797695 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet c5a9df833271480aa49feb7d9f925459 (table default.Table_25c2c368343741a3923a23f0a7cb2b62 [id=7d4523c5927040ba846c35a4b02172a7]) successfully deleted
I20260925 13:56:33.798496 20644 raft_consensus.cc:2243] T 3aaf29a2310a41d0971eded7ef51fdc9 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:33.799045 20644 raft_consensus.cc:2272] T 3aaf29a2310a41d0971eded7ef51fdc9 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:33.799613 20965 raft_consensus.cc:493] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bcc92728d49643efab5528d77c3d7828)
I20260925 13:56:33.800097 20965 raft_consensus.cc:515] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } }
I20260925 13:56:33.804260 19214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fc3953a2ba984a658630c423ec9b9728" candidate_term: 2 candidate_status { last_received { term: 1 index: 417 } } ignore_live_leader: false dest_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" is_pre_election: true
I20260925 13:56:33.806619 20965 leader_election.cc:290] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7dcc50a8531144a69ab0cc00df1f0df4 (127.18.182.254:40825), bcc92728d49643efab5528d77c3d7828 (127.18.182.253:39641)
W20260925 13:56:33.807682 19264 catalog_manager.cc:5136] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): alter failed for tablet 26644768f516412f8c95d68a76bb99c5 (table default.Table_bd52a86b8ff14bdbb06fcf88a60b1e82 [id=203024d67e444f839391c8670d7c7d3b]),no further retry: Not found: Tablet not found: 26644768f516412f8c95d68a76bb99c5
I20260925 13:56:33.807719 19290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fc3953a2ba984a658630c423ec9b9728" candidate_term: 2 candidate_status { last_received { term: 1 index: 417 } } ignore_live_leader: false dest_uuid: "bcc92728d49643efab5528d77c3d7828" is_pre_election: true
W20260925 13:56:33.808498 19264 catalog_manager.cc:5136] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): alter failed for tablet 26644768f516412f8c95d68a76bb99c5 (table default.Table_bd52a86b8ff14bdbb06fcf88a60b1e82 [id=203024d67e444f839391c8670d7c7d3b]),no further retry: Not found: Tablet not found: 26644768f516412f8c95d68a76bb99c5
I20260925 13:56:33.809465 20697 leader_election.cc:304] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc3953a2ba984a658630c423ec9b9728; no voters: 7dcc50a8531144a69ab0cc00df1f0df4, bcc92728d49643efab5528d77c3d7828
W20260925 13:56:33.791177 20715 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.194:33741 (request call id 9) took 848 ms (client timeout 1000 ms). Trace:
W20260925 13:56:33.820055 20715 rpcz_store.cc:269] 0925 13:56:32.943056 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:33.017031 (+ 73975us) service_pool.cc:224] Handling call
0925 13:56:33.787796 (+770765us) spinlock_profiling.cc:243] Waited 764 ms on lock 0x7b5400040454. stack: 00007fa24768b323 00007fa24768b220 00007fa24722f574 0000000000516964 00000000005168f9 00007fa24e7895c0 00007fa24e8286fa 00007fa250599d8c 00007fa25069801e 00007fa24e23cfb4 00007fa24e23cf42 00007fa24e23ce71 00007fa24e23cdec 00007fa24e23c0a2 00007fa24c4ee09c 00007fa24c4ed4d6
0925 13:56:33.791093 (+ 3297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1757668608}
I20260925 13:56:33.821225 20965 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:56:33.821960 19942 tablet_bootstrap.cc:492] T 505a24ebe7014f80b3c184329cc74a9a P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:33.822350 19942 ts_tablet_manager.cc:1403] T 505a24ebe7014f80b3c184329cc74a9a P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.057s user 0.008s sys 0.000s
I20260925 13:56:33.830497 20927 raft_consensus.cc:493] T b23948807b2c4df9bf9a4da5f767e3b5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:33.830844 20856 raft_consensus.cc:493] T 82c1002d2bcb4b11839e2f4a93b899ad P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:33.830969 20927 raft_consensus.cc:515] T b23948807b2c4df9bf9a4da5f767e3b5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:33.831267 20856 raft_consensus.cc:515] T 82c1002d2bcb4b11839e2f4a93b899ad P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:33.832806 20927 leader_election.cc:290] T b23948807b2c4df9bf9a4da5f767e3b5 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:33.833112 20856 leader_election.cc:290] T 82c1002d2bcb4b11839e2f4a93b899ad P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
W20260925 13:56:33.834914 19629 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.193:38269, user_credentials={real_user=slave}} blocked reactor thread for 53262.6us
I20260925 13:56:33.835539 19629 leader_election.cc:304] T 4d01a791445149e7bee9493e1eec5c6b P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
W20260925 13:56:33.836061 19629 raft_consensus.cc:2686] T 4d01a791445149e7bee9493e1eec5c6b P 7c1b52895a14482fb54a6758f5054f8f: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260925 13:56:33.839794 20761 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35262 (request call id 1) took 1152 ms. Trace:
I20260925 13:56:33.834471 19830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82c1002d2bcb4b11839e2f4a93b899ad" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:33.841288 19830 raft_consensus.cc:2468] T 82c1002d2bcb4b11839e2f4a93b899ad P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:33.842103 20644 ts_tablet_manager.cc:1916] T 3aaf29a2310a41d0971eded7ef51fdc9 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:33.842852 19631 leader_election.cc:304] T 82c1002d2bcb4b11839e2f4a93b899ad P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:33.835829 19833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23948807b2c4df9bf9a4da5f767e3b5" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:33.844486 19833 raft_consensus.cc:2468] T b23948807b2c4df9bf9a4da5f767e3b5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
W20260925 13:56:33.751080 19488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.194:38641, user_credentials={real_user=slave}} blocked reactor thread for 353653us
I20260925 13:56:33.783959 20794 consensus_queue.cc:1048] T f3b46e4fac694084b06b7086bcc596b9 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:33.787969 19490 leader_election.cc:304] T b23948807b2c4df9bf9a4da5f767e3b5 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:33.845566 20959 raft_consensus.cc:2804] T b23948807b2c4df9bf9a4da5f767e3b5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:33.845890 20959 raft_consensus.cc:493] T b23948807b2c4df9bf9a4da5f767e3b5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:33.846197 19488 leader_election.cc:304] T 7e544c93488541da9ebb588a465b27b7 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3; no voters:
I20260925 13:56:33.846467 19938 tablet_bootstrap.cc:654] T a218f033a132482a82bd1f60126fa0f9 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
W20260925 13:56:33.846481 19490 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 58606.3us
I20260925 13:56:33.846220 20959 raft_consensus.cc:3060] T b23948807b2c4df9bf9a4da5f767e3b5 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:33.849877 21006 tablet.cc:1751] T e3cf9676160c4af08f1f62ae952b4c57 P 8f578869bc084a5d8a9a69f143954eb3: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:33.854285 19836 raft_consensus.cc:2468] T e9dbcd85dc8647c68c120ba3fcfe1790 P f790493ba7ad403993b678f2df2b55be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 2.
I20260925 13:56:33.855535 20794 raft_consensus.cc:2804] T 7e544c93488541da9ebb588a465b27b7 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:33.855904 20794 raft_consensus.cc:493] T 7e544c93488541da9ebb588a465b27b7 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:33.856227 20794 raft_consensus.cc:3060] T 7e544c93488541da9ebb588a465b27b7 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:33.859692 20991 consensus_queue.cc:1048] T e9dbcd85dc8647c68c120ba3fcfe1790 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 13:56:33.860105 20868 consensus_queue.cc:1048] T eff33a56361946edb556ac611d2299eb P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:33.861845 19488 leader_election.cc:304] T 78d8c01c88d24deeae21d647e7f205ba P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3; no voters: f790493ba7ad403993b678f2df2b55be
W20260925 13:56:33.862196 19490 leader_election.cc:343] T 0e8aa12e95dc434a98a426499593d9a5 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 0e8aa12e95dc434a98a426499593d9a5
I20260925 13:56:33.862964 20868 raft_consensus.cc:2804] T 78d8c01c88d24deeae21d647e7f205ba P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260925 13:56:33.863061 19488 leader_election.cc:343] T 0e8aa12e95dc434a98a426499593d9a5 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Not found: Tablet not found: 0e8aa12e95dc434a98a426499593d9a5
I20260925 13:56:33.863278 20868 raft_consensus.cc:493] T 78d8c01c88d24deeae21d647e7f205ba P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:33.863435 19488 leader_election.cc:304] T 0e8aa12e95dc434a98a426499593d9a5 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3; no voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:33.863616 20868 raft_consensus.cc:3060] T 78d8c01c88d24deeae21d647e7f205ba P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
W20260925 13:56:33.863976 19488 raft_consensus.cc:2686] T 0e8aa12e95dc434a98a426499593d9a5 P 8f578869bc084a5d8a9a69f143954eb3: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260925 13:56:33.871932 19693 raft_consensus.cc:1240] T e9dbcd85dc8647c68c120ba3fcfe1790 P 7c1b52895a14482fb54a6758f5054f8f [term 2 FOLLOWER]: Rejecting Update request from peer 8f578869bc084a5d8a9a69f143954eb3 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260925 13:56:33.871222 20868 raft_consensus.cc:515] T 78d8c01c88d24deeae21d647e7f205ba P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:33.874643 20868 leader_election.cc:290] T 78d8c01c88d24deeae21d647e7f205ba P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:33.875481 20955 raft_consensus.cc:493] T 1ce5d15e67644247b95960f444acc621 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:33.875861 20955 raft_consensus.cc:515] T 1ce5d15e67644247b95960f444acc621 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:33.877619 20955 leader_election.cc:290] T 1ce5d15e67644247b95960f444acc621 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:33.877760 20991 consensus_queue.cc:1048] T eff33a56361946edb556ac611d2299eb P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:33.878937 20959 raft_consensus.cc:515] T b23948807b2c4df9bf9a4da5f767e3b5 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:33.880909 20959 leader_election.cc:290] T b23948807b2c4df9bf9a4da5f767e3b5 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:33.806322 19939 tablet_bootstrap.cc:492] T 11077cdb735840b5921e22f588059149 P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:33.881605 19939 ts_tablet_manager.cc:1403] T 11077cdb735840b5921e22f588059149 P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.386s user 0.020s sys 0.002s
I20260925 13:56:33.830228 20956 raft_consensus.cc:493] T 82c1002d2bcb4b11839e2f4a93b899ad P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:33.882220 19938 tablet_bootstrap.cc:492] T a218f033a132482a82bd1f60126fa0f9 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:33.882054 20956 raft_consensus.cc:515] T 82c1002d2bcb4b11839e2f4a93b899ad P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:33.882687 19938 ts_tablet_manager.cc:1403] T a218f033a132482a82bd1f60126fa0f9 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.117s user 0.010s sys 0.000s
I20260925 13:56:33.883967 20956 leader_election.cc:290] T 82c1002d2bcb4b11839e2f4a93b899ad P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:33.875195 20794 raft_consensus.cc:515] T 7e544c93488541da9ebb588a465b27b7 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:33.884974 19938 raft_consensus.cc:359] T a218f033a132482a82bd1f60126fa0f9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:33.886045 20794 leader_election.cc:290] T 7e544c93488541da9ebb588a465b27b7 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
W20260925 13:56:33.887977 19629 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.193:38269, user_credentials={real_user=slave}} blocked reactor thread for 50109.3us
I20260925 13:56:33.888245 19942 raft_consensus.cc:359] T 505a24ebe7014f80b3c184329cc74a9a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:33.888815 19942 raft_consensus.cc:385] T 505a24ebe7014f80b3c184329cc74a9a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:33.889062 19942 raft_consensus.cc:740] T 505a24ebe7014f80b3c184329cc74a9a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:33.889570 19942 consensus_queue.cc:260] T 505a24ebe7014f80b3c184329cc74a9a P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
W20260925 13:56:33.892026 19631 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 49243.9us
I20260925 13:56:33.893054 20927 raft_consensus.cc:2804] T 82c1002d2bcb4b11839e2f4a93b899ad P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:33.893339 20927 raft_consensus.cc:493] T 82c1002d2bcb4b11839e2f4a93b899ad P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:33.893604 20927 raft_consensus.cc:3060] T 82c1002d2bcb4b11839e2f4a93b899ad P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:33.896428 19631 leader_election.cc:304] T b23948807b2c4df9bf9a4da5f767e3b5 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:33.896485 19689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce5d15e67644247b95960f444acc621" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:33.897055 19689 raft_consensus.cc:2468] T 1ce5d15e67644247b95960f444acc621 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:33.897194 19631 leader_election.cc:304] T e9dbcd85dc8647c68c120ba3fcfe1790 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:33.897557 20988 raft_consensus.cc:2804] T b23948807b2c4df9bf9a4da5f767e3b5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:33.896504 19694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3172d99afee4a4c878e71b645037917" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:33.898550 19694 raft_consensus.cc:2393] T b3172d99afee4a4c878e71b645037917 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f578869bc084a5d8a9a69f143954eb3 in current term 1: Already voted for candidate 7c1b52895a14482fb54a6758f5054f8f in this term.
I20260925 13:56:33.899159 20856 raft_consensus.cc:2804] T e9dbcd85dc8647c68c120ba3fcfe1790 P 7c1b52895a14482fb54a6758f5054f8f [term 2 FOLLOWER]: Leader election won for term 2
I20260925 13:56:33.899729 20856 raft_consensus.cc:697] T e9dbcd85dc8647c68c120ba3fcfe1790 P 7c1b52895a14482fb54a6758f5054f8f [term 2 LEADER]: Becoming Leader. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Running, Role: LEADER
I20260925 13:56:33.900693 20856 consensus_queue.cc:237] T e9dbcd85dc8647c68c120ba3fcfe1790 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:33.897886 20988 raft_consensus.cc:493] T b23948807b2c4df9bf9a4da5f767e3b5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:33.904785 20988 raft_consensus.cc:3060] T b23948807b2c4df9bf9a4da5f767e3b5 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:33.906486 19938 raft_consensus.cc:385] T a218f033a132482a82bd1f60126fa0f9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:33.907877 19942 ts_tablet_manager.cc:1434] T 505a24ebe7014f80b3c184329cc74a9a P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.085s user 0.007s sys 0.000s
I20260925 13:56:33.908784 19942 tablet_bootstrap.cc:492] T 72183473a5c94629a58c2cd8e1c56f32 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:33.909103 19938 raft_consensus.cc:740] T a218f033a132482a82bd1f60126fa0f9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:33.910132 19938 consensus_queue.cc:260] T a218f033a132482a82bd1f60126fa0f9 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:33.911846 19693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d8c01c88d24deeae21d647e7f205ba" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:33.912668 19693 raft_consensus.cc:2393] T 78d8c01c88d24deeae21d647e7f205ba P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f578869bc084a5d8a9a69f143954eb3 in current term 1: Already voted for candidate 7c1b52895a14482fb54a6758f5054f8f in this term.
I20260925 13:56:33.914283 19942 tablet_bootstrap.cc:654] T 72183473a5c94629a58c2cd8e1c56f32 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:33.907199 20927 raft_consensus.cc:515] T 82c1002d2bcb4b11839e2f4a93b899ad P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:33.920650 19833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82c1002d2bcb4b11839e2f4a93b899ad" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:33.921166 19833 raft_consensus.cc:3060] T 82c1002d2bcb4b11839e2f4a93b899ad P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:33.928252 19271 master_service.cc:960] ReplaceTablet: received request to replace tablet eff33a56361946edb556ac611d2299eb from {username='slave'} at 127.0.0.1:39288
I20260925 13:56:33.928339 19833 raft_consensus.cc:2468] T 82c1002d2bcb4b11839e2f4a93b899ad P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
W20260925 13:56:33.929101 19490 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 51500.1us
I20260925 13:56:33.929497 19631 leader_election.cc:304] T 82c1002d2bcb4b11839e2f4a93b899ad P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:33.932474 19938 ts_tablet_manager.cc:1434] T a218f033a132482a82bd1f60126fa0f9 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.049s user 0.005s sys 0.000s
I20260925 13:56:33.938596 20993 raft_consensus.cc:2804] T 82c1002d2bcb4b11839e2f4a93b899ad P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:33.891764 19939 raft_consensus.cc:359] T 11077cdb735840b5921e22f588059149 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:33.942204 19939 raft_consensus.cc:385] T 11077cdb735840b5921e22f588059149 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:33.942476 19939 raft_consensus.cc:740] T 11077cdb735840b5921e22f588059149 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
W20260925 13:56:33.938985 19488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.194:38641, user_credentials={real_user=slave}} blocked reactor thread for 60161.6us
I20260925 13:56:33.944602 19939 consensus_queue.cc:260] T 11077cdb735840b5921e22f588059149 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:33.953015 20993 raft_consensus.cc:697] T 82c1002d2bcb4b11839e2f4a93b899ad P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Becoming Leader. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Running, Role: LEADER
W20260925 13:56:33.946159 21003 log.cc:927] Time spent T eff33a56361946edb556ac611d2299eb P 8f578869bc084a5d8a9a69f143954eb3: Append to log took a long time: real 0.127s user 0.001s sys 0.000s
W20260925 13:56:33.952853 19773 leader_election.cc:343] T c5a9df833271480aa49feb7d9f925459 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Not found: Tablet not found: c5a9df833271480aa49feb7d9f925459
I20260925 13:56:33.957428 19488 leader_election.cc:304] T 1ce5d15e67644247b95960f444acc621 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3; no voters:
I20260925 13:56:33.958001 19773 leader_election.cc:304] T b3172d99afee4a4c878e71b645037917 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:33.958352 19488 leader_election.cc:304] T b3172d99afee4a4c878e71b645037917 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3; no voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:33.959264 20891 raft_consensus.cc:2749] T b3172d99afee4a4c878e71b645037917 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260925 13:56:33.959663 19955 ts_tablet_manager.cc:1929] T 0e8aa12e95dc434a98a426499593d9a5 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:33.960003 19955 log.cc:1199] T 0e8aa12e95dc434a98a426499593d9a5 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/0e8aa12e95dc434a98a426499593d9a5
I20260925 13:56:33.960597 19955 ts_tablet_manager.cc:1950] T 0e8aa12e95dc434a98a426499593d9a5 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:33.961294 19938 tablet_bootstrap.cc:492] T 45ab2817d7e84185b7ec2bfddcf4997f P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:33.951249 19939 ts_tablet_manager.cc:1434] T 11077cdb735840b5921e22f588059149 P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.065s user 0.005s sys 0.000s
I20260925 13:56:33.962193 20959 consensus_queue.cc:1059] T e9dbcd85dc8647c68c120ba3fcfe1790 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:33.962766 19939 tablet_bootstrap.cc:492] T 37947005aaf14f27a6e5d1f30c67250c P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
I20260925 13:56:33.962971 20794 raft_consensus.cc:2804] T 1ce5d15e67644247b95960f444acc621 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:33.963420 20794 raft_consensus.cc:493] T 1ce5d15e67644247b95960f444acc621 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:33.963914 20794 raft_consensus.cc:3060] T 1ce5d15e67644247b95960f444acc621 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:33.963719 20959 raft_consensus.cc:3055] T e9dbcd85dc8647c68c120ba3fcfe1790 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Stepping down as leader of term 1
I20260925 13:56:33.967607 20955 raft_consensus.cc:2749] T b3172d99afee4a4c878e71b645037917 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260925 13:56:33.967689 20959 raft_consensus.cc:740] T e9dbcd85dc8647c68c120ba3fcfe1790 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Running, Role: LEADER
I20260925 13:56:33.969893 20927 leader_election.cc:290] T 82c1002d2bcb4b11839e2f4a93b899ad P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:33.970964 19830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce5d15e67644247b95960f444acc621" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:33.976512 19830 raft_consensus.cc:2468] T 1ce5d15e67644247b95960f444acc621 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:33.971395 19840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23948807b2c4df9bf9a4da5f767e3b5" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:33.977721 19840 raft_consensus.cc:3060] T b23948807b2c4df9bf9a4da5f767e3b5 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:33.980821 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23948807b2c4df9bf9a4da5f767e3b5" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:33.982985 19694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e544c93488541da9ebb588a465b27b7" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:33.982915 19693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82c1002d2bcb4b11839e2f4a93b899ad" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:33.983683 19694 raft_consensus.cc:2393] T 7e544c93488541da9ebb588a465b27b7 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f578869bc084a5d8a9a69f143954eb3 in current term 1: Already voted for candidate f790493ba7ad403993b678f2df2b55be in this term.
I20260925 13:56:33.970438 20988 raft_consensus.cc:515] T b23948807b2c4df9bf9a4da5f767e3b5 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:33.986181 20988 leader_election.cc:290] T b23948807b2c4df9bf9a4da5f767e3b5 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:33.986871 19695 raft_consensus.cc:2393] T b23948807b2c4df9bf9a4da5f767e3b5 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f578869bc084a5d8a9a69f143954eb3 in current term 1: Already voted for candidate 7c1b52895a14482fb54a6758f5054f8f in this term.
I20260925 13:56:33.972648 19833 raft_consensus.cc:1240] T e9dbcd85dc8647c68c120ba3fcfe1790 P f790493ba7ad403993b678f2df2b55be [term 2 FOLLOWER]: Rejecting Update request from peer 8f578869bc084a5d8a9a69f143954eb3 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260925 13:56:33.988893 19840 raft_consensus.cc:2468] T b23948807b2c4df9bf9a4da5f767e3b5 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:33.989579 20993 consensus_queue.cc:237] T 82c1002d2bcb4b11839e2f4a93b899ad P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
W20260925 13:56:33.990317 19265 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 40e326118fad4811b1d3ebd7f632ca92 (table default.Table_ddc133aff70b4778a2e87490f469da02 [id=7e353b31f2c04633b159015cb8a64d41], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 40e326118fad4811b1d3ebd7f632ca92 has no consensus state
I20260925 13:56:33.991837 19833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23948807b2c4df9bf9a4da5f767e3b5" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:33.992683 19833 raft_consensus.cc:2393] T b23948807b2c4df9bf9a4da5f767e3b5 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7c1b52895a14482fb54a6758f5054f8f in current term 1: Already voted for candidate 8f578869bc084a5d8a9a69f143954eb3 in this term.
I20260925 13:56:33.840096 20761 rpcz_store.cc:276] 0925 13:56:32.687855 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:32.730372 (+ 42517us) service_pool.cc:224] Handling call
0925 13:56:32.761819 (+ 31447us) raft_consensus.cc:1471] Updating replica for 59 ops
0925 13:56:32.871012 (+109193us) raft_consensus.cc:1534] Early marking committed up to index 0
0925 13:56:32.871042 (+ 30us) raft_consensus.cc:1539] Triggering prepare for 59 ops
0925 13:56:32.871339 (+ 297us) write_op.cc:270] Start()
0925 13:56:33.005580 (+134241us) write_op.cc:276] Timestamp: P: 1790344589033828 usec, L: 0
0925 13:56:33.006149 (+ 569us) write_op.cc:270] Start()
0925 13:56:33.006222 (+ 73us) write_op.cc:276] Timestamp: P: 1790344589149421 usec, L: 0
0925 13:56:33.006625 (+ 403us) write_op.cc:270] Start()
0925 13:56:33.006685 (+ 60us) write_op.cc:276] Timestamp: P: 1790344589152923 usec, L: 0
0925 13:56:33.007156 (+ 471us) write_op.cc:270] Start()
0925 13:56:33.007227 (+ 71us) write_op.cc:276] Timestamp: P: 1790344589351412 usec, L: 0
0925 13:56:33.048343 (+ 41116us) write_op.cc:270] Start()
0925 13:56:33.048426 (+ 83us) write_op.cc:276] Timestamp: P: 1790344589354843 usec, L: 0
0925 13:56:33.048887 (+ 461us) write_op.cc:270] Start()
0925 13:56:33.048954 (+ 67us) write_op.cc:276] Timestamp: P: 1790344589452740 usec, L: 0
0925 13:56:33.049328 (+ 374us) write_op.cc:270] Start()
0925 13:56:33.049445 (+ 117us) write_op.cc:276] Timestamp: P: 1790344589527192 usec, L: 0
0925 13:56:33.049813 (+ 368us) write_op.cc:270] Start()
0925 13:56:33.049878 (+ 65us) write_op.cc:276] Timestamp: P: 1790344589627336 usec, L: 0
0925 13:56:33.050252 (+ 374us) write_op.cc:270] Start()
0925 13:56:33.050312 (+ 60us) write_op.cc:276] Timestamp: P: 1790344589760551 usec, L: 0
0925 13:56:33.050661 (+ 349us) write_op.cc:270] Start()
0925 13:56:33.050720 (+ 59us) write_op.cc:276] Timestamp: P: 1790344589953301 usec, L: 0
0925 13:56:33.051055 (+ 335us) write_op.cc:270] Start()
0925 13:56:33.051128 (+ 73us) write_op.cc:276] Timestamp: P: 1790344589970769 usec, L: 0
0925 13:56:33.241606 (+190478us) write_op.cc:270] Start()
0925 13:56:33.241805 (+ 199us) write_op.cc:276] Timestamp: P: 1790344590164181 usec, L: 0
0925 13:56:33.242831 (+ 1026us) write_op.cc:270] Start()
0925 13:56:33.243021 (+ 190us) write_op.cc:276] Timestamp: P: 1790344590298247 usec, L: 0
0925 13:56:33.244070 (+ 1049us) write_op.cc:270] Start()
0925 13:56:33.244273 (+ 203us) write_op.cc:276] Timestamp: P: 1790344590392976 usec, L: 0
0925 13:56:33.246114 (+ 1841us) write_op.cc:270] Start()
0925 13:56:33.246352 (+ 238us) write_op.cc:276] Timestamp: P: 1790344590500551 usec, L: 0
0925 13:56:33.335935 (+ 89583us) write_op.cc:270] Start()
0925 13:56:33.336124 (+ 189us) write_op.cc:276] Timestamp: P: 1790344590502516 usec, L: 0
0925 13:56:33.336993 (+ 869us) write_op.cc:270] Start()
0925 13:56:33.337062 (+ 69us) write_op.cc:276] Timestamp: P: 1790344590504316 usec, L: 0
0925 13:56:33.337501 (+ 439us) write_op.cc:270] Start()
0925 13:56:33.337562 (+ 61us) write_op.cc:276] Timestamp: P: 1790344590506832 usec, L: 0
0925 13:56:33.337926 (+ 364us) write_op.cc:270] Start()
0925 13:56:33.338011 (+ 85us) write_op.cc:276] Timestamp: P: 1790344590600238 usec, L: 0
0925 13:56:33.338385 (+ 374us) write_op.cc:270] Start()
0925 13:56:33.338445 (+ 60us) write_op.cc:276] Timestamp: P: 1790344590759771 usec, L: 0
0925 13:56:33.338794 (+ 349us) write_op.cc:270] Start()
0925 13:56:33.338851 (+ 57us) write_op.cc:276] Timestamp: P: 1790344590788519 usec, L: 0
0925 13:56:33.339214 (+ 363us) write_op.cc:270] Start()
0925 13:56:33.339278 (+ 64us) write_op.cc:276] Timestamp: P: 1790344590812163 usec, L: 0
0925 13:56:33.414112 (+ 74834us) write_op.cc:270] Start()
0925 13:56:33.414202 (+ 90us) write_op.cc:276] Timestamp: P: 1790344590816537 usec, L: 0
0925 13:56:33.420886 (+ 6684us) write_op.cc:270] Start()
0925 13:56:33.420986 (+ 100us) write_op.cc:276] Timestamp: P: 1790344590818314 usec, L: 0
0925 13:56:33.422010 (+ 1024us) write_op.cc:270] Start()
0925 13:56:33.422463 (+ 453us) write_op.cc:276] Timestamp: P: 1790344590848892 usec, L: 0
0925 13:56:33.422972 (+ 509us) write_op.cc:270] Start()
0925 13:56:33.423050 (+ 78us) write_op.cc:276] Timestamp: P: 1790344590868524 usec, L: 0
0925 13:56:33.476388 (+ 53338us) write_op.cc:270] Start()
0925 13:56:33.476479 (+ 91us) write_op.cc:276] Timestamp: P: 1790344591224093 usec, L: 0
0925 13:56:33.476893 (+ 414us) write_op.cc:270] Start()
0925 13:56:33.476958 (+ 65us) write_op.cc:276] Timestamp: P: 1790344591282036 usec, L: 0
0925 13:56:33.477322 (+ 364us) write_op.cc:270] Start()
0925 13:56:33.477391 (+ 69us) write_op.cc:276] Timestamp: P: 1790344591422658 usec, L: 0
0925 13:56:33.477754 (+ 363us) write_op.cc:270] Start()
0925 13:56:33.477813 (+ 59us) write_op.cc:276] Timestamp: P: 1790344591470009 usec, L: 0
0925 13:56:33.478385 (+ 572us) write_op.cc:270] Start()
0925 13:56:33.478452 (+ 67us) write_op.cc:276] Timestamp: P: 1790344591584649 usec, L: 0
0925 13:56:33.478873 (+ 421us) write_op.cc:270] Start()
0925 13:56:33.478932 (+ 59us) write_op.cc:276] Timestamp: P: 1790344591587176 usec, L: 0
0925 13:56:33.479297 (+ 365us) write_op.cc:270] Start()
0925 13:56:33.479375 (+ 78us) write_op.cc:276] Timestamp: P: 1790344591590169 usec, L: 0
0925 13:56:33.479733 (+ 358us) write_op.cc:270] Start()
0925 13:56:33.479797 (+ 64us) write_op.cc:276] Timestamp: P: 1790344591655809 usec, L: 0
0925 13:56:33.480144 (+ 347us) write_op.cc:270] Start()
0925 13:56:33.480203 (+ 59us) write_op.cc:276] Timestamp: P: 1790344591682120 usec, L: 0
0925 13:56:33.480576 (+ 373us) write_op.cc:270] Start()
0925 13:56:33.480636 (+ 60us) write_op.cc:276] Timestamp: P: 1790344591730846 usec, L: 0
0925 13:56:33.480979 (+ 343us) write_op.cc:270] Start()
0925 13:56:33.481047 (+ 68us) write_op.cc:276] Timestamp: P: 1790344591734302 usec, L: 0
0925 13:56:33.536410 (+ 55363us) write_op.cc:270] Start()
0925 13:56:33.536507 (+ 97us) write_op.cc:276] Timestamp: P: 1790344591739660 usec, L: 0
0925 13:56:33.537027 (+ 520us) write_op.cc:270] Start()
0925 13:56:33.537100 (+ 73us) write_op.cc:276] Timestamp: P: 1790344591807920 usec, L: 0
0925 13:56:33.537526 (+ 426us) write_op.cc:270] Start()
0925 13:56:33.537602 (+ 76us) write_op.cc:276] Timestamp: P: 1790344591828189 usec, L: 0
0925 13:56:33.538003 (+ 401us) write_op.cc:270] Start()
0925 13:56:33.538066 (+ 63us) write_op.cc:276] Timestamp: P: 1790344591920460 usec, L: 0
0925 13:56:33.538445 (+ 379us) write_op.cc:270] Start()
0925 13:56:33.538508 (+ 63us) write_op.cc:276] Timestamp: P: 1790344591994812 usec, L: 0
0925 13:56:33.538914 (+ 406us) write_op.cc:270] Start()
0925 13:56:33.538980 (+ 66us) write_op.cc:276] Timestamp: P: 1790344592003010 usec, L: 0
0925 13:56:33.631566 (+ 92586us) write_op.cc:270] Start()
0925 13:56:33.631667 (+ 101us) write_op.cc:276] Timestamp: P: 1790344592026056 usec, L: 0
0925 13:56:33.632284 (+ 617us) write_op.cc:270] Start()
0925 13:56:33.632356 (+ 72us) write_op.cc:276] Timestamp: P: 1790344592027974 usec, L: 0
0925 13:56:33.632774 (+ 418us) write_op.cc:270] Start()
0925 13:56:33.632847 (+ 73us) write_op.cc:276] Timestamp: P: 1790344592097755 usec, L: 0
0925 13:56:33.633349 (+ 502us) write_op.cc:270] Start()
0925 13:56:33.633416 (+ 67us) write_op.cc:276] Timestamp: P: 1790344592118876 usec, L: 0
0925 13:56:33.633807 (+ 391us) write_op.cc:270] Start()
0925 13:56:33.633871 (+ 64us) write_op.cc:276] Timestamp: P: 1790344592120561 usec, L: 0
0925 13:56:33.634272 (+ 401us) write_op.cc:270] Start()
0925 13:56:33.634336 (+ 64us) write_op.cc:276] Timestamp: P: 1790344592253936 usec, L: 0
0925 13:56:33.634775 (+ 439us) write_op.cc:270] Start()
0925 13:56:33.634836 (+ 61us) write_op.cc:276] Timestamp: P: 1790344592258400 usec, L: 0
0925 13:56:33.635260 (+ 424us) write_op.cc:270] Start()
0925 13:56:33.635321 (+ 61us) write_op.cc:276] Timestamp: P: 1790344592260764 usec, L: 0
0925 13:56:33.671647 (+ 36326us) write_op.cc:270] Start()
0925 13:56:33.671716 (+ 69us) write_op.cc:276] Timestamp: P: 1790344592320047 usec, L: 0
0925 13:56:33.672145 (+ 429us) write_op.cc:270] Start()
0925 13:56:33.672207 (+ 62us) write_op.cc:276] Timestamp: P: 1790344592363247 usec, L: 0
0925 13:56:33.672577 (+ 370us) write_op.cc:270] Start()
0925 13:56:33.672639 (+ 62us) write_op.cc:276] Timestamp: P: 1790344592484683 usec, L: 0
0925 13:56:33.673022 (+ 383us) write_op.cc:270] Start()
0925 13:56:33.673089 (+ 67us) write_op.cc:276] Timestamp: P: 1790344592543766 usec, L: 0
0925 13:56:33.673487 (+ 398us) write_op.cc:270] Start()
0925 13:56:33.673556 (+ 69us) write_op.cc:276] Timestamp: P: 1790344592594104 usec, L: 0
0925 13:56:33.673938 (+ 382us) write_op.cc:270] Start()
0925 13:56:33.674001 (+ 63us) write_op.cc:276] Timestamp: P: 1790344592598007 usec, L: 0
0925 13:56:33.674409 (+ 408us) write_op.cc:270] Start()
0925 13:56:33.674473 (+ 64us) write_op.cc:276] Timestamp: P: 1790344592600695 usec, L: 0
0925 13:56:33.674844 (+ 371us) write_op.cc:270] Start()
0925 13:56:33.674907 (+ 63us) write_op.cc:276] Timestamp: P: 1790344592620938 usec, L: 0
0925 13:56:33.730779 (+ 55872us) log.cc:844] Serialized 36335 byte log entry
0925 13:56:33.751869 (+ 21090us) raft_consensus.cc:1645] Marking committed up to 417
0925 13:56:33.781802 (+ 29933us) raft_consensus.cc:1697] Filling consensus response to leader.
0925 13:56:33.839393 (+ 57591us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0925 13:56:33.839433 (+ 40us) raft_consensus.cc:1685] finished
0925 13:56:33.839444 (+ 11us) raft_consensus.cc:1691] UpdateReplicas() finished
0925 13:56:33.839742 (+ 298us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:33.752062 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.752186 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.752297 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.752429 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.755566 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.755676 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.755762 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.755848 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.755929 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.756029 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.756109 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.756190 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.756272 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.756347 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.756440 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.756524 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.756610 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.756691 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.756781 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.756865 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.756954 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.757033 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.757115 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.757209 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.757293 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.757370 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.757459 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.757533 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.757613 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.757696 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.757781 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.757869 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.757952 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.758061 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.758170 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.758279 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.758405 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.758520 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.758627 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.758747 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.758856 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.758960 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.759037 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.759110 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.780287 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.780388 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.780503 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.780623 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.780729 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.780830 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.780932 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.781034 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.781136 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.781242 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.781325 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.781429 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.781532 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.781644 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
0925 13:56:33.781743 (+ 0us) op_driver.cc:464] REPLICATION: finished
Metrics: {"child_traces":[["op",{"replication_time_us":880636}],["op",{"replication_time_us":745983}],["op",{"replication_time_us":745621}],["op",{"replication_time_us":745200}],["op",{"replication_time_us":704148}],["op",{"replication_time_us":706747}],["op",{"replication_time_us":706398}],["op",{"replication_time_us":705994}],["op",{"replication_time_us":705643}],["op",{"replication_time_us":705316}],["op",{"replication_time_us":705018}],["op",{"replication_time_us":514550}],["op",{"replication_time_us":513410}],["op",{"replication_time_us":512246}],["op",{"replication_time_us":510343}],["op",{"replication_time_us":420558}],["op",{"replication_time_us":419582}],["op",{"replication_time_us":419154}],["op",{"replication_time_us":418817}],["op",{"replication_time_us":418440}],["op",{"replication_time_us":418119}],["op",{"replication_time_us":417784}],["op",{"replication_time_us":342969}],["op",{"replication_time_us":336284}],["op",{"replication_time_us":335363}],["op",{"replication_time_us":334364}],["op",{"replication_time_us":281039}],["op",{"replication_time_us":280608}],["op",{"replication_time_us":280255}],["op",{"replication_time_us":279903}],["op",{"replication_time_us":279361}],["op",{"replication_time_us":278959}],["op",{"replication_time_us":278618}],["op",{"replication_time_us":278276}],["op",{"replication_time_us":277976}],["op",{"replication_time_us":277664}],["op",{"replication_time_us":277363}],["op",{"replication_time_us":222061}],["op",{"replication_time_us":221552}],["op",{"replication_time_us":221157}],["op",{"replication_time_us":220806}],["op",{"replication_time_us":220465}],["op",{"replication_time_us":220090}],["op",{"replication_time_us":127523}],["op",{"replication_time_us":147943}],["op",{"replication_time_us":147578}],["op",{"replication_time_us":147105}],["op",{"replication_time_us":146767}],["op",{"replication_time_us":146411}],["op",{"replication_time_us":146010}],["op",{"replication_time_us":145627}],["op",{"replication_time_us":109340}],["op",{"replication_time_us":108943}],["op",{"replication_time_us":108618}],["op",{"replication_time_us":108269}],["op",{"replication_time_us":107888}],["op",{"replication_time_us":107547}],["op",{"replication_time_us":107188}],["op",{"replication_time_us":106854}]]}
I20260925 13:56:34.015694 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9dbcd85dc8647c68c120ba3fcfe1790" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:34.009110 19833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e544c93488541da9ebb588a465b27b7" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:34.019692 19554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e544c93488541da9ebb588a465b27b7" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:34.020373 19554 raft_consensus.cc:2393] T 7e544c93488541da9ebb588a465b27b7 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f790493ba7ad403993b678f2df2b55be in current term 1: Already voted for candidate 8f578869bc084a5d8a9a69f143954eb3 in this term.
I20260925 13:56:34.019788 19551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82c1002d2bcb4b11839e2f4a93b899ad" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:34.021009 19840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82c1002d2bcb4b11839e2f4a93b899ad" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:34.021750 19840 raft_consensus.cc:2393] T 82c1002d2bcb4b11839e2f4a93b899ad P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8f578869bc084a5d8a9a69f143954eb3 in current term 1: Already voted for candidate 7c1b52895a14482fb54a6758f5054f8f in this term.
I20260925 13:56:34.016089 19938 tablet_bootstrap.cc:654] T 45ab2817d7e84185b7ec2bfddcf4997f P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:34.049926 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82c1002d2bcb4b11839e2f4a93b899ad" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:34.056563 19555 raft_consensus.cc:3060] T 82c1002d2bcb4b11839e2f4a93b899ad P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:34.052922 19549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d8c01c88d24deeae21d647e7f205ba" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:34.054105 19553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23948807b2c4df9bf9a4da5f767e3b5" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:34.070253 19549 raft_consensus.cc:2393] T 78d8c01c88d24deeae21d647e7f205ba P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7c1b52895a14482fb54a6758f5054f8f in current term 1: Already voted for candidate 8f578869bc084a5d8a9a69f143954eb3 in this term.
I20260925 13:56:34.071552 21006 tablet.cc:1751] T eff33a56361946edb556ac611d2299eb P 8f578869bc084a5d8a9a69f143954eb3: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:34.072475 19553 raft_consensus.cc:2393] T b23948807b2c4df9bf9a4da5f767e3b5 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7c1b52895a14482fb54a6758f5054f8f in current term 1: Already voted for candidate 8f578869bc084a5d8a9a69f143954eb3 in this term.
I20260925 13:56:34.084300 19942 tablet_bootstrap.cc:492] T 72183473a5c94629a58c2cd8e1c56f32 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:34.084681 19942 ts_tablet_manager.cc:1403] T 72183473a5c94629a58c2cd8e1c56f32 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.176s user 0.008s sys 0.006s
I20260925 13:56:34.089183 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 226) took 6460 ms. Trace:
I20260925 13:56:34.089605 19955 rpcz_store.cc:276] 0925 13:56:27.628740 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:27.628839 (+ 99us) service_pool.cc:224] Handling call
0925 13:56:33.434621 (+5805782us) ts_tablet_manager.cc:1151] Deleting tablet 0e8aa12e95dc434a98a426499593d9a5
0925 13:56:33.434672 (+ 51us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:33.773578 (+338906us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:33.959550 (+185972us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:34.089126 (+129576us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":280838,"tablet-delete.queue_time_us":5677329}
I20260925 13:56:34.042630 19550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37947005aaf14f27a6e5d1f30c67250c" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:34.093597 19553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b23948807b2c4df9bf9a4da5f767e3b5" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:34.094293 19553 raft_consensus.cc:2393] T b23948807b2c4df9bf9a4da5f767e3b5 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7c1b52895a14482fb54a6758f5054f8f in current term 1: Already voted for candidate 8f578869bc084a5d8a9a69f143954eb3 in this term.
I20260925 13:56:34.101689 20794 raft_consensus.cc:515] T 1ce5d15e67644247b95960f444acc621 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:34.103621 20794 leader_election.cc:290] T 1ce5d15e67644247b95960f444acc621 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Requested vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:34.109638 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet 0e8aa12e95dc434a98a426499593d9a5 (table default.Table_b42a6e2de4e74cb7a6f01dba22f3b94c [id=03497b028f664f1794f4ff6b53912124]) successfully deleted
I20260925 13:56:34.112857 19939 tablet_bootstrap.cc:654] T 37947005aaf14f27a6e5d1f30c67250c P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:34.086951 19942 raft_consensus.cc:359] T 72183473a5c94629a58c2cd8e1c56f32 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:34.115860 19942 raft_consensus.cc:385] T 72183473a5c94629a58c2cd8e1c56f32 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:34.116153 19942 raft_consensus.cc:740] T 72183473a5c94629a58c2cd8e1c56f32 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:34.118333 19629 leader_election.cc:304] T 78d8c01c88d24deeae21d647e7f205ba P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:34.119536 20993 raft_consensus.cc:2749] T 78d8c01c88d24deeae21d647e7f205ba P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260925 13:56:34.123111 21029 tablet.cc:1751] T 5f8a96ae6eec4f15a221f9aad537c094 P f790493ba7ad403993b678f2df2b55be: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:34.127183 19840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce5d15e67644247b95960f444acc621" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:34.127727 19840 raft_consensus.cc:3060] T 1ce5d15e67644247b95960f444acc621 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:34.131412 20644 ts_tablet_manager.cc:1929] T 3aaf29a2310a41d0971eded7ef51fdc9 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:56:34.131758 20644 log.cc:1199] T 3aaf29a2310a41d0971eded7ef51fdc9 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/3aaf29a2310a41d0971eded7ef51fdc9
I20260925 13:56:34.131986 19629 leader_election.cc:304] T b23948807b2c4df9bf9a4da5f767e3b5 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f; no voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:34.132452 20644 ts_tablet_manager.cc:1950] T 3aaf29a2310a41d0971eded7ef51fdc9 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:34.133502 21016 raft_consensus.cc:2749] T b23948807b2c4df9bf9a4da5f767e3b5 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260925 13:56:34.134332 20644 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 243) took 4736 ms. Trace:
I20260925 13:56:34.134632 20644 rpcz_store.cc:276] 0925 13:56:29.397461 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:29.456713 (+ 59252us) service_pool.cc:224] Handling call
0925 13:56:33.797551 (+4340838us) ts_tablet_manager.cc:1151] Deleting tablet 3aaf29a2310a41d0971eded7ef51fdc9
0925 13:56:33.797590 (+ 39us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:33.988456 (+190866us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:34.071261 (+ 82805us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:34.134281 (+ 63020us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4335393}
I20260925 13:56:34.135001 21025 tablet.cc:1751] T 5f8a96ae6eec4f15a221f9aad537c094 P 7c1b52895a14482fb54a6758f5054f8f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:34.135571 19942 consensus_queue.cc:260] T 72183473a5c94629a58c2cd8e1c56f32 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:34.141305 19942 ts_tablet_manager.cc:1434] T 72183473a5c94629a58c2cd8e1c56f32 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.056s user 0.007s sys 0.000s
I20260925 13:56:34.142139 19942 tablet_bootstrap.cc:492] T 31b5f47966a74ec3b59c7c20479ba702 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:34.145006 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 235) took 6158 ms. Trace:
I20260925 13:56:34.146221 19955 rpcz_store.cc:276] 0925 13:56:27.986776 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:27.987282 (+ 506us) service_pool.cc:224] Handling call
0925 13:56:34.144720 (+6157438us) ts_tablet_manager.cc:1151] Deleting tablet 1513c0048bd244028d3dfa8d118bc14e
0925 13:56:34.144776 (+ 56us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:34.144952 (+ 176us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6156670}
W20260925 13:56:34.147442 19264 catalog_manager.cc:5030] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): delete failed for tablet 1513c0048bd244028d3dfa8d118bc14e because the tablet was not found. No further retry: Not found: Tablet not found: 1513c0048bd244028d3dfa8d118bc14e
W20260925 13:56:34.147920 19490 leader_election.cc:343] T 78d8c01c88d24deeae21d647e7f205ba P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 78d8c01c88d24deeae21d647e7f205ba
I20260925 13:56:34.107630 19555 raft_consensus.cc:2468] T 82c1002d2bcb4b11839e2f4a93b899ad P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
I20260925 13:56:34.020051 20959 consensus_queue.cc:260] T e9dbcd85dc8647c68c120ba3fcfe1790 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:34.148279 19490 leader_election.cc:304] T 78d8c01c88d24deeae21d647e7f205ba P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3; no voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:34.161159 20868 raft_consensus.cc:2749] T 78d8c01c88d24deeae21d647e7f205ba P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260925 13:56:34.161209 19490 leader_election.cc:304] T b23948807b2c4df9bf9a4da5f767e3b5 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:34.162664 19490 leader_election.cc:304] T 7e544c93488541da9ebb588a465b27b7 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3; no voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:34.163697 19490 leader_election.cc:304] T 82c1002d2bcb4b11839e2f4a93b899ad P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3; no voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be
I20260925 13:56:34.163738 20868 consensus_queue.cc:1059] T e9dbcd85dc8647c68c120ba3fcfe1790 P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 13:56:34.169770 21032 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bcc92728d49643efab5528d77c3d7828)
I20260925 13:56:34.170272 21032 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } }
I20260925 13:56:34.171546 19840 raft_consensus.cc:2468] T 1ce5d15e67644247b95960f444acc621 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:34.182539 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 3aaf29a2310a41d0971eded7ef51fdc9 (table default.Table_95a1130b07a34c82a6a6d7bb1c8245c8 [id=f4beec04dea942f98007b1f4cf33c9ec]) successfully deleted
I20260925 13:56:34.172292 19938 tablet_bootstrap.cc:492] T 45ab2817d7e84185b7ec2bfddcf4997f P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:34.183137 19938 ts_tablet_manager.cc:1403] T 45ab2817d7e84185b7ec2bfddcf4997f P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.224s user 0.015s sys 0.006s
I20260925 13:56:34.185379 19938 raft_consensus.cc:359] T 45ab2817d7e84185b7ec2bfddcf4997f P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:34.186020 19938 raft_consensus.cc:385] T 45ab2817d7e84185b7ec2bfddcf4997f P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:34.186301 19938 raft_consensus.cc:740] T 45ab2817d7e84185b7ec2bfddcf4997f P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:34.187005 19938 consensus_queue.cc:260] T 45ab2817d7e84185b7ec2bfddcf4997f P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:34.198982 19290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" candidate_term: 2 candidate_status { last_received { term: 1 index: 424 } } ignore_live_leader: false dest_uuid: "bcc92728d49643efab5528d77c3d7828" is_pre_election: true
I20260925 13:56:34.221167 19938 ts_tablet_manager.cc:1434] T 45ab2817d7e84185b7ec2bfddcf4997f P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.038s user 0.004s sys 0.000s
I20260925 13:56:34.207650 20644 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 245) took 4352 ms. Trace:
I20260925 13:56:34.221607 20644 rpcz_store.cc:276] 0925 13:56:29.859547 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:29.859653 (+ 106us) service_pool.cc:224] Handling call
0925 13:56:34.182356 (+4322703us) ts_tablet_manager.cc:1151] Deleting tablet 61f141c40ce64fbcb68e01ac15009e3c
0925 13:56:34.207177 (+ 24821us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:34.207584 (+ 407us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4321873}
I20260925 13:56:34.222713 20644 tablet_replica.cc:333] T a218f033a132482a82bd1f60126fa0f9 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:34.223763 20644 raft_consensus.cc:2243] T a218f033a132482a82bd1f60126fa0f9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus shutting down.
W20260925 13:56:34.223708 19266 catalog_manager.cc:5043] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): delete failed for tablet 61f141c40ce64fbcb68e01ac15009e3c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 61f141c40ce64fbcb68e01ac15009e3c already in progress: creating tablet
I20260925 13:56:34.224318 20644 raft_consensus.cc:2272] T a218f033a132482a82bd1f60126fa0f9 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:34.199055 20868 raft_consensus.cc:2804] T b23948807b2c4df9bf9a4da5f767e3b5 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:34.200034 20955 raft_consensus.cc:2749] T 82c1002d2bcb4b11839e2f4a93b899ad P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:34.230250 21032 leader_election.cc:290] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bcc92728d49643efab5528d77c3d7828 (127.18.182.253:39641), fc3953a2ba984a658630c423ec9b9728 (127.18.182.252:36181)
I20260925 13:56:34.230496 19942 tablet_bootstrap.cc:654] T 31b5f47966a74ec3b59c7c20479ba702 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:34.228215 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 236) took 6138 ms. Trace:
I20260925 13:56:34.231236 19955 rpcz_store.cc:276] 0925 13:56:28.089845 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:28.090074 (+ 229us) service_pool.cc:224] Handling call
0925 13:56:34.150653 (+6060579us) ts_tablet_manager.cc:1151] Deleting tablet 26644768f516412f8c95d68a76bb99c5
0925 13:56:34.168970 (+ 18317us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:34.228160 (+ 59190us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6057382}
I20260925 13:56:34.200119 20794 raft_consensus.cc:2749] T 7e544c93488541da9ebb588a465b27b7 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260925 13:56:34.230549 20959 raft_consensus.cc:3060] T e9dbcd85dc8647c68c120ba3fcfe1790 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Advancing to term 2
I20260925 13:56:34.240299 19938 tablet_bootstrap.cc:492] T 540132db164c403da72b9377078c4bf0 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:34.240361 21027 raft_consensus.cc:493] T 908df914cfec490cb7058dbe7b501c97 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:34.241050 20644 ts_tablet_manager.cc:1916] T a218f033a132482a82bd1f60126fa0f9 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:34.240842 21027 raft_consensus.cc:515] T 908df914cfec490cb7058dbe7b501c97 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:34.243914 21027 leader_election.cc:290] T 908df914cfec490cb7058dbe7b501c97 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:34.265195 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 237) took 6167 ms. Trace:
I20260925 13:56:34.265623 19955 rpcz_store.cc:276] 0925 13:56:28.097990 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:28.101243 (+ 3253us) service_pool.cc:224] Handling call
0925 13:56:34.264878 (+6163635us) ts_tablet_manager.cc:1151] Deleting tablet 2a1f34dba13143b097a38bab3782c404
0925 13:56:34.264931 (+ 53us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:34.265136 (+ 205us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6156923}
I20260925 13:56:34.275475 20831 raft_consensus.cc:493] T f146a9aee3054d7491e74101df0e28e1 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:34.275947 20831 raft_consensus.cc:515] T f146a9aee3054d7491e74101df0e28e1 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:34.277803 20831 leader_election.cc:290] T f146a9aee3054d7491e74101df0e28e1 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:34.279742 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f146a9aee3054d7491e74101df0e28e1" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:34.280905 19264 catalog_manager.cc:5030] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): delete failed for tablet 26644768f516412f8c95d68a76bb99c5 because the tablet was not found. No further retry: Not found: Tablet not found: 26644768f516412f8c95d68a76bb99c5
W20260925 13:56:34.281529 19264 catalog_manager.cc:5030] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): delete failed for tablet 2a1f34dba13143b097a38bab3782c404 because the tablet was not found. No further retry: Not found: Tablet not found: 2a1f34dba13143b097a38bab3782c404
W20260925 13:56:34.282126 19773 leader_election.cc:343] T f146a9aee3054d7491e74101df0e28e1 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:34.292877 19955 tablet_replica.cc:333] T c161ae3edfed4f8488c770ecc6332607 P 8f578869bc084a5d8a9a69f143954eb3: stopping tablet replica
I20260925 13:56:34.293648 19955 raft_consensus.cc:2243] T c161ae3edfed4f8488c770ecc6332607 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:34.294625 19955 pending_rounds.cc:70] T c161ae3edfed4f8488c770ecc6332607 P 8f578869bc084a5d8a9a69f143954eb3: Trying to abort 1 pending ops.
I20260925 13:56:34.294902 19955 pending_rounds.cc:77] T c161ae3edfed4f8488c770ecc6332607 P 8f578869bc084a5d8a9a69f143954eb3: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251421340323840 op_type: NO_OP noop_request { }
I20260925 13:56:34.295344 19955 raft_consensus.cc:2889] T c161ae3edfed4f8488c770ecc6332607 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:56:34.295744 19955 raft_consensus.cc:2272] T c161ae3edfed4f8488c770ecc6332607 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:34.298100 19955 ts_tablet_manager.cc:1916] T c161ae3edfed4f8488c770ecc6332607 P 8f578869bc084a5d8a9a69f143954eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:34.304445 19938 tablet_bootstrap.cc:654] T 540132db164c403da72b9377078c4bf0 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:34.311090 19942 tablet_bootstrap.cc:492] T 31b5f47966a74ec3b59c7c20479ba702 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:34.311926 19942 ts_tablet_manager.cc:1403] T 31b5f47966a74ec3b59c7c20479ba702 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.170s user 0.010s sys 0.000s
I20260925 13:56:34.315218 19942 raft_consensus.cc:359] T 31b5f47966a74ec3b59c7c20479ba702 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:34.308652 20868 raft_consensus.cc:697] T b23948807b2c4df9bf9a4da5f767e3b5 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Running, Role: LEADER
I20260925 13:56:34.319990 20868 consensus_queue.cc:237] T b23948807b2c4df9bf9a4da5f767e3b5 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
W20260925 13:56:34.316066 19488 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.194:38641, user_credentials={real_user=slave}} blocked reactor thread for 241862us
W20260925 13:56:34.337430 19488 leader_election.cc:343] T 908df914cfec490cb7058dbe7b501c97 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Not found: Tablet not found: 908df914cfec490cb7058dbe7b501c97
I20260925 13:56:34.309280 20831 raft_consensus.cc:493] T 31b5f47966a74ec3b59c7c20479ba702 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:34.339700 20831 raft_consensus.cc:515] T 31b5f47966a74ec3b59c7c20479ba702 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:34.341743 20831 leader_election.cc:290] T 31b5f47966a74ec3b59c7c20479ba702 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:34.328474 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce5d15e67644247b95960f444acc621" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:34.355759 19942 raft_consensus.cc:385] T 31b5f47966a74ec3b59c7c20479ba702 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:34.355897 19692 raft_consensus.cc:3060] T 1ce5d15e67644247b95960f444acc621 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:34.330255 21025 tablet.cc:1751] T eff33a56361946edb556ac611d2299eb P 7c1b52895a14482fb54a6758f5054f8f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:34.342670 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31b5f47966a74ec3b59c7c20479ba702" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:34.356101 19942 raft_consensus.cc:740] T 31b5f47966a74ec3b59c7c20479ba702 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:34.361286 19942 consensus_queue.cc:260] T 31b5f47966a74ec3b59c7c20479ba702 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:34.363633 19695 raft_consensus.cc:2468] T 31b5f47966a74ec3b59c7c20479ba702 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 0.
I20260925 13:56:34.365293 19773 leader_election.cc:304] T 31b5f47966a74ec3b59c7c20479ba702 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:34.366170 20831 raft_consensus.cc:2804] T 31b5f47966a74ec3b59c7c20479ba702 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:34.366487 20831 raft_consensus.cc:493] T 31b5f47966a74ec3b59c7c20479ba702 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:34.366773 20831 raft_consensus.cc:3060] T 31b5f47966a74ec3b59c7c20479ba702 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:34.374846 19692 raft_consensus.cc:2468] T 1ce5d15e67644247b95960f444acc621 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 1.
I20260925 13:56:34.378187 20644 ts_tablet_manager.cc:1929] T a218f033a132482a82bd1f60126fa0f9 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260925 13:56:34.338650 19490 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 174127us
I20260925 13:56:34.381984 19490 leader_election.cc:304] T 1ce5d15e67644247b95960f444acc621 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:34.389513 20761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" candidate_term: 2 candidate_status { last_received { term: 1 index: 424 } } ignore_live_leader: false dest_uuid: "fc3953a2ba984a658630c423ec9b9728" is_pre_election: true
I20260925 13:56:34.391479 19191 leader_election.cc:304] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7dcc50a8531144a69ab0cc00df1f0df4; no voters: bcc92728d49643efab5528d77c3d7828, fc3953a2ba984a658630c423ec9b9728
I20260925 13:56:34.393451 21032 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 7dcc50a8531144a69ab0cc00df1f0df4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:56:34.396768 20795 raft_consensus.cc:2804] T 1ce5d15e67644247b95960f444acc621 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:34.397213 20795 raft_consensus.cc:697] T 1ce5d15e67644247b95960f444acc621 P 8f578869bc084a5d8a9a69f143954eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Running, Role: LEADER
I20260925 13:56:34.397934 20795 consensus_queue.cc:237] T 1ce5d15e67644247b95960f444acc621 P 8f578869bc084a5d8a9a69f143954eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } }
I20260925 13:56:34.405751 19942 ts_tablet_manager.cc:1434] T 31b5f47966a74ec3b59c7c20479ba702 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.093s user 0.007s sys 0.000s
I20260925 13:56:34.381633 20831 raft_consensus.cc:515] T 31b5f47966a74ec3b59c7c20479ba702 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:34.415328 20644 log.cc:1199] T a218f033a132482a82bd1f60126fa0f9 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/a218f033a132482a82bd1f60126fa0f9
I20260925 13:56:34.416142 20644 ts_tablet_manager.cc:1950] T a218f033a132482a82bd1f60126fa0f9 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:34.423655 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31b5f47966a74ec3b59c7c20479ba702" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:34.424160 19692 raft_consensus.cc:3060] T 31b5f47966a74ec3b59c7c20479ba702 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:34.425890 20644 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 246) took 4558 ms. Trace:
I20260925 13:56:34.430734 19942 tablet_bootstrap.cc:492] T 11077cdb735840b5921e22f588059149 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:34.431447 20644 rpcz_store.cc:276] 0925 13:56:29.867758 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:29.867856 (+ 98us) service_pool.cc:224] Handling call
0925 13:56:34.222549 (+4354693us) ts_tablet_manager.cc:1151] Deleting tablet a218f033a132482a82bd1f60126fa0f9
0925 13:56:34.222612 (+ 63us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:34.291209 (+ 68597us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:34.378078 (+ 86869us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:34.425835 (+ 47757us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4349077}
I20260925 13:56:34.340896 19939 tablet_bootstrap.cc:492] T 37947005aaf14f27a6e5d1f30c67250c P 8f578869bc084a5d8a9a69f143954eb3: No bootstrap required, opened a new log
I20260925 13:56:34.433243 19939 ts_tablet_manager.cc:1403] T 37947005aaf14f27a6e5d1f30c67250c P 8f578869bc084a5d8a9a69f143954eb3: Time spent bootstrapping tablet: real 0.471s user 0.023s sys 0.006s
I20260925 13:56:34.438232 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f146a9aee3054d7491e74101df0e28e1" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:34.439915 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet a218f033a132482a82bd1f60126fa0f9 (table default.Table_67306a96449b4a7bbb4b96498a396cea [id=bcf9d7c5e4f44267b9204bc3c2d90584]) successfully deleted
I20260925 13:56:34.440044 20644 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 244) took 4585 ms. Trace:
I20260925 13:56:34.433727 19938 tablet_bootstrap.cc:492] T 540132db164c403da72b9377078c4bf0 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:34.440529 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31b5f47966a74ec3b59c7c20479ba702" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3" is_pre_election: true
I20260925 13:56:34.440300 20644 rpcz_store.cc:276] 0925 13:56:29.855155 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:29.855433 (+ 278us) service_pool.cc:224] Handling call
0925 13:56:34.439744 (+4584311us) ts_tablet_manager.cc:1151] Deleting tablet 8b858870f1254e72ae2af17c8192ffa0
0925 13:56:34.439780 (+ 36us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:34.439995 (+ 215us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4566046}
I20260925 13:56:34.440807 19938 ts_tablet_manager.cc:1403] T 540132db164c403da72b9377078c4bf0 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.201s user 0.009s sys 0.002s
W20260925 13:56:34.441284 19266 catalog_manager.cc:5043] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): delete failed for tablet 8b858870f1254e72ae2af17c8192ffa0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b858870f1254e72ae2af17c8192ffa0 already in progress: creating tablet
I20260925 13:56:34.441395 20644 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 249) took 3594 ms. Trace:
I20260925 13:56:34.441681 20644 rpcz_store.cc:276] 0925 13:56:30.847374 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:30.868869 (+ 21495us) service_pool.cc:224] Handling call
0925 13:56:34.441101 (+3572232us) ts_tablet_manager.cc:1151] Deleting tablet 7d55b56fa80143248f4ccc14898381b6
0925 13:56:34.441137 (+ 36us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:34.441347 (+ 210us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3536751}
I20260925 13:56:34.442790 19555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31b5f47966a74ec3b59c7c20479ba702" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f578869bc084a5d8a9a69f143954eb3"
I20260925 13:56:34.448096 19942 tablet_bootstrap.cc:654] T 11077cdb735840b5921e22f588059149 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:34.460397 19692 raft_consensus.cc:2468] T 31b5f47966a74ec3b59c7c20479ba702 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 1.
I20260925 13:56:34.465229 19939 raft_consensus.cc:359] T 37947005aaf14f27a6e5d1f30c67250c P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:34.465860 19939 raft_consensus.cc:385] T 37947005aaf14f27a6e5d1f30c67250c P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:34.443408 19938 raft_consensus.cc:359] T 540132db164c403da72b9377078c4bf0 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:34.466117 19939 raft_consensus.cc:740] T 37947005aaf14f27a6e5d1f30c67250c P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578869bc084a5d8a9a69f143954eb3, State: Initialized, Role: FOLLOWER
I20260925 13:56:34.417871 20831 leader_election.cc:290] T 31b5f47966a74ec3b59c7c20479ba702 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
W20260925 13:56:34.444875 19773 leader_election.cc:343] T 37947005aaf14f27a6e5d1f30c67250c P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:34.461165 21029 tablet.cc:1751] T eff33a56361946edb556ac611d2299eb P f790493ba7ad403993b678f2df2b55be: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:34.466706 19773 leader_election.cc:304] T 37947005aaf14f27a6e5d1f30c67250c P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:34.467336 20831 raft_consensus.cc:2749] T 37947005aaf14f27a6e5d1f30c67250c P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:34.467154 19938 raft_consensus.cc:385] T 540132db164c403da72b9377078c4bf0 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:34.468364 19938 raft_consensus.cc:740] T 540132db164c403da72b9377078c4bf0 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:34.468683 19773 leader_election.cc:304] T 31b5f47966a74ec3b59c7c20479ba702 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:34.469298 20831 raft_consensus.cc:2804] T 31b5f47966a74ec3b59c7c20479ba702 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election won for term 1
W20260925 13:56:34.469945 19266 catalog_manager.cc:5043] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): delete failed for tablet 7d55b56fa80143248f4ccc14898381b6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7d55b56fa80143248f4ccc14898381b6 already in progress: creating tablet
I20260925 13:56:34.469149 19938 consensus_queue.cc:260] T 540132db164c403da72b9377078c4bf0 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:34.472790 19773 leader_election.cc:343] T f146a9aee3054d7491e74101df0e28e1 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:34.473153 19773 leader_election.cc:304] T f146a9aee3054d7491e74101df0e28e1 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:34.473476 20831 raft_consensus.cc:697] T 31b5f47966a74ec3b59c7c20479ba702 P f790493ba7ad403993b678f2df2b55be [term 1 LEADER]: Becoming Leader. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Running, Role: LEADER
W20260925 13:56:34.473964 19773 leader_election.cc:343] T 31b5f47966a74ec3b59c7c20479ba702 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:34.474676 19773 leader_election.cc:343] T 31b5f47966a74ec3b59c7c20479ba702 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:34.474315 20831 consensus_queue.cc:237] T 31b5f47966a74ec3b59c7c20479ba702 P f790493ba7ad403993b678f2df2b55be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:34.476270 20891 raft_consensus.cc:2749] T f146a9aee3054d7491e74101df0e28e1 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:34.478441 21023 consensus_queue.cc:1048] T 7e544c93488541da9ebb588a465b27b7 P f790493ba7ad403993b678f2df2b55be [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:34.481916 20644 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 247) took 3635 ms. Trace:
I20260925 13:56:34.482215 20644 rpcz_store.cc:276] 0925 13:56:30.846139 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:30.857480 (+ 11341us) service_pool.cc:224] Handling call
0925 13:56:34.466201 (+3608721us) ts_tablet_manager.cc:1151] Deleting tablet 89bd21e08943403489e63c7c1895393a
0925 13:56:34.481618 (+ 15417us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:34.481866 (+ 248us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3542278}
I20260925 13:56:34.483291 20644 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 250) took 3614 ms. Trace:
I20260925 13:56:34.487782 20644 rpcz_store.cc:276] 0925 13:56:30.869175 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:30.894757 (+ 25582us) service_pool.cc:224] Handling call
0925 13:56:34.483009 (+3588252us) ts_tablet_manager.cc:1151] Deleting tablet 8e5f54ef08054e7fb231c361615e2ab2
0925 13:56:34.483046 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:34.483246 (+ 200us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3521778}
W20260925 13:56:34.489425 19266 catalog_manager.cc:5043] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): delete failed for tablet 89bd21e08943403489e63c7c1895393a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 89bd21e08943403489e63c7c1895393a already in progress: creating tablet
I20260925 13:56:34.490164 19938 ts_tablet_manager.cc:1434] T 540132db164c403da72b9377078c4bf0 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.049s user 0.005s sys 0.000s
W20260925 13:56:34.490826 19266 catalog_manager.cc:5043] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): delete failed for tablet 8e5f54ef08054e7fb231c361615e2ab2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8e5f54ef08054e7fb231c361615e2ab2 already in progress: creating tablet
I20260925 13:56:34.490244 20644 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 254) took 3343 ms. Trace:
I20260925 13:56:34.500010 20644 rpcz_store.cc:276] 0925 13:56:31.147239 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:31.181692 (+ 34453us) service_pool.cc:224] Handling call
0925 13:56:34.489946 (+3308254us) ts_tablet_manager.cc:1151] Deleting tablet d2303a934d974e37ab6482d563492871
0925 13:56:34.489989 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:34.490197 (+ 208us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3307703}
I20260925 13:56:34.501262 20644 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 256) took 2416 ms. Trace:
I20260925 13:56:34.501562 20644 rpcz_store.cc:276] 0925 13:56:32.085000 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:32.085109 (+ 109us) service_pool.cc:224] Handling call
0925 13:56:34.500923 (+2415814us) ts_tablet_manager.cc:1151] Deleting tablet 9c5ef5b050624127990737acab3a8e9f
0925 13:56:34.500965 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:34.501211 (+ 246us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2415269}
W20260925 13:56:34.502523 19266 catalog_manager.cc:5043] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): delete failed for tablet d2303a934d974e37ab6482d563492871 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d2303a934d974e37ab6482d563492871 already in progress: creating tablet
W20260925 13:56:34.503516 19266 catalog_manager.cc:5043] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): delete failed for tablet 9c5ef5b050624127990737acab3a8e9f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9c5ef5b050624127990737acab3a8e9f already in progress: creating tablet
I20260925 13:56:34.504421 20795 raft_consensus.cc:493] T 11077cdb735840b5921e22f588059149 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:34.491053 19938 tablet_bootstrap.cc:492] T e4b29a3d0edc481cba6c61cfb78f8f40 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:34.504873 20795 raft_consensus.cc:515] T 11077cdb735840b5921e22f588059149 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:34.506902 20795 leader_election.cc:290] T 11077cdb735840b5921e22f588059149 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:34.513947 19939 consensus_queue.cc:260] T 37947005aaf14f27a6e5d1f30c67250c P 8f578869bc084a5d8a9a69f143954eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:34.516395 19939 ts_tablet_manager.cc:1434] T 37947005aaf14f27a6e5d1f30c67250c P 8f578869bc084a5d8a9a69f143954eb3: Time spent starting tablet: real 0.083s user 0.005s sys 0.000s
I20260925 13:56:34.519174 19695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11077cdb735840b5921e22f588059149" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:34.519645 20644 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 257) took 2434 ms. Trace:
I20260925 13:56:34.519941 20644 rpcz_store.cc:276] 0925 13:56:32.085262 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:32.100909 (+ 15647us) service_pool.cc:224] Handling call
0925 13:56:34.502281 (+2401372us) ts_tablet_manager.cc:1151] Deleting tablet 84188f52b54b477393d6a8433cbd1cb9
0925 13:56:34.519323 (+ 17042us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:34.519594 (+ 271us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2398513}
I20260925 13:56:34.525434 19942 tablet_bootstrap.cc:492] T 11077cdb735840b5921e22f588059149 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
W20260925 13:56:34.526341 19266 catalog_manager.cc:5043] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): delete failed for tablet 84188f52b54b477393d6a8433cbd1cb9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 84188f52b54b477393d6a8433cbd1cb9 already in progress: creating tablet
I20260925 13:56:34.526499 20644 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 259) took 2424 ms. Trace:
I20260925 13:56:34.526994 19942 ts_tablet_manager.cc:1403] T 11077cdb735840b5921e22f588059149 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.096s user 0.010s sys 0.000s
I20260925 13:56:34.526832 20644 rpcz_store.cc:276] 0925 13:56:32.101846 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:32.102492 (+ 646us) service_pool.cc:224] Handling call
0925 13:56:34.526136 (+2423644us) ts_tablet_manager.cc:1151] Deleting tablet 43692c9ae2dc460d9e32efad41c263c1
0925 13:56:34.526177 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:34.526445 (+ 268us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2422209}
I20260925 13:56:34.527892 20644 tablet_replica.cc:333] T 000aad39953c4688b543c7d882a08ec8 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:34.528568 20644 raft_consensus.cc:2243] T 000aad39953c4688b543c7d882a08ec8 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:34.529587 20644 raft_consensus.cc:2272] T 000aad39953c4688b543c7d882a08ec8 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 13:56:34.529974 19266 catalog_manager.cc:5043] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): delete failed for tablet 43692c9ae2dc460d9e32efad41c263c1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 43692c9ae2dc460d9e32efad41c263c1 already in progress: creating tablet
I20260925 13:56:34.531781 19938 tablet_bootstrap.cc:654] T e4b29a3d0edc481cba6c61cfb78f8f40 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:34.535718 19833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11077cdb735840b5921e22f588059149" candidate_uuid: "8f578869bc084a5d8a9a69f143954eb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:34.536260 19833 raft_consensus.cc:2468] T 11077cdb735840b5921e22f588059149 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f578869bc084a5d8a9a69f143954eb3 in term 0.
I20260925 13:56:34.537963 20644 ts_tablet_manager.cc:1916] T 000aad39953c4688b543c7d882a08ec8 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:34.544739 19939 tablet_bootstrap.cc:492] T 505a24ebe7014f80b3c184329cc74a9a P 8f578869bc084a5d8a9a69f143954eb3: Bootstrap starting.
W20260925 13:56:34.520628 19488 leader_election.cc:343] T 11077cdb735840b5921e22f588059149 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:34.553372 19938 tablet_bootstrap.cc:492] T e4b29a3d0edc481cba6c61cfb78f8f40 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:34.553833 19938 ts_tablet_manager.cc:1403] T e4b29a3d0edc481cba6c61cfb78f8f40 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.063s user 0.009s sys 0.001s
I20260925 13:56:34.557226 19938 raft_consensus.cc:359] T e4b29a3d0edc481cba6c61cfb78f8f40 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:34.562084 19938 raft_consensus.cc:385] T e4b29a3d0edc481cba6c61cfb78f8f40 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:34.562382 19938 raft_consensus.cc:740] T e4b29a3d0edc481cba6c61cfb78f8f40 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:34.563036 19938 consensus_queue.cc:260] T e4b29a3d0edc481cba6c61cfb78f8f40 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:34.565248 19938 ts_tablet_manager.cc:1434] T e4b29a3d0edc481cba6c61cfb78f8f40 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.011s user 0.002s sys 0.003s
I20260925 13:56:34.562346 19942 raft_consensus.cc:359] T 11077cdb735840b5921e22f588059149 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:34.573669 19942 raft_consensus.cc:385] T 11077cdb735840b5921e22f588059149 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:34.573920 19942 raft_consensus.cc:740] T 11077cdb735840b5921e22f588059149 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:34.574441 19938 tablet_bootstrap.cc:492] T de2b3e07bd634bc08368989a9062ba08 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:34.574515 19942 consensus_queue.cc:260] T 11077cdb735840b5921e22f588059149 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:34.577109 21006 tablet.cc:1751] T 5f8a96ae6eec4f15a221f9aad537c094 P 8f578869bc084a5d8a9a69f143954eb3: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:56:34.579846 19938 tablet_bootstrap.cc:654] T de2b3e07bd634bc08368989a9062ba08 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:34.592935 19942 ts_tablet_manager.cc:1434] T 11077cdb735840b5921e22f588059149 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.064s user 0.004s sys 0.001s
I20260925 13:56:34.571657 20644 ts_tablet_manager.cc:1929] T 000aad39953c4688b543c7d882a08ec8 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:34.593350 20644 log.cc:1199] T 000aad39953c4688b543c7d882a08ec8 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/000aad39953c4688b543c7d882a08ec8
I20260925 13:56:34.593801 19942 tablet_bootstrap.cc:492] T a218f033a132482a82bd1f60126fa0f9 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:34.594206 20644 ts_tablet_manager.cc:1950] T 000aad39953c4688b543c7d882a08ec8 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:34.596541 20644 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 260) took 2493 ms. Trace:
I20260925 13:56:34.596889 20644 rpcz_store.cc:276] 0925 13:56:32.103153 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:32.103497 (+ 344us) service_pool.cc:224] Handling call
0925 13:56:34.527750 (+2424253us) ts_tablet_manager.cc:1151] Deleting tablet 000aad39953c4688b543c7d882a08ec8
0925 13:56:34.527788 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:34.551099 (+ 23311us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:34.571541 (+ 20442us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:34.596054 (+ 24513us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2423750}
I20260925 13:56:34.598048 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 000aad39953c4688b543c7d882a08ec8 (table default.Table_3a3e48a0641a43a1a783c6ba6374a708 [id=e64fe44a237f48859a288a87a565ee5e]) successfully deleted
I20260925 13:56:34.598497 20644 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 264) took 2378 ms. Trace:
I20260925 13:56:34.598842 20644 rpcz_store.cc:276] 0925 13:56:32.219675 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:32.219786 (+ 111us) service_pool.cc:224] Handling call
0925 13:56:34.598194 (+2378408us) ts_tablet_manager.cc:1151] Deleting tablet 4b36d5f94de04dca98b62e8128602cbb
0925 13:56:34.598233 (+ 39us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:34.598448 (+ 215us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2377873}
W20260925 13:56:34.600134 19266 catalog_manager.cc:5043] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): delete failed for tablet 4b36d5f94de04dca98b62e8128602cbb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4b36d5f94de04dca98b62e8128602cbb already in progress: creating tablet
I20260925 13:56:34.600514 20644 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 266) took 2266 ms. Trace:
I20260925 13:56:34.600970 20644 rpcz_store.cc:276] 0925 13:56:32.334050 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:32.335708 (+ 1658us) service_pool.cc:224] Handling call
0925 13:56:34.600230 (+2264522us) ts_tablet_manager.cc:1151] Deleting tablet 67ff370d64d64e9da3a44747e44df9c8
0925 13:56:34.600264 (+ 34us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:34.600470 (+ 206us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2258547}
W20260925 13:56:34.602594 19266 catalog_manager.cc:5043] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): delete failed for tablet 67ff370d64d64e9da3a44747e44df9c8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 67ff370d64d64e9da3a44747e44df9c8 already in progress: creating tablet
I20260925 13:56:34.603227 20644 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51442 (request call id 277) took 1201 ms. Trace:
I20260925 13:56:34.604884 20644 rpcz_store.cc:276] 0925 13:56:33.401641 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:33.411829 (+ 10188us) service_pool.cc:224] Handling call
0925 13:56:34.602922 (+1191093us) ts_tablet_manager.cc:1151] Deleting tablet f753f6e45ade49ffa3128d5f4e09584a
0925 13:56:34.602962 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:34.603181 (+ 219us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1162233}
I20260925 13:56:34.608901 19938 tablet_bootstrap.cc:492] T de2b3e07bd634bc08368989a9062ba08 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:34.609740 19938 ts_tablet_manager.cc:1403] T de2b3e07bd634bc08368989a9062ba08 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.035s user 0.010s sys 0.001s
W20260925 13:56:34.609988 19266 catalog_manager.cc:5043] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): delete failed for tablet f753f6e45ade49ffa3128d5f4e09584a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f753f6e45ade49ffa3128d5f4e09584a already in progress: creating tablet
I20260925 13:56:34.612524 19938 raft_consensus.cc:359] T de2b3e07bd634bc08368989a9062ba08 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:34.613294 19938 raft_consensus.cc:385] T de2b3e07bd634bc08368989a9062ba08 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:34.614777 19938 raft_consensus.cc:740] T de2b3e07bd634bc08368989a9062ba08 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:34.615582 19938 consensus_queue.cc:260] T de2b3e07bd634bc08368989a9062ba08 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:34.622634 19938 ts_tablet_manager.cc:1434] T de2b3e07bd634bc08368989a9062ba08 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.013s user 0.004s sys 0.000s
I20260925 13:56:34.623570 19938 tablet_bootstrap.cc:492] T 72183473a5c94629a58c2cd8e1c56f32 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:34.624464 19955 ts_tablet_manager.cc:1929] T c161ae3edfed4f8488c770ecc6332607 P 8f578869bc084a5d8a9a69f143954eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:34.624944 19955 log.cc:1199] T c161ae3edfed4f8488c770ecc6332607 P 8f578869bc084a5d8a9a69f143954eb3: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/c161ae3edfed4f8488c770ecc6332607
I20260925 13:56:34.625692 19955 ts_tablet_manager.cc:1950] T c161ae3edfed4f8488c770ecc6332607 P 8f578869bc084a5d8a9a69f143954eb3: Deleting consensus metadata
I20260925 13:56:34.627720 20993 raft_consensus.cc:493] T 72183473a5c94629a58c2cd8e1c56f32 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:34.628528 19938 tablet_bootstrap.cc:654] T 72183473a5c94629a58c2cd8e1c56f32 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:34.628159 20993 raft_consensus.cc:515] T 72183473a5c94629a58c2cd8e1c56f32 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:34.630151 20993 leader_election.cc:290] T 72183473a5c94629a58c2cd8e1c56f32 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:34.632033 19833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72183473a5c94629a58c2cd8e1c56f32" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
W20260925 13:56:34.633255 19631 leader_election.cc:343] T 72183473a5c94629a58c2cd8e1c56f32 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:34.650177 19942 tablet_bootstrap.cc:654] T a218f033a132482a82bd1f60126fa0f9 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:34.653007 19938 tablet_bootstrap.cc:492] T 72183473a5c94629a58c2cd8e1c56f32 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:34.653402 19938 ts_tablet_manager.cc:1403] T 72183473a5c94629a58c2cd8e1c56f32 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.030s user 0.007s sys 0.000s
I20260925 13:56:34.655530 19938 raft_consensus.cc:359] T 72183473a5c94629a58c2cd8e1c56f32 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:34.656155 19938 raft_consensus.cc:385] T 72183473a5c94629a58c2cd8e1c56f32 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:34.656380 19938 raft_consensus.cc:740] T 72183473a5c94629a58c2cd8e1c56f32 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:34.656895 19938 consensus_queue.cc:260] T 72183473a5c94629a58c2cd8e1c56f32 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:34.658433 19938 ts_tablet_manager.cc:1434] T 72183473a5c94629a58c2cd8e1c56f32 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260925 13:56:34.668859 19938 tablet_bootstrap.cc:492] T a0109e4115174d8eb8174a1a2236150f P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:34.674501 19938 tablet_bootstrap.cc:654] T a0109e4115174d8eb8174a1a2236150f P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:34.697165 19942 tablet_bootstrap.cc:492] T a218f033a132482a82bd1f60126fa0f9 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:34.697589 19942 ts_tablet_manager.cc:1403] T a218f033a132482a82bd1f60126fa0f9 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.104s user 0.009s sys 0.000s
I20260925 13:56:34.700887 19942 raft_consensus.cc:359] T a218f033a132482a82bd1f60126fa0f9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:34.707079 19942 raft_consensus.cc:385] T a218f033a132482a82bd1f60126fa0f9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:34.706987 20891 raft_consensus.cc:493] T 540132db164c403da72b9377078c4bf0 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:34.707319 19942 raft_consensus.cc:740] T a218f033a132482a82bd1f60126fa0f9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:34.707458 20891 raft_consensus.cc:515] T 540132db164c403da72b9377078c4bf0 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:34.707774 19272 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 31b5f47966a74ec3b59c7c20479ba702 (table default.Table_bd9c5f2059394324857aa7fef30188f7 [id=e30a4ca9b0bf4c11a6691a896304c57e]). Expected version 1 got 0
I20260925 13:56:34.708115 19942 consensus_queue.cc:260] T a218f033a132482a82bd1f60126fa0f9 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:34.709267 20891 leader_election.cc:290] T 540132db164c403da72b9377078c4bf0 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
W20260925 13:56:34.710899 19272 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.195:57799 (request call id 45) took 1061 ms (client timeout 1000 ms). Trace:
I20260925 13:56:34.710889 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540132db164c403da72b9377078c4bf0" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:34.712121 19773 leader_election.cc:343] T 540132db164c403da72b9377078c4bf0 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:34.712999 19802 tablet_service.cc:1558] Processing DeleteTablet for tablet 000aad39953c4688b543c7d882a08ec8 with delete_type TABLET_DATA_DELETED (replaced by tablet 3927adc85b2746d0977a3cb6f84ad68f) from {username='slave'} at 127.0.0.1:51442
W20260925 13:56:34.714426 19266 catalog_manager.cc:5030] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): delete failed for tablet 000aad39953c4688b543c7d882a08ec8 because the tablet was not found. No further retry: Not found: Tablet not found: 000aad39953c4688b543c7d882a08ec8
I20260925 13:56:34.716823 19279 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 5f8a96ae6eec4f15a221f9aad537c094 (table default.Table_3a06919b658e46f893318b4e1ec8b52f [id=e6f2a872536b4e55b071231020903581]). Expected version 1 got 0
I20260925 13:56:34.717481 19279 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet b3172d99afee4a4c878e71b645037917 (table default.Table_bd52a86b8ff14bdbb06fcf88a60b1e82 [id=203024d67e444f839391c8670d7c7d3b]). Expected version 2 got 0
I20260925 13:56:34.718134 19279 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 7e544c93488541da9ebb588a465b27b7 (table default.Table_4f849dc343f5448eb14a195bb0777d12 [id=f71e790b94b34e3d88f7539e396bc94d]). Expected version 1 got 0
I20260925 13:56:34.718909 19279 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet b23948807b2c4df9bf9a4da5f767e3b5 (table default.Table_fd205659579c49559fd2d121b32274d4 [id=0165d602da03423f82104b16da9a20b0]). Expected version 1 got 0
W20260925 13:56:34.723054 19280 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.193:37907 (request call id 42) took 960 ms (client timeout 1000 ms). Trace:
I20260925 13:56:34.722678 20891 raft_consensus.cc:493] T e4b29a3d0edc481cba6c61cfb78f8f40 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:34.723539 20891 raft_consensus.cc:515] T e4b29a3d0edc481cba6c61cfb78f8f40 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:34.725287 20891 leader_election.cc:290] T e4b29a3d0edc481cba6c61cfb78f8f40 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:34.727171 19668 tablet_service.cc:1511] Processing CreateTablet for tablet 836fc262f8c54a2dbbbb5ab885b033da (DEFAULT_TABLE table=default.Table_b77c6557f7004edd8970cde538de23a9 [id=a715598baed04e3fba297af3b41b9fe6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:34.728353 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4b29a3d0edc481cba6c61cfb78f8f40" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:34.729027 19802 tablet_service.cc:1511] Processing CreateTablet for tablet 836fc262f8c54a2dbbbb5ab885b033da (DEFAULT_TABLE table=default.Table_b77c6557f7004edd8970cde538de23a9 [id=a715598baed04e3fba297af3b41b9fe6]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 13:56:34.729812 19773 leader_election.cc:343] T e4b29a3d0edc481cba6c61cfb78f8f40 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:34.730050 19802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 836fc262f8c54a2dbbbb5ab885b033da. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:34.730008 19280 rpcz_store.cc:269] 0925 13:56:33.763082 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:33.763209 (+ 127us) service_pool.cc:224] Handling call
0925 13:56:34.723001 (+959792us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":6}
W20260925 13:56:34.711184 19272 rpcz_store.cc:269] 0925 13:56:33.649733 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:33.649841 (+ 108us) service_pool.cc:224] Handling call
0925 13:56:34.710833 (+1060992us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":3}
I20260925 13:56:34.735517 20891 raft_consensus.cc:493] T 72183473a5c94629a58c2cd8e1c56f32 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:34.735953 20891 raft_consensus.cc:515] T 72183473a5c94629a58c2cd8e1c56f32 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:34.736588 19942 ts_tablet_manager.cc:1434] T a218f033a132482a82bd1f60126fa0f9 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.039s user 0.007s sys 0.000s
I20260925 13:56:34.738605 19669 tablet_service.cc:1511] Processing CreateTablet for tablet 6e5c173440ec4b8ea0409e7734f9249e (DEFAULT_TABLE table=default.Table_640b2a7e12424a0c960c5ee25a876667 [id=9210775ea77d4e7e868b8d625857aa82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:34.739290 19817 tablet_service.cc:1511] Processing CreateTablet for tablet 6e5c173440ec4b8ea0409e7734f9249e (DEFAULT_TABLE table=default.Table_640b2a7e12424a0c960c5ee25a876667 [id=9210775ea77d4e7e868b8d625857aa82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:34.740180 19817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e5c173440ec4b8ea0409e7734f9249e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:34.740249 20891 leader_election.cc:290] T 72183473a5c94629a58c2cd8e1c56f32 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:34.746057 19819 tablet_service.cc:1511] Processing CreateTablet for tablet 40e326118fad4811b1d3ebd7f632ca92 (DEFAULT_TABLE table=default.Table_ddc133aff70b4778a2e87490f469da02 [id=7e353b31f2c04633b159015cb8a64d41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:34.746924 19819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40e326118fad4811b1d3ebd7f632ca92. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:34.750250 19279 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.194:34125 (request call id 49) took 1066 ms (client timeout 1000 ms). Trace:
I20260925 13:56:34.751197 19818 tablet_service.cc:1511] Processing CreateTablet for tablet 0697a7bab74e4221918964644a0a0bce (DEFAULT_TABLE table=default.Table_845bbf47906f4e3fb7c047f2780dd6e3 [id=f0a63fe5a50d483980f5e3dd91faf817]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:34.752123 19818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0697a7bab74e4221918964644a0a0bce. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:34.753711 19490 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 158894us
W20260925 13:56:34.754374 19490 leader_election.cc:343] T 908df914cfec490cb7058dbe7b501c97 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: 908df914cfec490cb7058dbe7b501c97
I20260925 13:56:34.754850 19490 leader_election.cc:304] T 908df914cfec490cb7058dbe7b501c97 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3; no voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be
W20260925 13:56:34.755630 21024 log.cc:927] Time spent T f3b46e4fac694084b06b7086bcc596b9 P 8f578869bc084a5d8a9a69f143954eb3: Append to log took a long time: real 0.158s user 0.000s sys 0.000s
I20260925 13:56:34.756984 20891 raft_consensus.cc:493] T 45ab2817d7e84185b7ec2bfddcf4997f P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:34.757428 20891 raft_consensus.cc:515] T 45ab2817d7e84185b7ec2bfddcf4997f P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:34.759168 20891 leader_election.cc:290] T 45ab2817d7e84185b7ec2bfddcf4997f P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:34.759317 19666 tablet_service.cc:1511] Processing CreateTablet for tablet 0697a7bab74e4221918964644a0a0bce (DEFAULT_TABLE table=default.Table_845bbf47906f4e3fb7c047f2780dd6e3 [id=f0a63fe5a50d483980f5e3dd91faf817]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:34.759733 19671 tablet_service.cc:1511] Processing CreateTablet for tablet 40e326118fad4811b1d3ebd7f632ca92 (DEFAULT_TABLE table=default.Table_ddc133aff70b4778a2e87490f469da02 [id=7e353b31f2c04633b159015cb8a64d41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:34.760103 19675 tablet_service.cc:1558] Processing DeleteTablet for tablet 78d8c01c88d24deeae21d647e7f205ba with delete_type TABLET_DATA_DELETED (replaced by tablet ab1ba6ef73a046b59ffac150f7c73953) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:34.760370 19674 tablet_service.cc:1558] Processing DeleteTablet for tablet c5a9df833271480aa49feb7d9f925459 with delete_type TABLET_DATA_DELETED (replaced by tablet 236e308d8694405cb2b874164fa16485) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:34.761273 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72183473a5c94629a58c2cd8e1c56f32" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:34.761763 19692 raft_consensus.cc:2468] T 72183473a5c94629a58c2cd8e1c56f32 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 0.
I20260925 13:56:34.768590 19773 leader_election.cc:304] T 72183473a5c94629a58c2cd8e1c56f32 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:34.769572 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45ab2817d7e84185b7ec2bfddcf4997f" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:34.775553 20891 raft_consensus.cc:2804] T 72183473a5c94629a58c2cd8e1c56f32 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:34.775929 20891 raft_consensus.cc:493] T 72183473a5c94629a58c2cd8e1c56f32 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:34.776268 20891 raft_consensus.cc:3060] T 72183473a5c94629a58c2cd8e1c56f32 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
W20260925 13:56:34.777451 19773 leader_election.cc:343] T 45ab2817d7e84185b7ec2bfddcf4997f P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:34.781028 19939 tablet_bootstrap.cc:654] T 505a24ebe7014f80b3c184329cc74a9a P 8f578869bc084a5d8a9a69f143954eb3: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:34.785492 20891 raft_consensus.cc:515] T 72183473a5c94629a58c2cd8e1c56f32 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:34.791502 20891 leader_election.cc:290] T 72183473a5c94629a58c2cd8e1c56f32 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:34.793020 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72183473a5c94629a58c2cd8e1c56f32" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f"
I20260925 13:56:34.793465 19692 raft_consensus.cc:3060] T 72183473a5c94629a58c2cd8e1c56f32 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:34.797763 21031 raft_consensus.cc:2749] T 908df914cfec490cb7058dbe7b501c97 P 8f578869bc084a5d8a9a69f143954eb3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:34.797706 19518 tablet_service.cc:1511] Processing CreateTablet for tablet 6e5c173440ec4b8ea0409e7734f9249e (DEFAULT_TABLE table=default.Table_640b2a7e12424a0c960c5ee25a876667 [id=9210775ea77d4e7e868b8d625857aa82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:34.798856 19518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e5c173440ec4b8ea0409e7734f9249e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:34.799580 19490 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 45245us
I20260925 13:56:34.799921 19668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 836fc262f8c54a2dbbbb5ab885b033da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:34.801946 19490 leader_election.cc:304] T 11077cdb735840b5921e22f588059149 P 8f578869bc084a5d8a9a69f143954eb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8f578869bc084a5d8a9a69f143954eb3, f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f
I20260925 13:56:34.803251 19942 tablet_bootstrap.cc:492] T f146a9aee3054d7491e74101df0e28e1 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:34.803748 19669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e5c173440ec4b8ea0409e7734f9249e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:34.807551 19520 tablet_service.cc:1511] Processing CreateTablet for tablet 0697a7bab74e4221918964644a0a0bce (DEFAULT_TABLE table=default.Table_845bbf47906f4e3fb7c047f2780dd6e3 [id=f0a63fe5a50d483980f5e3dd91faf817]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:34.807826 19666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0697a7bab74e4221918964644a0a0bce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:34.808490 19520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0697a7bab74e4221918964644a0a0bce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:34.808374 19523 tablet_service.cc:1511] Processing CreateTablet for tablet 40e326118fad4811b1d3ebd7f632ca92 (DEFAULT_TABLE table=default.Table_ddc133aff70b4778a2e87490f469da02 [id=7e353b31f2c04633b159015cb8a64d41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:34.809315 19523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40e326118fad4811b1d3ebd7f632ca92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:34.810503 19529 tablet_service.cc:1511] Processing CreateTablet for tablet acf234ea540a442c9ed3d8a4291c0b58 (DEFAULT_TABLE table=default.Table_8189c34636b04c0881a90a5311af243e [id=b12f135fa98a473e9279eb9844872089]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:34.811386 19529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acf234ea540a442c9ed3d8a4291c0b58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:34.811805 19519 tablet_service.cc:1511] Processing CreateTablet for tablet 836fc262f8c54a2dbbbb5ab885b033da (DEFAULT_TABLE table=default.Table_b77c6557f7004edd8970cde538de23a9 [id=a715598baed04e3fba297af3b41b9fe6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:34.812721 19519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 836fc262f8c54a2dbbbb5ab885b033da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:34.812896 19671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40e326118fad4811b1d3ebd7f632ca92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:34.815526 21055 tablet_replica.cc:333] T 78d8c01c88d24deeae21d647e7f205ba P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:34.816434 21055 raft_consensus.cc:2243] T 78d8c01c88d24deeae21d647e7f205ba P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:34.816852 21055 raft_consensus.cc:2272] T 78d8c01c88d24deeae21d647e7f205ba P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:34.818553 21055 ts_tablet_manager.cc:1916] T 78d8c01c88d24deeae21d647e7f205ba P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:34.820663 19675 tablet_service.cc:1511] Processing CreateTablet for tablet acf234ea540a442c9ed3d8a4291c0b58 (DEFAULT_TABLE table=default.Table_8189c34636b04c0881a90a5311af243e [id=b12f135fa98a473e9279eb9844872089]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:34.821559 19675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acf234ea540a442c9ed3d8a4291c0b58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:34.823001 19942 tablet_bootstrap.cc:654] T f146a9aee3054d7491e74101df0e28e1 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:34.827636 20993 raft_consensus.cc:493] T 505a24ebe7014f80b3c184329cc74a9a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:34.828068 20993 raft_consensus.cc:515] T 505a24ebe7014f80b3c184329cc74a9a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:34.831142 20993 leader_election.cc:290] T 505a24ebe7014f80b3c184329cc74a9a P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
W20260925 13:56:34.841399 19279 rpcz_store.cc:269] 0925 13:56:33.684042 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:33.687445 (+ 3403us) service_pool.cc:224] Handling call
0925 13:56:34.750180 (+1062735us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":10}
I20260925 13:56:34.851979 19955 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40018 (request call id 233) took 6959 ms. Trace:
I20260925 13:56:34.852284 19955 rpcz_store.cc:276] 0925 13:56:27.892812 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:27.955845 (+ 63033us) service_pool.cc:224] Handling call
0925 13:56:34.266827 (+6310982us) ts_tablet_manager.cc:1151] Deleting tablet c161ae3edfed4f8488c770ecc6332607
0925 13:56:34.292770 (+ 25943us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:56:34.497718 (+204948us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:34.624328 (+126610us) tablet_metadata.cc:665] Metadata flushed
0925 13:56:34.851927 (+227599us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48331,"tablet-delete.queue_time_us":6130036}
I20260925 13:56:34.854929 19264 catalog_manager.cc:5063] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): tablet c161ae3edfed4f8488c770ecc6332607 (table default.Table_475efa7896f84e658eb81238ceb58d8f [id=5e975bd1dd6e487e97377bc2896e6a3f]) successfully deleted
I20260925 13:56:34.861611 21055 ts_tablet_manager.cc:1929] T 78d8c01c88d24deeae21d647e7f205ba P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:34.862644 21055 log.cc:1199] T 78d8c01c88d24deeae21d647e7f205ba P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/78d8c01c88d24deeae21d647e7f205ba
I20260925 13:56:34.863556 21055 ts_tablet_manager.cc:1950] T 78d8c01c88d24deeae21d647e7f205ba P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:34.865932 21055 tablet_replica.cc:333] T c5a9df833271480aa49feb7d9f925459 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:34.866762 21055 raft_consensus.cc:2243] T c5a9df833271480aa49feb7d9f925459 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:34.867304 21055 raft_consensus.cc:2272] T c5a9df833271480aa49feb7d9f925459 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:34.868217 19276 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet 3d754e17930549189d65becb98afe2a6
I20260925 13:56:34.869720 19276 catalog_manager.cc:6934] ReplaceTablet: tablet 3d754e17930549189d65becb98afe2a6 deleted and replaced by tablet fe5068f8a43a4474ac189b522fdc494e
I20260925 13:56:34.870050 19276 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:39288 (request call id 354) took 1264 ms. Trace:
I20260925 13:56:34.870432 21055 ts_tablet_manager.cc:1916] T c5a9df833271480aa49feb7d9f925459 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:34.872911 19531 tablet_service.cc:1558] Processing DeleteTablet for tablet 3d754e17930549189d65becb98afe2a6 with delete_type TABLET_DATA_DELETED (replaced by tablet fe5068f8a43a4474ac189b522fdc494e) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:34.873857 19271 catalog_manager.cc:6133] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: Sending DeleteTablet for 3 replicas of tablet eff33a56361946edb556ac611d2299eb
I20260925 13:56:34.875177 19271 catalog_manager.cc:6934] ReplaceTablet: tablet eff33a56361946edb556ac611d2299eb deleted and replaced by tablet 45a578a5d0d94488b72b841a3057da69
I20260925 13:56:34.880631 19163 master-stress-test.cc:581] Killed test thread 9/10
I20260925 13:56:34.880891 19942 tablet_bootstrap.cc:492] T f146a9aee3054d7491e74101df0e28e1 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:34.880982 19163 master-stress-test.cc:579] Killing test thread 10/10
I20260925 13:56:34.881289 19942 ts_tablet_manager.cc:1403] T f146a9aee3054d7491e74101df0e28e1 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.078s user 0.008s sys 0.002s
I20260925 13:56:34.882544 19531 tablet_service.cc:1558] Processing DeleteTablet for tablet eff33a56361946edb556ac611d2299eb with delete_type TABLET_DATA_DELETED (replaced by tablet 45a578a5d0d94488b72b841a3057da69) from {username='slave'} at 127.0.0.1:40018
I20260925 13:56:34.870334 19276 rpcz_store.cc:276] 0925 13:56:33.605421 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:33.605543 (+ 122us) service_pool.cc:224] Handling call
0925 13:56:34.869991 (+1264448us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:56:34.743960 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:56:34.744370 (+ 410us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:56:34.744406 (+ 36us) write_op.cc:435] Acquired schema lock
0925 13:56:34.744422 (+ 16us) tablet.cc:662] Decoding operations
0925 13:56:34.744791 (+ 369us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:56:34.744837 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0925 13:56:34.744860 (+ 23us) tablet.cc:685] Acquiring locks for 2 operations
0925 13:56:34.745080 (+ 220us) tablet.cc:701] Row locks acquired
0925 13:56:34.745096 (+ 16us) write_op.cc:260] PREPARE: finished
0925 13:56:34.745211 (+ 115us) write_op.cc:270] Start()
0925 13:56:34.745297 (+ 86us) write_op.cc:276] Timestamp: P: 1790344594745172 usec, L: 0
0925 13:56:34.745318 (+ 21us) op_driver.cc:348] REPLICATION: starting
0925 13:56:34.778864 (+ 33546us) log.cc:844] Serialized 631 byte log entry
0925 13:56:34.780361 (+ 1497us) spinlock_profiling.cc:243] Waited 33 ms on lock 0x7b5400040454. stack: 00007fdf0a30b323 00007fdf0a30b220 00007fdf09eaf574 0000000000516964 00000000005168f9 00007fdf11408aa1 00007fdf1149b5e8 00007fdf1192b741 00007fdf1192af18 00007fdf1192fbb1 00007fdf1192fb69 00007fdf1192faf9 00007fdf1192fac1 00007fdf1192eced 00000000005bd4e4 00000000005bd319
0925 13:56:34.833877 (+ 53516us) op_driver.cc:464] REPLICATION: finished
0925 13:56:34.842620 (+ 8743us) write_op.cc:301] APPLY: starting
0925 13:56:34.842678 (+ 58us) tablet.cc:1394] starting BulkCheckPresence
0925 13:56:34.844174 (+ 1496us) tablet.cc:1397] finished BulkCheckPresence
0925 13:56:34.844195 (+ 21us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:56:34.844548 (+ 353us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:56:34.861726 (+ 17178us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 13:56:34.861774 (+ 48us) write_op.cc:312] APPLY: finished
0925 13:56:34.861975 (+ 201us) log.cc:844] Serialized 29 byte log entry
0925 13:56:34.862139 (+ 164us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:56:34.862203 (+ 64us) write_op.cc:454] Released schema lock
0925 13:56:34.862235 (+ 32us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":8564,"num_ops":2,"prepare.queue_time_us":29345,"prepare.run_cpu_time_us":3690,"prepare.run_wall_time_us":36612,"raft.queue_time_us":40553,"raft.run_cpu_time_us":718,"raft.run_wall_time_us":718,"replication_time_us":88472,"spinlock_wait_cycles":75867008}]]}
I20260925 13:56:34.884542 20993 raft_consensus.cc:493] T 11077cdb735840b5921e22f588059149 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:34.884944 20993 raft_consensus.cc:515] T 11077cdb735840b5921e22f588059149 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:34.886181 21055 ts_tablet_manager.cc:1929] T c5a9df833271480aa49feb7d9f925459 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:34.886613 21055 log.cc:1199] T c5a9df833271480aa49feb7d9f925459 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/c5a9df833271480aa49feb7d9f925459
I20260925 13:56:34.887773 21055 ts_tablet_manager.cc:1950] T c5a9df833271480aa49feb7d9f925459 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:34.887919 19277 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet f3b46e4fac694084b06b7086bcc596b9 (table default.Table_2efda975ab0a464eaf96f1f9a48eff1c [id=c31f813d79a244afa726df0238376aba]). Expected version 1 got 0
I20260925 13:56:34.889010 19277 catalog_manager.cc:5838] T 31b5f47966a74ec3b59c7c20479ba702 P f790493ba7ad403993b678f2df2b55be reported cstate change: term changed from 0 to 1, leader changed from <none> to f790493ba7ad403993b678f2df2b55be (127.18.182.195). New cstate: current_term: 1 leader_uuid: "f790493ba7ad403993b678f2df2b55be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:34.890496 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 78d8c01c88d24deeae21d647e7f205ba (table default.Table_17642b3f89974e5c97036637fe6d04d3 [id=4ca863b59dfd4410822e15705934b503]) successfully deleted
I20260925 13:56:34.890730 19163 master-stress-test.cc:581] Killed test thread 10/10
I20260925 13:56:34.891093 19163 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskYmJttM/build/tsan/bin/kudu with pid 19460
I20260925 13:56:34.891188 19942 raft_consensus.cc:359] T f146a9aee3054d7491e74101df0e28e1 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:34.892241 19942 raft_consensus.cc:385] T f146a9aee3054d7491e74101df0e28e1 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:34.892482 19942 raft_consensus.cc:740] T f146a9aee3054d7491e74101df0e28e1 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:34.892976 19942 consensus_queue.cc:260] T f146a9aee3054d7491e74101df0e28e1 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:34.894749 19942 ts_tablet_manager.cc:1434] T f146a9aee3054d7491e74101df0e28e1 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.013s user 0.006s sys 0.001s
I20260925 13:56:34.890372 19277 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 31b5f47966a74ec3b59c7c20479ba702 (table default.Table_bd9c5f2059394324857aa7fef30188f7 [id=e30a4ca9b0bf4c11a6691a896304c57e]). Expected version 1 got 0
I20260925 13:56:34.896209 19814 tablet_service.cc:1558] Processing DeleteTablet for tablet 3d754e17930549189d65becb98afe2a6 with delete_type TABLET_DATA_DELETED (replaced by tablet fe5068f8a43a4474ac189b522fdc494e) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:34.898206 19814 tablet_service.cc:1558] Processing DeleteTablet for tablet eff33a56361946edb556ac611d2299eb with delete_type TABLET_DATA_DELETED (replaced by tablet 45a578a5d0d94488b72b841a3057da69) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:34.899740 19830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505a24ebe7014f80b3c184329cc74a9a" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:34.900298 19830 raft_consensus.cc:2468] T 505a24ebe7014f80b3c184329cc74a9a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:34.900655 19277 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 45ab2817d7e84185b7ec2bfddcf4997f (table default.Table_8b37ba6f398943df8ab95b19f0b7dec6 [id=9cb2dc12a91e4de087414b492cd0dcfb]). Expected version 1 got 0
I20260925 13:56:34.900451 19802 tablet_service.cc:1511] Processing CreateTablet for tablet acf234ea540a442c9ed3d8a4291c0b58 (DEFAULT_TABLE table=default.Table_8189c34636b04c0881a90a5311af243e [id=b12f135fa98a473e9279eb9844872089]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:34.901911 19802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acf234ea540a442c9ed3d8a4291c0b58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:34.903568 19671 tablet_service.cc:1558] Processing DeleteTablet for tablet eff33a56361946edb556ac611d2299eb with delete_type TABLET_DATA_DELETED (replaced by tablet 45a578a5d0d94488b72b841a3057da69) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:34.904064 20644 tablet_replica.cc:333] T eff33a56361946edb556ac611d2299eb P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:34.904241 21055 tablet_replica.cc:333] T eff33a56361946edb556ac611d2299eb P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:34.904803 20644 raft_consensus.cc:2243] T eff33a56361946edb556ac611d2299eb P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:34.904978 21055 raft_consensus.cc:2243] T eff33a56361946edb556ac611d2299eb P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:34.905251 20644 raft_consensus.cc:2272] T eff33a56361946edb556ac611d2299eb P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:34.905274 19277 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet e4b29a3d0edc481cba6c61cfb78f8f40 (table default.Table_240f5ce2f31541eda914442e2cab887a [id=45fc848c759b4365a5c69016b6aa3309]). Expected version 1 got 0
I20260925 13:56:34.905428 21055 raft_consensus.cc:2272] T eff33a56361946edb556ac611d2299eb P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:34.905830 19277 catalog_manager.cc:5838] T 7e544c93488541da9ebb588a465b27b7 P f790493ba7ad403993b678f2df2b55be reported cstate change: term changed from 0 to 1, leader changed from <none> to f790493ba7ad403993b678f2df2b55be (127.18.182.195). New cstate: current_term: 1 leader_uuid: "f790493ba7ad403993b678f2df2b55be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:34.907471 20644 ts_tablet_manager.cc:1916] T eff33a56361946edb556ac611d2299eb P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:34.908702 19938 tablet_bootstrap.cc:492] T a0109e4115174d8eb8174a1a2236150f P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:34.909065 19938 ts_tablet_manager.cc:1403] T a0109e4115174d8eb8174a1a2236150f P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.240s user 0.028s sys 0.041s
I20260925 13:56:34.906984 19277 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 7e544c93488541da9ebb588a465b27b7 (table default.Table_4f849dc343f5448eb14a195bb0777d12 [id=f71e790b94b34e3d88f7539e396bc94d]). Expected version 1 got 0
I20260925 13:56:34.911787 19692 raft_consensus.cc:2468] T 72183473a5c94629a58c2cd8e1c56f32 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f790493ba7ad403993b678f2df2b55be in term 1.
I20260925 13:56:34.911803 21055 ts_tablet_manager.cc:1916] T eff33a56361946edb556ac611d2299eb P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:34.912104 19277 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 540132db164c403da72b9377078c4bf0 (table default.Table_5321051e9f6e4cd28b9b77619c908dc9 [id=9dac142084c948f9a50cfc9acf24d461]). Expected version 2 got 0
I20260925 13:56:34.913895 19277 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet de2b3e07bd634bc08368989a9062ba08 (table default.Table_cd8babe4e16340b18a41ba2e3aa63cd8 [id=be2ddd2647094a8788d52ef46500426f]). Expected version 1 got 0
I20260925 13:56:34.918367 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet c5a9df833271480aa49feb7d9f925459 (table default.Table_25c2c368343741a3923a23f0a7cb2b62 [id=7d4523c5927040ba846c35a4b02172a7]) successfully deleted
I20260925 13:56:34.911290 19938 raft_consensus.cc:359] T a0109e4115174d8eb8174a1a2236150f P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:34.919556 19671 tablet_service.cc:1511] Processing CreateTablet for tablet c5efc89d491f45c3be0ef84da1142d1c (DEFAULT_TABLE table=default.Table_9f3ab8123a784bceb2bebdf1798fb643 [id=fe1090b1dc564b0b82a37f7b9b746660]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:34.919896 19938 raft_consensus.cc:385] T a0109e4115174d8eb8174a1a2236150f P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:34.920140 19938 raft_consensus.cc:740] T a0109e4115174d8eb8174a1a2236150f P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:34.920442 19671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5efc89d491f45c3be0ef84da1142d1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:34.922313 19666 tablet_service.cc:1511] Processing CreateTablet for tablet a426f65001f3420fb5b3672c57ae53d1 (DEFAULT_TABLE table=default.Table_9ae8fc862af64663b8f5f175a4085a03 [id=fa78293b520f449fa0e961b2a14a373b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:34.923455 19666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a426f65001f3420fb5b3672c57ae53d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:34.931684 19938 consensus_queue.cc:260] T a0109e4115174d8eb8174a1a2236150f P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:34.933954 19669 tablet_service.cc:1511] Processing CreateTablet for tablet e7bad034cd324571922567126188de9d (DEFAULT_TABLE table=default.Table_86e65c13d8024eba9016537e6f430289 [id=5e48591aeea646359f00fb7b86009efa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:34.934743 19938 ts_tablet_manager.cc:1434] T a0109e4115174d8eb8174a1a2236150f P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.025s user 0.005s sys 0.000s
I20260925 13:56:34.934804 19669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7bad034cd324571922567126188de9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:34.942051 20993 leader_election.cc:290] T 11077cdb735840b5921e22f588059149 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:34.942989 19830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11077cdb735840b5921e22f588059149" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:34.943565 19830 raft_consensus.cc:2468] T 11077cdb735840b5921e22f588059149 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
W20260925 13:56:34.945662 19266 catalog_manager.cc:5043] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): delete failed for tablet 3d754e17930549189d65becb98afe2a6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3d754e17930549189d65becb98afe2a6 already in progress: creating tablet
I20260925 13:56:34.946889 19814 tablet_service.cc:1511] Processing CreateTablet for tablet c5efc89d491f45c3be0ef84da1142d1c (DEFAULT_TABLE table=default.Table_9f3ab8123a784bceb2bebdf1798fb643 [id=fe1090b1dc564b0b82a37f7b9b746660]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:34.948261 19814 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5efc89d491f45c3be0ef84da1142d1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:34.956130 19631 leader_election.cc:304] T 505a24ebe7014f80b3c184329cc74a9a P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:34.961129 19272 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 7e544c93488541da9ebb588a465b27b7 (table default.Table_4f849dc343f5448eb14a195bb0777d12 [id=f71e790b94b34e3d88f7539e396bc94d]). Expected version 1 got 0
I20260925 13:56:34.961949 19272 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet b23948807b2c4df9bf9a4da5f767e3b5 (table default.Table_fd205659579c49559fd2d121b32274d4 [id=0165d602da03423f82104b16da9a20b0]). Expected version 1 got 0
I20260925 13:56:34.962641 21058 consensus_queue.cc:1048] T 82c1002d2bcb4b11839e2f4a93b899ad P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:34.963522 19938 tablet_bootstrap.cc:492] T fd448eab74b344838b3b92cad83a1294 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:34.964053 21058 raft_consensus.cc:2804] T 505a24ebe7014f80b3c184329cc74a9a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:34.964438 21058 raft_consensus.cc:493] T 505a24ebe7014f80b3c184329cc74a9a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:34.964803 21058 raft_consensus.cc:3060] T 505a24ebe7014f80b3c184329cc74a9a P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:34.973250 19938 tablet_bootstrap.cc:654] T fd448eab74b344838b3b92cad83a1294 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:34.982182 19272 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet f3b46e4fac694084b06b7086bcc596b9 (table default.Table_2efda975ab0a464eaf96f1f9a48eff1c [id=c31f813d79a244afa726df0238376aba]). Expected version 1 got 0
I20260925 13:56:34.987751 19674 tablet_service.cc:1511] Processing CreateTablet for tablet f188ed756b3e456a86b95ca04f759d68 (DEFAULT_TABLE table=default.Table_f3eb4354192a4817a78558d8a6e64673 [id=b4cbf71a783c42d6978a01b7d0e8633c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:34.988684 19674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f188ed756b3e456a86b95ca04f759d68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:34.992200 19773 leader_election.cc:304] T 72183473a5c94629a58c2cd8e1c56f32 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:34.992863 20891 raft_consensus.cc:2804] T 72183473a5c94629a58c2cd8e1c56f32 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:34.993283 20891 raft_consensus.cc:697] T 72183473a5c94629a58c2cd8e1c56f32 P f790493ba7ad403993b678f2df2b55be [term 1 LEADER]: Becoming Leader. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Running, Role: LEADER
I20260925 13:56:34.994005 20891 consensus_queue.cc:237] T 72183473a5c94629a58c2cd8e1c56f32 P f790493ba7ad403993b678f2df2b55be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:34.994889 19631 leader_election.cc:304] T 11077cdb735840b5921e22f588059149 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:34.995288 19817 tablet_service.cc:1511] Processing CreateTablet for tablet e7bad034cd324571922567126188de9d (DEFAULT_TABLE table=default.Table_86e65c13d8024eba9016537e6f430289 [id=5e48591aeea646359f00fb7b86009efa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:34.996244 19817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7bad034cd324571922567126188de9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:35.002427 21062 raft_consensus.cc:2804] T 11077cdb735840b5921e22f588059149 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:35.002941 21062 raft_consensus.cc:493] T 11077cdb735840b5921e22f588059149 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:35.005705 19830 raft_consensus.cc:1275] T e9dbcd85dc8647c68c120ba3fcfe1790 P f790493ba7ad403993b678f2df2b55be [term 2 FOLLOWER]: Refusing update from remote peer 7c1b52895a14482fb54a6758f5054f8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260925 13:56:35.003582 19668 tablet_service.cc:1558] Processing DeleteTablet for tablet 3d754e17930549189d65becb98afe2a6 with delete_type TABLET_DATA_DELETED (replaced by tablet fe5068f8a43a4474ac189b522fdc494e) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:35.018199 19942 tablet_bootstrap.cc:492] T 45ab2817d7e84185b7ec2bfddcf4997f P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:35.024916 19938 tablet_bootstrap.cc:492] T fd448eab74b344838b3b92cad83a1294 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:35.025341 19938 ts_tablet_manager.cc:1403] T fd448eab74b344838b3b92cad83a1294 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.062s user 0.009s sys 0.000s
I20260925 13:56:35.024819 19818 tablet_service.cc:1511] Processing CreateTablet for tablet a426f65001f3420fb5b3672c57ae53d1 (DEFAULT_TABLE table=default.Table_9ae8fc862af64663b8f5f175a4085a03 [id=fa78293b520f449fa0e961b2a14a373b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:35.030119 19818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a426f65001f3420fb5b3672c57ae53d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:34.984510 19272 catalog_manager.cc:5838] T 82c1002d2bcb4b11839e2f4a93b899ad P 7c1b52895a14482fb54a6758f5054f8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194). New cstate: current_term: 1 leader_uuid: "7c1b52895a14482fb54a6758f5054f8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:35.044155 19272 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 505a24ebe7014f80b3c184329cc74a9a (table default.Table_f071a5a526034801828291eb010c40c0 [id=2e58540132af4ed298c50ddea3540cdf]). Expected version 1 got 0
I20260925 13:56:35.044802 19272 catalog_manager.cc:5838] T e9dbcd85dc8647c68c120ba3fcfe1790 P 7c1b52895a14482fb54a6758f5054f8f reported cstate change: term changed from 1 to 2, leader changed from 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193) to 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194). New cstate: current_term: 2 leader_uuid: "7c1b52895a14482fb54a6758f5054f8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:35.046084 19272 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet e9dbcd85dc8647c68c120ba3fcfe1790 (table default.Table_5bf9b87b5083469fad04eb3a80a4b8d5 [id=443536a0bd4d4d568d52f6ffe3d06ecb]). Expected version 1 got 0
I20260925 13:56:35.046695 19272 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet b3172d99afee4a4c878e71b645037917 (table default.Table_bd52a86b8ff14bdbb06fcf88a60b1e82 [id=203024d67e444f839391c8670d7c7d3b]). Expected version 2 got 0
I20260925 13:56:35.047257 19272 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 11077cdb735840b5921e22f588059149 (table default.Table_e4f4f8c05f904aa485bd1b4c5a96d9c8 [id=c219cb8664c64ce5aba0ed135c171937]). Expected version 1 got 0
I20260925 13:56:35.047804 19272 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 31b5f47966a74ec3b59c7c20479ba702 (table default.Table_bd9c5f2059394324857aa7fef30188f7 [id=e30a4ca9b0bf4c11a6691a896304c57e]). Expected version 1 got 0
I20260925 13:56:35.041010 19938 raft_consensus.cc:359] T fd448eab74b344838b3b92cad83a1294 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:35.055312 19938 raft_consensus.cc:385] T fd448eab74b344838b3b92cad83a1294 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:35.055593 19938 raft_consensus.cc:740] T fd448eab74b344838b3b92cad83a1294 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:35.056241 19938 consensus_queue.cc:260] T fd448eab74b344838b3b92cad83a1294 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:35.058072 19938 ts_tablet_manager.cc:1434] T fd448eab74b344838b3b92cad83a1294 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.032s user 0.005s sys 0.000s
I20260925 13:56:35.058926 19938 tablet_bootstrap.cc:492] T 9c5ef5b050624127990737acab3a8e9f P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:35.054140 19819 tablet_service.cc:1511] Processing CreateTablet for tablet f188ed756b3e456a86b95ca04f759d68 (DEFAULT_TABLE table=default.Table_f3eb4354192a4817a78558d8a6e64673 [id=b4cbf71a783c42d6978a01b7d0e8633c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:35.064421 19809 tablet_service.cc:1511] Processing CreateTablet for tablet 3927adc85b2746d0977a3cb6f84ad68f (DEFAULT_TABLE table=default.Table_3a3e48a0641a43a1a783c6ba6374a708 [id=e64fe44a237f48859a288a87a565ee5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:35.065071 19819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f188ed756b3e456a86b95ca04f759d68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:35.065577 19809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3927adc85b2746d0977a3cb6f84ad68f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:35.064415 19938 tablet_bootstrap.cc:654] T 9c5ef5b050624127990737acab3a8e9f P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:35.069725 19816 tablet_service.cc:1511] Processing CreateTablet for tablet e05204d176c146ec831aa5d7271654f2 (DEFAULT_TABLE table=default.Table_5d37f80782a44056aa57b74959fd9c78 [id=39884152b9b04243a1e6ef43b13ccd0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:35.076629 19816 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e05204d176c146ec831aa5d7271654f2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:56:35.086494 19264 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 3d754e17930549189d65becb98afe2a6 (table default.Table_6dd81ec6a73d459581359a7c93b23f9c [id=d2bc135460ab4f2fad4740a281bfc6e2], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet 3d754e17930549189d65becb98afe2a6 consensus state has no leader
I20260925 13:56:35.087409 19277 catalog_manager.cc:6391] T 00000000000000000000000000000000 P bcc92728d49643efab5528d77c3d7828: default.Table_3a06919b658e46f893318b4e1ec8b52f [id=e6f2a872536b4e55b071231020903581] alter complete (version 1)
W20260925 13:56:35.090744 19277 catalog_manager.cc:6038] Failed to send AlterTable RPC for tablet 540132db164c403da72b9377078c4bf0 (table default.Table_5321051e9f6e4cd28b9b77619c908dc9 [id=9dac142084c948f9a50cfc9acf24d461], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 540132db164c403da72b9377078c4bf0 consensus state has no leader
I20260925 13:56:35.091485 19668 tablet_service.cc:1511] Processing CreateTablet for tablet e05204d176c146ec831aa5d7271654f2 (DEFAULT_TABLE table=default.Table_5d37f80782a44056aa57b74959fd9c78 [id=39884152b9b04243a1e6ef43b13ccd0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:35.092468 19668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e05204d176c146ec831aa5d7271654f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:35.093345 19675 tablet_service.cc:1511] Processing CreateTablet for tablet 3927adc85b2746d0977a3cb6f84ad68f (DEFAULT_TABLE table=default.Table_3a3e48a0641a43a1a783c6ba6374a708 [id=e64fe44a237f48859a288a87a565ee5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:35.094251 19675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3927adc85b2746d0977a3cb6f84ad68f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:35.104462 20644 ts_tablet_manager.cc:1929] T eff33a56361946edb556ac611d2299eb P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:56:35.104873 20644 log.cc:1199] T eff33a56361946edb556ac611d2299eb P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/eff33a56361946edb556ac611d2299eb
I20260925 13:56:35.105662 20644 ts_tablet_manager.cc:1950] T eff33a56361946edb556ac611d2299eb P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:35.112352 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet eff33a56361946edb556ac611d2299eb (table default.Table_33e9cd401c9d45d0a1af408bda5e3bfa [id=2c74fb31a35541c4a2dd0b2171ba41ce]) successfully deleted
I20260925 13:56:35.115801 19276 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet a0109e4115174d8eb8174a1a2236150f (table default.Table_be48ddb268274ca685a6b7995ec4f0ee [id=af68d1e4a6874f999362917b2feb6efe]). Expected version 2 got 0
I20260925 13:56:35.118803 19938 tablet_bootstrap.cc:492] T 9c5ef5b050624127990737acab3a8e9f P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:35.119201 19938 ts_tablet_manager.cc:1403] T 9c5ef5b050624127990737acab3a8e9f P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.060s user 0.013s sys 0.004s
I20260925 13:56:35.120764 19819 tablet_service.cc:1558] Processing DeleteTablet for tablet 72183473a5c94629a58c2cd8e1c56f32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:24 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:35.122295 19938 raft_consensus.cc:359] T 9c5ef5b050624127990737acab3a8e9f P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:35.123047 19938 raft_consensus.cc:385] T 9c5ef5b050624127990737acab3a8e9f P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:35.123446 19938 raft_consensus.cc:740] T 9c5ef5b050624127990737acab3a8e9f P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:35.125535 19938 consensus_queue.cc:260] T 9c5ef5b050624127990737acab3a8e9f P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:35.127977 19938 ts_tablet_manager.cc:1434] T 9c5ef5b050624127990737acab3a8e9f P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.008s user 0.005s sys 0.001s
I20260925 13:56:35.128897 19938 tablet_bootstrap.cc:492] T 869bc08603b444b4885603d81b97a1db P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:35.023674 21062 raft_consensus.cc:3060] T 11077cdb735840b5921e22f588059149 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:35.138657 19938 tablet_bootstrap.cc:654] T 869bc08603b444b4885603d81b97a1db P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:35.139176 20644 tablet_replica.cc:333] T 72183473a5c94629a58c2cd8e1c56f32 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:35.146895 20644 raft_consensus.cc:2243] T 72183473a5c94629a58c2cd8e1c56f32 P f790493ba7ad403993b678f2df2b55be [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:56:35.147662 20644 pending_rounds.cc:70] T 72183473a5c94629a58c2cd8e1c56f32 P f790493ba7ad403993b678f2df2b55be: Trying to abort 1 pending ops.
I20260925 13:56:35.147881 21055 ts_tablet_manager.cc:1929] T eff33a56361946edb556ac611d2299eb P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:56:35.147935 20644 pending_rounds.cc:77] T 72183473a5c94629a58c2cd8e1c56f32 P f790493ba7ad403993b678f2df2b55be: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251461104271360 op_type: NO_OP noop_request { }
I20260925 13:56:35.148319 21055 log.cc:1199] T eff33a56361946edb556ac611d2299eb P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/eff33a56361946edb556ac611d2299eb
I20260925 13:56:35.148388 20644 raft_consensus.cc:2889] T 72183473a5c94629a58c2cd8e1c56f32 P f790493ba7ad403993b678f2df2b55be [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:56:35.148809 20644 raft_consensus.cc:2272] T 72183473a5c94629a58c2cd8e1c56f32 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:35.149113 21055 ts_tablet_manager.cc:1950] T eff33a56361946edb556ac611d2299eb P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:35.151851 20644 ts_tablet_manager.cc:1916] T 72183473a5c94629a58c2cd8e1c56f32 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 13:56:35.151973 19631 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 157154us
I20260925 13:56:34.971318 21058 raft_consensus.cc:515] T 505a24ebe7014f80b3c184329cc74a9a P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:35.163028 19814 tablet_service.cc:1511] Processing CreateTablet for tablet 6f8c55dc693e4b4d8bfdcc18160145ef (DEFAULT_TABLE table=default.Table_3dab1a66e22c4656b3c8eff23a8b5f0e [id=4e583562c98544e488dc96553f53f6b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:35.169998 19938 tablet_bootstrap.cc:492] T 869bc08603b444b4885603d81b97a1db P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:35.170482 19938 ts_tablet_manager.cc:1403] T 869bc08603b444b4885603d81b97a1db P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.042s user 0.004s sys 0.005s
I20260925 13:56:35.173471 19938 raft_consensus.cc:359] T 869bc08603b444b4885603d81b97a1db P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:35.174202 19938 raft_consensus.cc:385] T 869bc08603b444b4885603d81b97a1db P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:35.174510 19938 raft_consensus.cc:740] T 869bc08603b444b4885603d81b97a1db P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:35.174389 20993 consensus_queue.cc:1048] T e9dbcd85dc8647c68c120ba3fcfe1790 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:35.175222 19938 consensus_queue.cc:260] T 869bc08603b444b4885603d81b97a1db P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:35.179039 21058 leader_election.cc:290] T 505a24ebe7014f80b3c184329cc74a9a P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:35.179273 19942 tablet_bootstrap.cc:654] T 45ab2817d7e84185b7ec2bfddcf4997f P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:35.181187 19938 ts_tablet_manager.cc:1434] T 869bc08603b444b4885603d81b97a1db P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.010s user 0.008s sys 0.000s
I20260925 13:56:35.183458 19802 tablet_service.cc:1511] Processing CreateTablet for tablet a22335bf08cf4f77be1bfe3127ef6e5c (DEFAULT_TABLE table=default.Table_6ad3c828fd44418fbe6f3b0234f8f6f4 [id=09984b5c39d7432682ce19681468f11e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:35.197163 20644 ts_tablet_manager.cc:1929] T 72183473a5c94629a58c2cd8e1c56f32 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:56:35.197607 20644 log.cc:1199] T 72183473a5c94629a58c2cd8e1c56f32 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/72183473a5c94629a58c2cd8e1c56f32
I20260925 13:56:35.198325 20644 ts_tablet_manager.cc:1950] T 72183473a5c94629a58c2cd8e1c56f32 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:35.206859 21062 raft_consensus.cc:515] T 11077cdb735840b5921e22f588059149 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:35.214356 21062 leader_election.cc:290] T 11077cdb735840b5921e22f588059149 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:35.218894 19814 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f8c55dc693e4b4d8bfdcc18160145ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:35.233554 19819 tablet_service.cc:1558] Processing DeleteTablet for tablet 9c5ef5b050624127990737acab3a8e9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:31 UTC) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:35.241891 19942 tablet_bootstrap.cc:492] T 45ab2817d7e84185b7ec2bfddcf4997f P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:35.242333 19942 ts_tablet_manager.cc:1403] T 45ab2817d7e84185b7ec2bfddcf4997f P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.224s user 0.014s sys 0.001s
I20260925 13:56:35.245841 19942 raft_consensus.cc:359] T 45ab2817d7e84185b7ec2bfddcf4997f P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:35.246707 19942 raft_consensus.cc:385] T 45ab2817d7e84185b7ec2bfddcf4997f P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:35.246968 19942 raft_consensus.cc:740] T 45ab2817d7e84185b7ec2bfddcf4997f P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:35.247581 19942 consensus_queue.cc:260] T 45ab2817d7e84185b7ec2bfddcf4997f P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:35.256177 20644 tablet_replica.cc:333] T 9c5ef5b050624127990737acab3a8e9f P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:35.256937 20644 raft_consensus.cc:2243] T 9c5ef5b050624127990737acab3a8e9f P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:35.257352 20644 raft_consensus.cc:2272] T 9c5ef5b050624127990737acab3a8e9f P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:35.258409 19802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a22335bf08cf4f77be1bfe3127ef6e5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:35.264041 19938 tablet_bootstrap.cc:492] T 80ded435c38e4a84a639cd9dcad25cde P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:35.268877 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 72183473a5c94629a58c2cd8e1c56f32 (table default.Table_0b05542218754c9c8bbf04c76ba736b5 [id=5965407f056548edadae1a81a5c56103]) successfully deleted
I20260925 13:56:35.268997 20891 raft_consensus.cc:493] T de2b3e07bd634bc08368989a9062ba08 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:35.269408 20891 raft_consensus.cc:515] T de2b3e07bd634bc08368989a9062ba08 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:35.270630 20644 ts_tablet_manager.cc:1916] T 9c5ef5b050624127990737acab3a8e9f P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:35.271543 20891 leader_election.cc:290] T de2b3e07bd634bc08368989a9062ba08 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:35.273850 19938 tablet_bootstrap.cc:654] T 80ded435c38e4a84a639cd9dcad25cde P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:35.287582 19942 ts_tablet_manager.cc:1434] T 45ab2817d7e84185b7ec2bfddcf4997f P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.045s user 0.009s sys 0.000s
I20260925 13:56:35.288475 19942 tablet_bootstrap.cc:492] T 37947005aaf14f27a6e5d1f30c67250c P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:35.289074 20891 raft_consensus.cc:493] T a0109e4115174d8eb8174a1a2236150f P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:35.289568 20891 raft_consensus.cc:515] T a0109e4115174d8eb8174a1a2236150f P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:35.292296 20891 leader_election.cc:290] T a0109e4115174d8eb8174a1a2236150f P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:35.295264 19938 tablet_bootstrap.cc:492] T 80ded435c38e4a84a639cd9dcad25cde P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:35.295691 19938 ts_tablet_manager.cc:1403] T 80ded435c38e4a84a639cd9dcad25cde P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.032s user 0.008s sys 0.001s
I20260925 13:56:35.297952 19938 raft_consensus.cc:359] T 80ded435c38e4a84a639cd9dcad25cde P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:35.298527 19938 raft_consensus.cc:385] T 80ded435c38e4a84a639cd9dcad25cde P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:35.298766 19938 raft_consensus.cc:740] T 80ded435c38e4a84a639cd9dcad25cde P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:35.301352 20644 ts_tablet_manager.cc:1929] T 9c5ef5b050624127990737acab3a8e9f P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:35.301720 20644 log.cc:1199] T 9c5ef5b050624127990737acab3a8e9f P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/9c5ef5b050624127990737acab3a8e9f
I20260925 13:56:35.302464 20644 ts_tablet_manager.cc:1950] T 9c5ef5b050624127990737acab3a8e9f P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:35.299340 19938 consensus_queue.cc:260] T 80ded435c38e4a84a639cd9dcad25cde P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:35.306932 19938 ts_tablet_manager.cc:1434] T 80ded435c38e4a84a639cd9dcad25cde P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20260925 13:56:35.318873 19938 tablet_bootstrap.cc:492] T 69934b9a49354c92b5b13e182247ef77 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:35.319991 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 9c5ef5b050624127990737acab3a8e9f (table default.Table_8a9f077513e94c6d985f3c86954c3ccd [id=64639a1e491042f7b89cd60d23d68a6c]) successfully deleted
I20260925 13:56:35.324280 19938 tablet_bootstrap.cc:654] T 69934b9a49354c92b5b13e182247ef77 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
W20260925 13:56:35.378405 19631 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 214032us
I20260925 13:56:35.382800 19833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "505a24ebe7014f80b3c184329cc74a9a" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:35.383337 19833 raft_consensus.cc:3060] T 505a24ebe7014f80b3c184329cc74a9a P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:35.387549 19836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11077cdb735840b5921e22f588059149" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:35.388259 19836 raft_consensus.cc:3060] T 11077cdb735840b5921e22f588059149 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:35.392083 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet eff33a56361946edb556ac611d2299eb (table default.Table_33e9cd401c9d45d0a1af408bda5e3bfa [id=2c74fb31a35541c4a2dd0b2171ba41ce]) successfully deleted
W20260925 13:56:35.392952 19264 catalog_manager.cc:5043] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 3d754e17930549189d65becb98afe2a6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3d754e17930549189d65becb98afe2a6 already in progress: creating tablet
I20260925 13:56:35.397006 20891 raft_consensus.cc:493] T 869bc08603b444b4885603d81b97a1db P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:35.397099 19833 raft_consensus.cc:2468] T 505a24ebe7014f80b3c184329cc74a9a P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
I20260925 13:56:35.397567 20891 raft_consensus.cc:515] T 869bc08603b444b4885603d81b97a1db P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:35.400148 19836 raft_consensus.cc:2468] T 11077cdb735840b5921e22f588059149 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
I20260925 13:56:35.401321 19938 tablet_bootstrap.cc:492] T 69934b9a49354c92b5b13e182247ef77 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:35.401777 19938 ts_tablet_manager.cc:1403] T 69934b9a49354c92b5b13e182247ef77 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.083s user 0.015s sys 0.021s
I20260925 13:56:35.404032 20891 leader_election.cc:290] T 869bc08603b444b4885603d81b97a1db P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:35.405985 19938 raft_consensus.cc:359] T 69934b9a49354c92b5b13e182247ef77 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:35.406689 19938 raft_consensus.cc:385] T 69934b9a49354c92b5b13e182247ef77 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:35.406996 19938 raft_consensus.cc:740] T 69934b9a49354c92b5b13e182247ef77 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:35.407704 19938 consensus_queue.cc:260] T 69934b9a49354c92b5b13e182247ef77 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:35.410211 19938 ts_tablet_manager.cc:1434] T 69934b9a49354c92b5b13e182247ef77 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.008s user 0.002s sys 0.003s
I20260925 13:56:35.411729 21063 raft_consensus.cc:493] T a218f033a132482a82bd1f60126fa0f9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:35.412670 21063 raft_consensus.cc:515] T a218f033a132482a82bd1f60126fa0f9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:35.429705 19942 tablet_bootstrap.cc:654] T 37947005aaf14f27a6e5d1f30c67250c P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:35.452270 21063 leader_election.cc:290] T a218f033a132482a82bd1f60126fa0f9 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:35.419304 21066 raft_consensus.cc:493] T f146a9aee3054d7491e74101df0e28e1 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:35.453017 21066 raft_consensus.cc:515] T f146a9aee3054d7491e74101df0e28e1 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:35.454989 21066 leader_election.cc:290] T f146a9aee3054d7491e74101df0e28e1 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:35.459869 19938 tablet_bootstrap.cc:492] T ab1ba6ef73a046b59ffac150f7c73953 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:35.462268 20891 raft_consensus.cc:493] T fd448eab74b344838b3b92cad83a1294 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:35.462749 20891 raft_consensus.cc:515] T fd448eab74b344838b3b92cad83a1294 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:35.465219 20891 leader_election.cc:290] T fd448eab74b344838b3b92cad83a1294 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:35.478099 19938 tablet_bootstrap.cc:654] T ab1ba6ef73a046b59ffac150f7c73953 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:35.486205 19942 tablet_bootstrap.cc:492] T 37947005aaf14f27a6e5d1f30c67250c P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:35.486706 19942 ts_tablet_manager.cc:1403] T 37947005aaf14f27a6e5d1f30c67250c P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.198s user 0.003s sys 0.012s
I20260925 13:56:35.490350 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de2b3e07bd634bc08368989a9062ba08" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:35.496065 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0109e4115174d8eb8174a1a2236150f" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:35.500725 19694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "869bc08603b444b4885603d81b97a1db" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:35.502066 19694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd448eab74b344838b3b92cad83a1294" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
I20260925 13:56:35.504185 19666 tablet_service.cc:1511] Processing CreateTablet for tablet 6f8c55dc693e4b4d8bfdcc18160145ef (DEFAULT_TABLE table=default.Table_3dab1a66e22c4656b3c8eff23a8b5f0e [id=4e583562c98544e488dc96553f53f6b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:35.505003 19668 tablet_service.cc:1558] Processing DeleteTablet for tablet 78d8c01c88d24deeae21d647e7f205ba with delete_type TABLET_DATA_DELETED (replaced by tablet ab1ba6ef73a046b59ffac150f7c73953) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:35.505748 19666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f8c55dc693e4b4d8bfdcc18160145ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:35.508131 19675 tablet_service.cc:1511] Processing CreateTablet for tablet a22335bf08cf4f77be1bfe3127ef6e5c (DEFAULT_TABLE table=default.Table_6ad3c828fd44418fbe6f3b0234f8f6f4 [id=09984b5c39d7432682ce19681468f11e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:35.509044 19675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a22335bf08cf4f77be1bfe3127ef6e5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:35.499017 19942 raft_consensus.cc:359] T 37947005aaf14f27a6e5d1f30c67250c P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:35.511438 19942 raft_consensus.cc:385] T 37947005aaf14f27a6e5d1f30c67250c P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:35.511704 19942 raft_consensus.cc:740] T 37947005aaf14f27a6e5d1f30c67250c P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:35.512355 19671 tablet_service.cc:1558] Processing DeleteTablet for tablet 72183473a5c94629a58c2cd8e1c56f32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:24 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:35.512333 19942 consensus_queue.cc:260] T 37947005aaf14f27a6e5d1f30c67250c P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:35.513161 21055 tablet_replica.cc:333] T 72183473a5c94629a58c2cd8e1c56f32 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:35.514125 21055 raft_consensus.cc:2243] T 72183473a5c94629a58c2cd8e1c56f32 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:35.515398 19671 tablet_service.cc:1558] Processing DeleteTablet for tablet c5a9df833271480aa49feb7d9f925459 with delete_type TABLET_DATA_DELETED (replaced by tablet 236e308d8694405cb2b874164fa16485) from {username='slave'} at 127.0.0.1:45948
W20260925 13:56:35.517429 19773 leader_election.cc:343] T de2b3e07bd634bc08368989a9062ba08 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:35.517884 19631 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 123908us
I20260925 13:56:35.525019 21055 raft_consensus.cc:2272] T 72183473a5c94629a58c2cd8e1c56f32 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:35.527177 21055 ts_tablet_manager.cc:1916] T 72183473a5c94629a58c2cd8e1c56f32 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 13:56:35.534281 19773 leader_election.cc:343] T a0109e4115174d8eb8174a1a2236150f P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:35.537400 19631 leader_election.cc:304] T 505a24ebe7014f80b3c184329cc74a9a P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:35.538247 19631 leader_election.cc:304] T 11077cdb735840b5921e22f588059149 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:35.539932 21073 raft_consensus.cc:2804] T 505a24ebe7014f80b3c184329cc74a9a P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:35.541303 20993 raft_consensus.cc:2804] T 11077cdb735840b5921e22f588059149 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:35.541857 20993 raft_consensus.cc:697] T 11077cdb735840b5921e22f588059149 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Becoming Leader. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Running, Role: LEADER
I20260925 13:56:35.542693 20993 consensus_queue.cc:237] T 11077cdb735840b5921e22f588059149 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
W20260925 13:56:35.535200 19264 catalog_manager.cc:5030] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet 78d8c01c88d24deeae21d647e7f205ba because the tablet was not found. No further retry: Not found: Tablet not found: 78d8c01c88d24deeae21d647e7f205ba
I20260925 13:56:35.552321 21073 raft_consensus.cc:697] T 505a24ebe7014f80b3c184329cc74a9a P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Becoming Leader. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Running, Role: LEADER
I20260925 13:56:35.553169 21073 consensus_queue.cc:237] T 505a24ebe7014f80b3c184329cc74a9a P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:35.558825 19942 ts_tablet_manager.cc:1434] T 37947005aaf14f27a6e5d1f30c67250c P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.072s user 0.005s sys 0.003s
I20260925 13:56:35.559824 19942 tablet_bootstrap.cc:492] T de2b3e07bd634bc08368989a9062ba08 P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:35.560631 19938 tablet_bootstrap.cc:492] T ab1ba6ef73a046b59ffac150f7c73953 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:35.560999 19938 ts_tablet_manager.cc:1403] T ab1ba6ef73a046b59ffac150f7c73953 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.101s user 0.006s sys 0.004s
W20260925 13:56:35.562821 19773 leader_election.cc:343] T 869bc08603b444b4885603d81b97a1db P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:56:35.563407 19773 leader_election.cc:343] T fd448eab74b344838b3b92cad83a1294 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:35.563268 19938 raft_consensus.cc:359] T ab1ba6ef73a046b59ffac150f7c73953 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:35.563974 19938 raft_consensus.cc:385] T ab1ba6ef73a046b59ffac150f7c73953 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:35.564255 19938 raft_consensus.cc:740] T ab1ba6ef73a046b59ffac150f7c73953 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:35.565027 19938 consensus_queue.cc:260] T ab1ba6ef73a046b59ffac150f7c73953 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:35.573589 19836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f146a9aee3054d7491e74101df0e28e1" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:35.574177 19836 raft_consensus.cc:2468] T f146a9aee3054d7491e74101df0e28e1 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:35.582389 19938 ts_tablet_manager.cc:1434] T ab1ba6ef73a046b59ffac150f7c73953 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.020s user 0.006s sys 0.001s
I20260925 13:56:35.583340 19938 tablet_bootstrap.cc:492] T fa8330217e794edf9e26a368d8b34685 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
W20260925 13:56:35.589088 19631 leader_election.cc:343] T a218f033a132482a82bd1f60126fa0f9 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): Not found: Tablet not found: a218f033a132482a82bd1f60126fa0f9
I20260925 13:56:35.590381 19631 leader_election.cc:304] T f146a9aee3054d7491e74101df0e28e1 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:35.591555 21062 raft_consensus.cc:2804] T f146a9aee3054d7491e74101df0e28e1 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:35.591920 21062 raft_consensus.cc:493] T f146a9aee3054d7491e74101df0e28e1 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:35.592209 21062 raft_consensus.cc:3060] T f146a9aee3054d7491e74101df0e28e1 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:35.608736 20993 raft_consensus.cc:493] T 45ab2817d7e84185b7ec2bfddcf4997f P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:35.609261 20993 raft_consensus.cc:515] T 45ab2817d7e84185b7ec2bfddcf4997f P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:35.606815 21062 raft_consensus.cc:515] T f146a9aee3054d7491e74101df0e28e1 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:35.622210 21090 raft_consensus.cc:493] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bcc92728d49643efab5528d77c3d7828)
I20260925 13:56:35.624070 21062 leader_election.cc:290] T f146a9aee3054d7491e74101df0e28e1 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:35.628453 19840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f146a9aee3054d7491e74101df0e28e1" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:35.629112 19840 raft_consensus.cc:3060] T f146a9aee3054d7491e74101df0e28e1 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:35.630496 19830 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45ab2817d7e84185b7ec2bfddcf4997f" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be" is_pre_election: true
I20260925 13:56:35.631497 19938 tablet_bootstrap.cc:654] T fa8330217e794edf9e26a368d8b34685 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:35.631532 19830 raft_consensus.cc:2468] T 45ab2817d7e84185b7ec2bfddcf4997f P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 0.
I20260925 13:56:35.617673 20993 leader_election.cc:290] T 45ab2817d7e84185b7ec2bfddcf4997f P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
I20260925 13:56:35.641036 19840 raft_consensus.cc:2468] T f146a9aee3054d7491e74101df0e28e1 P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
I20260925 13:56:35.645397 19631 leader_election.cc:304] T 45ab2817d7e84185b7ec2bfddcf4997f P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:35.647086 20993 raft_consensus.cc:2804] T 45ab2817d7e84185b7ec2bfddcf4997f P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:35.633381 21090 raft_consensus.cc:515] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 40825 } } peers { permanent_uuid: "bcc92728d49643efab5528d77c3d7828" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 39641 } } peers { permanent_uuid: "fc3953a2ba984a658630c423ec9b9728" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 36181 } }
I20260925 13:56:35.647470 20993 raft_consensus.cc:493] T 45ab2817d7e84185b7ec2bfddcf4997f P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:35.647511 19631 leader_election.cc:304] T f146a9aee3054d7491e74101df0e28e1 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:35.649103 21090 leader_election.cc:290] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7dcc50a8531144a69ab0cc00df1f0df4 (127.18.182.254:40825), bcc92728d49643efab5528d77c3d7828 (127.18.182.253:39641)
I20260925 13:56:35.649039 20891 consensus_queue.cc:1048] T 31b5f47966a74ec3b59c7c20479ba702 P f790493ba7ad403993b678f2df2b55be [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:35.651093 19215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fc3953a2ba984a658630c423ec9b9728" candidate_term: 2 candidate_status { last_received { term: 1 index: 430 } } ignore_live_leader: false dest_uuid: "7dcc50a8531144a69ab0cc00df1f0df4" is_pre_election: true
I20260925 13:56:35.653237 19290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fc3953a2ba984a658630c423ec9b9728" candidate_term: 2 candidate_status { last_received { term: 1 index: 430 } } ignore_live_leader: false dest_uuid: "bcc92728d49643efab5528d77c3d7828" is_pre_election: true
I20260925 13:56:35.654670 20697 leader_election.cc:304] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc3953a2ba984a658630c423ec9b9728; no voters: 7dcc50a8531144a69ab0cc00df1f0df4, bcc92728d49643efab5528d77c3d7828
I20260925 13:56:35.675539 19938 tablet_bootstrap.cc:492] T fa8330217e794edf9e26a368d8b34685 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:35.675995 19938 ts_tablet_manager.cc:1403] T fa8330217e794edf9e26a368d8b34685 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.093s user 0.009s sys 0.003s
I20260925 13:56:35.678398 19938 raft_consensus.cc:359] T fa8330217e794edf9e26a368d8b34685 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:35.678988 19938 raft_consensus.cc:385] T fa8330217e794edf9e26a368d8b34685 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:35.679237 19938 raft_consensus.cc:740] T fa8330217e794edf9e26a368d8b34685 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:35.679282 21090 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fc3953a2ba984a658630c423ec9b9728 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:56:35.679945 19938 consensus_queue.cc:260] T fa8330217e794edf9e26a368d8b34685 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:35.682335 19938 ts_tablet_manager.cc:1434] T fa8330217e794edf9e26a368d8b34685 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260925 13:56:35.683162 19938 tablet_bootstrap.cc:492] T 65ad0197267d47158751bc44c539b555 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:35.691002 20993 raft_consensus.cc:3060] T 45ab2817d7e84185b7ec2bfddcf4997f P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Advancing to term 1
W20260925 13:56:35.691828 19631 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.195:46649, user_credentials={real_user=slave}} blocked reactor thread for 44359.9us
I20260925 13:56:35.693370 19276 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 45ab2817d7e84185b7ec2bfddcf4997f (table default.Table_8b37ba6f398943df8ab95b19f0b7dec6 [id=9cb2dc12a91e4de087414b492cd0dcfb]). Expected version 1 got 0
I20260925 13:56:35.693526 19272 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 69934b9a49354c92b5b13e182247ef77 (table default.Table_49155edad5db40549fa76e9195967efc [id=50c3843b240b47e1bfd1bfd07e3d4875]). Expected version 1 got 0
I20260925 13:56:35.699819 19802 tablet_service.cc:1511] Processing CreateTablet for tablet d7af7d06a0ac4d3eaa2282d923dae669 (DEFAULT_TABLE table=default.Table_416e65530d694bc6b6f507558740dd0b [id=29ae3ac568a04871b7eccde682003852]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:35.702734 19802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7af7d06a0ac4d3eaa2282d923dae669. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:35.713980 19818 tablet_service.cc:1511] Processing CreateTablet for tablet c65fcfac935e4687b63c324b69d48bd6 (DEFAULT_TABLE table=default.Table_408aac5f0ca04822a4e768d814b70669 [id=fa1fbf7a9fea47d4b7c24e93d88bbdaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:35.716779 19276 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 69934b9a49354c92b5b13e182247ef77 (table default.Table_49155edad5db40549fa76e9195967efc [id=50c3843b240b47e1bfd1bfd07e3d4875]). Expected version 1 got 0
I20260925 13:56:35.717414 19276 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet ab1ba6ef73a046b59ffac150f7c73953 (table default.Table_17642b3f89974e5c97036637fe6d04d3 [id=4ca863b59dfd4410822e15705934b503]). Expected version 1 got 0
I20260925 13:56:35.719174 19814 tablet_service.cc:1511] Processing CreateTablet for tablet 31a0d1ea17b64dcc85a675439fc1661d (DEFAULT_TABLE table=default.Table_77f9464ff4694e999c6250b01465c6c0 [id=523c904f55d943f69845a0fcb94556a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:35.718319 19276 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet e9dbcd85dc8647c68c120ba3fcfe1790 (table default.Table_5bf9b87b5083469fad04eb3a80a4b8d5 [id=443536a0bd4d4d568d52f6ffe3d06ecb]). Expected version 1 got 0
I20260925 13:56:35.720444 19276 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet fa8330217e794edf9e26a368d8b34685 (table default.Table_0eb4fb3fbd9c4440afd9c10c8c318c56 [id=e633d43cec1d40f1b2df3ee09a1ca3d3]). Expected version 1 got 0
I20260925 13:56:35.717324 21062 raft_consensus.cc:2804] T f146a9aee3054d7491e74101df0e28e1 P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:35.729152 19666 tablet_service.cc:1511] Processing CreateTablet for tablet d7af7d06a0ac4d3eaa2282d923dae669 (DEFAULT_TABLE table=default.Table_416e65530d694bc6b6f507558740dd0b [id=29ae3ac568a04871b7eccde682003852]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:35.732458 19666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7af7d06a0ac4d3eaa2282d923dae669. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:35.729283 19671 tablet_service.cc:1558] Processing DeleteTablet for tablet a218f033a132482a82bd1f60126fa0f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:56:29 UTC) from {username='slave'} at 127.0.0.1:45948
I20260925 13:56:35.729809 19668 tablet_service.cc:1511] Processing CreateTablet for tablet c65fcfac935e4687b63c324b69d48bd6 (DEFAULT_TABLE table=default.Table_408aac5f0ca04822a4e768d814b70669 [id=fa1fbf7a9fea47d4b7c24e93d88bbdaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:35.736168 19674 tablet_service.cc:1511] Processing CreateTablet for tablet 31a0d1ea17b64dcc85a675439fc1661d (DEFAULT_TABLE table=default.Table_77f9464ff4694e999c6250b01465c6c0 [id=523c904f55d943f69845a0fcb94556a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:35.736923 19668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c65fcfac935e4687b63c324b69d48bd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:35.730438 21062 raft_consensus.cc:697] T f146a9aee3054d7491e74101df0e28e1 P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Becoming Leader. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Running, Role: LEADER
I20260925 13:56:35.741726 19674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31a0d1ea17b64dcc85a675439fc1661d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:35.742252 21062 consensus_queue.cc:237] T f146a9aee3054d7491e74101df0e28e1 P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } }
I20260925 13:56:35.728412 21055 ts_tablet_manager.cc:1929] T 72183473a5c94629a58c2cd8e1c56f32 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:35.744068 21055 log.cc:1199] T 72183473a5c94629a58c2cd8e1c56f32 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/72183473a5c94629a58c2cd8e1c56f32
I20260925 13:56:35.744915 21055 ts_tablet_manager.cc:1950] T 72183473a5c94629a58c2cd8e1c56f32 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:35.746713 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet 72183473a5c94629a58c2cd8e1c56f32 (table default.Table_0b05542218754c9c8bbf04c76ba736b5 [id=5965407f056548edadae1a81a5c56103]) successfully deleted
I20260925 13:56:35.749359 19818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c65fcfac935e4687b63c324b69d48bd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:35.751901 19814 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31a0d1ea17b64dcc85a675439fc1661d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:35.757179 20891 raft_consensus.cc:493] T fa8330217e794edf9e26a368d8b34685 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:35.757629 20891 raft_consensus.cc:515] T fa8330217e794edf9e26a368d8b34685 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:35.764213 20891 leader_election.cc:290] T fa8330217e794edf9e26a368d8b34685 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:35.773516 19692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa8330217e794edf9e26a368d8b34685" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:35.775740 19773 leader_election.cc:343] T fa8330217e794edf9e26a368d8b34685 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:35.723309 20993 raft_consensus.cc:515] T 45ab2817d7e84185b7ec2bfddcf4997f P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:35.788630 19938 tablet_bootstrap.cc:700] T 65ad0197267d47158751bc44c539b555 P f790493ba7ad403993b678f2df2b55be: Time spent opening tablet: real 0.101s user 0.004s sys 0.000s
I20260925 13:56:35.789245 19938 tablet_bootstrap.cc:654] T 65ad0197267d47158751bc44c539b555 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:35.791544 19942 tablet_bootstrap.cc:654] T de2b3e07bd634bc08368989a9062ba08 P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:35.798929 20993 leader_election.cc:290] T 45ab2817d7e84185b7ec2bfddcf4997f P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Requested vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
W20260925 13:56:35.800230 19264 catalog_manager.cc:5030] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): delete failed for tablet c5a9df833271480aa49feb7d9f925459 because the tablet was not found. No further retry: Not found: Tablet not found: c5a9df833271480aa49feb7d9f925459
I20260925 13:56:35.801106 21055 tablet_replica.cc:333] T a218f033a132482a82bd1f60126fa0f9 P 7c1b52895a14482fb54a6758f5054f8f: stopping tablet replica
I20260925 13:56:35.801353 19840 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45ab2817d7e84185b7ec2bfddcf4997f" candidate_uuid: "7c1b52895a14482fb54a6758f5054f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f790493ba7ad403993b678f2df2b55be"
I20260925 13:56:35.801914 19840 raft_consensus.cc:3060] T 45ab2817d7e84185b7ec2bfddcf4997f P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:35.802058 21055 raft_consensus.cc:2243] T a218f033a132482a82bd1f60126fa0f9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:35.802551 21055 raft_consensus.cc:2272] T a218f033a132482a82bd1f60126fa0f9 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:35.804387 21055 ts_tablet_manager.cc:1916] T a218f033a132482a82bd1f60126fa0f9 P 7c1b52895a14482fb54a6758f5054f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:35.813509 19840 raft_consensus.cc:2468] T 45ab2817d7e84185b7ec2bfddcf4997f P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c1b52895a14482fb54a6758f5054f8f in term 1.
I20260925 13:56:35.816071 19816 tablet_service.cc:1511] Processing CreateTablet for tablet 45a578a5d0d94488b72b841a3057da69 (DEFAULT_TABLE table=default.Table_33e9cd401c9d45d0a1af408bda5e3bfa [id=2c74fb31a35541c4a2dd0b2171ba41ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:35.820313 19816 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45a578a5d0d94488b72b841a3057da69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:35.824450 19674 tablet_service.cc:1511] Processing CreateTablet for tablet 45a578a5d0d94488b72b841a3057da69 (DEFAULT_TABLE table=default.Table_33e9cd401c9d45d0a1af408bda5e3bfa [id=2c74fb31a35541c4a2dd0b2171ba41ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:35.829597 19674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45a578a5d0d94488b72b841a3057da69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:35.831148 19631 leader_election.cc:304] T 45ab2817d7e84185b7ec2bfddcf4997f P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1b52895a14482fb54a6758f5054f8f, f790493ba7ad403993b678f2df2b55be; no voters:
I20260925 13:56:35.840726 19809 tablet_service.cc:1511] Processing CreateTablet for tablet fe5068f8a43a4474ac189b522fdc494e (DEFAULT_TABLE table=default.Table_6dd81ec6a73d459581359a7c93b23f9c [id=d2bc135460ab4f2fad4740a281bfc6e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:35.841920 19809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe5068f8a43a4474ac189b522fdc494e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:35.831959 20993 raft_consensus.cc:2804] T 45ab2817d7e84185b7ec2bfddcf4997f P 7c1b52895a14482fb54a6758f5054f8f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:56:35.844813 20993 raft_consensus.cc:697] T 45ab2817d7e84185b7ec2bfddcf4997f P 7c1b52895a14482fb54a6758f5054f8f [term 1 LEADER]: Becoming Leader. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Running, Role: LEADER
I20260925 13:56:35.844106 19666 tablet_service.cc:1511] Processing CreateTablet for tablet fe5068f8a43a4474ac189b522fdc494e (DEFAULT_TABLE table=default.Table_6dd81ec6a73d459581359a7c93b23f9c [id=d2bc135460ab4f2fad4740a281bfc6e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:35.845549 20993 consensus_queue.cc:237] T 45ab2817d7e84185b7ec2bfddcf4997f P 7c1b52895a14482fb54a6758f5054f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:35.846652 19666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe5068f8a43a4474ac189b522fdc494e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:35.847980 19675 tablet_service.cc:1511] Processing CreateTablet for tablet 11195738058044bbbcfaf33e6e625f88 (DEFAULT_TABLE table=default.Table_b1ea6241873b405d80c47c0d7cdb85b7 [id=6b03dba362094d318c6ec267c7378a34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:35.848896 19675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11195738058044bbbcfaf33e6e625f88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:35.874035 19802 tablet_service.cc:1511] Processing CreateTablet for tablet 11195738058044bbbcfaf33e6e625f88 (DEFAULT_TABLE table=default.Table_b1ea6241873b405d80c47c0d7cdb85b7 [id=6b03dba362094d318c6ec267c7378a34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:56:35.875211 19802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11195738058044bbbcfaf33e6e625f88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:35.889426 19276 catalog_manager.cc:5938] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649) does not have the latest schema for tablet 31b5f47966a74ec3b59c7c20479ba702 (table default.Table_bd9c5f2059394324857aa7fef30188f7 [id=e30a4ca9b0bf4c11a6691a896304c57e]). Expected version 1 got 0
I20260925 13:56:35.923148 19938 tablet_bootstrap.cc:492] T 65ad0197267d47158751bc44c539b555 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:35.924360 19938 ts_tablet_manager.cc:1403] T 65ad0197267d47158751bc44c539b555 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.241s user 0.010s sys 0.000s
I20260925 13:56:35.937803 20891 raft_consensus.cc:493] T 69934b9a49354c92b5b13e182247ef77 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:35.938237 20891 raft_consensus.cc:515] T 69934b9a49354c92b5b13e182247ef77 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:35.937999 19276 catalog_manager.cc:5838] T f146a9aee3054d7491e74101df0e28e1 P 7c1b52895a14482fb54a6758f5054f8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194). New cstate: current_term: 1 leader_uuid: "7c1b52895a14482fb54a6758f5054f8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: HEALTHY } } }
I20260925 13:56:35.940054 20891 leader_election.cc:290] T 69934b9a49354c92b5b13e182247ef77 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:35.940861 19276 catalog_manager.cc:5838] T 505a24ebe7014f80b3c184329cc74a9a P 7c1b52895a14482fb54a6758f5054f8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194). New cstate: current_term: 1 leader_uuid: "7c1b52895a14482fb54a6758f5054f8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: HEALTHY } } }
I20260925 13:56:35.941606 19938 raft_consensus.cc:359] T 65ad0197267d47158751bc44c539b555 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:35.942207 19938 raft_consensus.cc:385] T 65ad0197267d47158751bc44c539b555 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:35.942497 19938 raft_consensus.cc:740] T 65ad0197267d47158751bc44c539b555 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:35.943094 19938 consensus_queue.cc:260] T 65ad0197267d47158751bc44c539b555 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:35.946892 19694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69934b9a49354c92b5b13e182247ef77" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:35.948518 19773 leader_election.cc:343] T 69934b9a49354c92b5b13e182247ef77 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:35.954005 19276 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 505a24ebe7014f80b3c184329cc74a9a (table default.Table_f071a5a526034801828291eb010c40c0 [id=2e58540132af4ed298c50ddea3540cdf]). Expected version 1 got 0
I20260925 13:56:35.960351 19942 tablet_bootstrap.cc:492] T de2b3e07bd634bc08368989a9062ba08 P 7c1b52895a14482fb54a6758f5054f8f: No bootstrap required, opened a new log
I20260925 13:56:35.960752 19942 ts_tablet_manager.cc:1403] T de2b3e07bd634bc08368989a9062ba08 P 7c1b52895a14482fb54a6758f5054f8f: Time spent bootstrapping tablet: real 0.401s user 0.014s sys 0.022s
I20260925 13:56:35.963047 19942 raft_consensus.cc:359] T de2b3e07bd634bc08368989a9062ba08 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:35.964105 19942 raft_consensus.cc:385] T de2b3e07bd634bc08368989a9062ba08 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:35.958472 19276 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 31b5f47966a74ec3b59c7c20479ba702 (table default.Table_bd9c5f2059394324857aa7fef30188f7 [id=e30a4ca9b0bf4c11a6691a896304c57e]). Expected version 1 got 0
I20260925 13:56:35.964367 19942 raft_consensus.cc:740] T de2b3e07bd634bc08368989a9062ba08 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1b52895a14482fb54a6758f5054f8f, State: Initialized, Role: FOLLOWER
I20260925 13:56:35.965042 19942 consensus_queue.cc:260] T de2b3e07bd634bc08368989a9062ba08 P 7c1b52895a14482fb54a6758f5054f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:35.965655 21055 ts_tablet_manager.cc:1929] T a218f033a132482a82bd1f60126fa0f9 P 7c1b52895a14482fb54a6758f5054f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:35.966080 21055 log.cc:1199] T a218f033a132482a82bd1f60126fa0f9 P 7c1b52895a14482fb54a6758f5054f8f: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/a218f033a132482a82bd1f60126fa0f9
I20260925 13:56:35.966814 21055 ts_tablet_manager.cc:1950] T a218f033a132482a82bd1f60126fa0f9 P 7c1b52895a14482fb54a6758f5054f8f: Deleting consensus metadata
I20260925 13:56:35.964917 19276 catalog_manager.cc:5838] T 11077cdb735840b5921e22f588059149 P 7c1b52895a14482fb54a6758f5054f8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194). New cstate: current_term: 1 leader_uuid: "7c1b52895a14482fb54a6758f5054f8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: HEALTHY } } }
I20260925 13:56:35.968531 19276 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 11077cdb735840b5921e22f588059149 (table default.Table_e4f4f8c05f904aa485bd1b4c5a96d9c8 [id=c219cb8664c64ce5aba0ed135c171937]). Expected version 1 got 0
I20260925 13:56:35.968924 19942 ts_tablet_manager.cc:1434] T de2b3e07bd634bc08368989a9062ba08 P 7c1b52895a14482fb54a6758f5054f8f: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260925 13:56:35.995728 19938 ts_tablet_manager.cc:1434] T 65ad0197267d47158751bc44c539b555 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.071s user 0.005s sys 0.000s
W20260925 13:56:36.003592 19632 connection.cc:573] server connection from 127.18.182.193:58657 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 13:56:36.005976 19629 connection.cc:573] client connection to 127.18.182.193:38269 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 13:56:36.006417 19629 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 13:56:36.013729 20700 connection.cc:573] server connection from 127.18.182.193:60609 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 13:56:36.002911 19775 connection.cc:573] server connection from 127.18.182.193:58915 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 13:56:36.011842 20891 raft_consensus.cc:493] T ab1ba6ef73a046b59ffac150f7c73953 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:36.013428 19809 tablet_service.cc:1558] Processing DeleteTablet for tablet 65ad0197267d47158751bc44c539b555 with delete_type TABLET_DATA_DELETED (replaced by tablet f806a82ee59c4da79e2f30647adc3c60) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:36.019407 20891 raft_consensus.cc:515] T ab1ba6ef73a046b59ffac150f7c73953 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:36.020406 19942 tablet_bootstrap.cc:492] T a0109e4115174d8eb8174a1a2236150f P 7c1b52895a14482fb54a6758f5054f8f: Bootstrap starting.
I20260925 13:56:36.021148 20891 leader_election.cc:290] T ab1ba6ef73a046b59ffac150f7c73953 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:36.022919 19694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab1ba6ef73a046b59ffac150f7c73953" candidate_uuid: "f790493ba7ad403993b678f2df2b55be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1b52895a14482fb54a6758f5054f8f" is_pre_election: true
W20260925 13:56:36.024215 19773 leader_election.cc:343] T ab1ba6ef73a046b59ffac150f7c73953 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:56:36.026336 19264 catalog_manager.cc:5063] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): tablet a218f033a132482a82bd1f60126fa0f9 (table default.Table_67306a96449b4a7bbb4b96498a396cea [id=bcf9d7c5e4f44267b9204bc3c2d90584]) successfully deleted
I20260925 13:56:36.038916 19942 tablet_bootstrap.cc:654] T a0109e4115174d8eb8174a1a2236150f P 7c1b52895a14482fb54a6758f5054f8f: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:36.050936 19938 tablet_bootstrap.cc:492] T 43692c9ae2dc460d9e32efad41c263c1 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:36.061482 21081 raft_consensus.cc:493] T de2b3e07bd634bc08368989a9062ba08 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:36.061897 21081 raft_consensus.cc:515] T de2b3e07bd634bc08368989a9062ba08 P 7c1b52895a14482fb54a6758f5054f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:36.062513 19192 connection.cc:573] server connection from 127.18.182.193:42207 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 13:56:36.064158 21081 leader_election.cc:290] T de2b3e07bd634bc08368989a9062ba08 P 7c1b52895a14482fb54a6758f5054f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649)
W20260925 13:56:36.064713 19264 connection.cc:573] client connection to 127.18.182.193:38269 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 13:56:36.065038 19264 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107) [suppressed 6 similar messages]
I20260925 13:56:36.066730 19163 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskYmJttM/build/tsan/bin/kudu with pid 19610
W20260925 13:56:36.068162 19773 connection.cc:573] client connection to 127.18.182.193:38269 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 13:56:36.068598 19773 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 13:56:36.069952 19276 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 31b5f47966a74ec3b59c7c20479ba702 (table default.Table_bd9c5f2059394324857aa7fef30188f7 [id=e30a4ca9b0bf4c11a6691a896304c57e]). Expected version 1 got 0
I20260925 13:56:36.070643 19276 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet e9dbcd85dc8647c68c120ba3fcfe1790 (table default.Table_5bf9b87b5083469fad04eb3a80a4b8d5 [id=443536a0bd4d4d568d52f6ffe3d06ecb]). Expected version 1 got 0
I20260925 13:56:36.072086 21107 tablet_replica.cc:333] T 65ad0197267d47158751bc44c539b555 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:36.072999 21107 raft_consensus.cc:2243] T 65ad0197267d47158751bc44c539b555 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:36.073448 21107 raft_consensus.cc:2272] T 65ad0197267d47158751bc44c539b555 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:36.081300 21107 ts_tablet_manager.cc:1916] T 65ad0197267d47158751bc44c539b555 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:36.071245 19276 catalog_manager.cc:5838] T 45ab2817d7e84185b7ec2bfddcf4997f P 7c1b52895a14482fb54a6758f5054f8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194). New cstate: current_term: 1 leader_uuid: "7c1b52895a14482fb54a6758f5054f8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } health_report { overall_health: UNKNOWN } } }
I20260925 13:56:36.082792 19276 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet 45ab2817d7e84185b7ec2bfddcf4997f (table default.Table_8b37ba6f398943df8ab95b19f0b7dec6 [id=9cb2dc12a91e4de087414b492cd0dcfb]). Expected version 1 got 0
I20260925 13:56:36.083817 19276 catalog_manager.cc:5938] TS 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641) does not have the latest schema for tablet de2b3e07bd634bc08368989a9062ba08 (table default.Table_cd8babe4e16340b18a41ba2e3aa63cd8 [id=be2ddd2647094a8788d52ef46500426f]). Expected version 1 got 0
W20260925 13:56:36.101059 19267 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet d7c8beb65fd64639b22374fe4d3ef400 (table default.Table_626a2de34ead48f4a50d72ea44cc25f2 [id=f4560686cb314de99329f63b5902971d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet d7c8beb65fd64639b22374fe4d3ef400 consensus state has no leader
I20260925 13:56:36.113201 20891 raft_consensus.cc:493] T 80ded435c38e4a84a639cd9dcad25cde P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:36.113621 20891 raft_consensus.cc:515] T 80ded435c38e4a84a639cd9dcad25cde P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:36.115244 20891 leader_election.cc:290] T 80ded435c38e4a84a639cd9dcad25cde P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
W20260925 13:56:36.121914 19264 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.18.182.193:38269, user_credentials={real_user=slave}} blocked reactor thread for 56883.1us
I20260925 13:56:36.131448 21107 ts_tablet_manager.cc:1929] T 65ad0197267d47158751bc44c539b555 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:36.131922 21107 log.cc:1199] T 65ad0197267d47158751bc44c539b555 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/65ad0197267d47158751bc44c539b555
I20260925 13:56:36.132679 19938 tablet_bootstrap.cc:654] T 43692c9ae2dc460d9e32efad41c263c1 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:36.132702 21107 ts_tablet_manager.cc:1950] T 65ad0197267d47158751bc44c539b555 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:36.157531 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 65ad0197267d47158751bc44c539b555 (table default.Table_727855fadd224779a3638b48e6b0266f [id=4e57e9392b3f43e3a767b7c4b641d498]) successfully deleted
W20260925 13:56:36.158890 19773 consensus_peers.cc:597] T 5f8a96ae6eec4f15a221f9aad537c094 P f790493ba7ad403993b678f2df2b55be -> Peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Couldn't send request to peer 8f578869bc084a5d8a9a69f143954eb3. Status: Network error: Client connection negotiation failed: client connection to 127.18.182.193:38269: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260925 13:56:36.159317 19773 consensus_peers.cc:597] T 7e544c93488541da9ebb588a465b27b7 P f790493ba7ad403993b678f2df2b55be -> Peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Couldn't send request to peer 8f578869bc084a5d8a9a69f143954eb3. Status: Network error: Client connection negotiation failed: client connection to 127.18.182.193:38269: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260925 13:56:36.159721 19773 leader_election.cc:336] T 540132db164c403da72b9377078c4bf0 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Network error: Client connection negotiation failed: client connection to 127.18.182.193:38269: connect: Connection refused (error 111)
I20260925 13:56:36.160106 19773 leader_election.cc:304] T 540132db164c403da72b9377078c4bf0 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:36.160751 20891 raft_consensus.cc:2749] T 540132db164c403da72b9377078c4bf0 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:56:36.161540 19773 leader_election.cc:336] T e4b29a3d0edc481cba6c61cfb78f8f40 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Network error: Client connection negotiation failed: client connection to 127.18.182.193:38269: connect: Connection refused (error 111)
I20260925 13:56:36.162262 19773 leader_election.cc:304] T e4b29a3d0edc481cba6c61cfb78f8f40 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:36.163079 19938 tablet_bootstrap.cc:492] T 43692c9ae2dc460d9e32efad41c263c1 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:36.163503 19938 ts_tablet_manager.cc:1403] T 43692c9ae2dc460d9e32efad41c263c1 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.113s user 0.010s sys 0.000s
W20260925 13:56:36.170950 19264 catalog_manager.cc:4790] TS 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5788849b7336496488e5401b4ac91191: Network error: Client connection negotiation failed: client connection to 127.18.182.193:38269: connect: Connection refused (error 111)
I20260925 13:56:36.171496 20891 raft_consensus.cc:2749] T e4b29a3d0edc481cba6c61cfb78f8f40 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:56:36.172381 19773 leader_election.cc:336] T 72183473a5c94629a58c2cd8e1c56f32 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Network error: Client connection negotiation failed: client connection to 127.18.182.193:38269: connect: Connection refused (error 111)
W20260925 13:56:36.172971 19773 leader_election.cc:336] T 45ab2817d7e84185b7ec2bfddcf4997f P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Network error: Client connection negotiation failed: client connection to 127.18.182.193:38269: connect: Connection refused (error 111)
I20260925 13:56:36.173316 19773 leader_election.cc:304] T 45ab2817d7e84185b7ec2bfddcf4997f P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:36.173925 20891 raft_consensus.cc:2749] T 45ab2817d7e84185b7ec2bfddcf4997f P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:56:36.174979 19773 leader_election.cc:336] T 72183473a5c94629a58c2cd8e1c56f32 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Network error: Client connection negotiation failed: client connection to 127.18.182.193:38269: connect: Connection refused (error 111)
W20260925 13:56:36.176417 19773 consensus_peers.cc:597] T 1215d875209a40d38cc527e2795d6c75 P f790493ba7ad403993b678f2df2b55be -> Peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Couldn't send request to peer 8f578869bc084a5d8a9a69f143954eb3. Status: Network error: Client connection negotiation failed: client connection to 127.18.182.193:38269: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 13:56:36.176242 19938 raft_consensus.cc:359] T 43692c9ae2dc460d9e32efad41c263c1 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:36.176868 19938 raft_consensus.cc:385] T 43692c9ae2dc460d9e32efad41c263c1 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:36.177179 19938 raft_consensus.cc:740] T 43692c9ae2dc460d9e32efad41c263c1 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
W20260925 13:56:36.176903 19773 consensus_peers.cc:597] T f50ea68e3b3c4459b3f5e5b0df724724 P f790493ba7ad403993b678f2df2b55be -> Peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Couldn't send request to peer 8f578869bc084a5d8a9a69f143954eb3. Status: Network error: Client connection negotiation failed: client connection to 127.18.182.193:38269: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260925 13:56:36.177995 19773 leader_election.cc:336] T de2b3e07bd634bc08368989a9062ba08 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Network error: Client connection negotiation failed: client connection to 127.18.182.193:38269: connect: Connection refused (error 111)
I20260925 13:56:36.181715 19773 leader_election.cc:304] T de2b3e07bd634bc08368989a9062ba08 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:36.182341 20891 raft_consensus.cc:2749] T de2b3e07bd634bc08368989a9062ba08 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:56:36.183130 19773 leader_election.cc:336] T a0109e4115174d8eb8174a1a2236150f P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Network error: Client connection negotiation failed: client connection to 127.18.182.193:38269: connect: Connection refused (error 111)
I20260925 13:56:36.183558 19773 leader_election.cc:304] T a0109e4115174d8eb8174a1a2236150f P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:36.184187 20891 raft_consensus.cc:2749] T a0109e4115174d8eb8174a1a2236150f P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:56:36.184962 19773 consensus_peers.cc:597] T b12a6c6cff024d70b53cc2c9169dd48b P f790493ba7ad403993b678f2df2b55be -> Peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Couldn't send request to peer 8f578869bc084a5d8a9a69f143954eb3. Status: Network error: Client connection negotiation failed: client connection to 127.18.182.193:38269: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260925 13:56:36.185387 19773 leader_election.cc:336] T 869bc08603b444b4885603d81b97a1db P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Network error: Client connection negotiation failed: client connection to 127.18.182.193:38269: connect: Connection refused (error 111)
I20260925 13:56:36.185781 19773 leader_election.cc:304] T 869bc08603b444b4885603d81b97a1db P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:36.186707 20891 raft_consensus.cc:2749] T 869bc08603b444b4885603d81b97a1db P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:36.177796 19938 consensus_queue.cc:260] T 43692c9ae2dc460d9e32efad41c263c1 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
W20260925 13:56:36.187628 19773 consensus_peers.cc:597] T 31b5f47966a74ec3b59c7c20479ba702 P f790493ba7ad403993b678f2df2b55be -> Peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Couldn't send request to peer 8f578869bc084a5d8a9a69f143954eb3. Status: Network error: Client connection negotiation failed: client connection to 127.18.182.193:38269: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260925 13:56:36.188027 19773 leader_election.cc:336] T fd448eab74b344838b3b92cad83a1294 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Network error: Client connection negotiation failed: client connection to 127.18.182.193:38269: connect: Connection refused (error 111)
I20260925 13:56:36.188426 19773 leader_election.cc:304] T fd448eab74b344838b3b92cad83a1294 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:36.189040 20891 raft_consensus.cc:2749] T fd448eab74b344838b3b92cad83a1294 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:56:36.189782 19773 leader_election.cc:336] T fa8330217e794edf9e26a368d8b34685 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Network error: Client connection negotiation failed: client connection to 127.18.182.193:38269: connect: Connection refused (error 111)
I20260925 13:56:36.190172 19773 leader_election.cc:304] T fa8330217e794edf9e26a368d8b34685 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:36.190768 20891 raft_consensus.cc:2749] T fa8330217e794edf9e26a368d8b34685 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:56:36.191515 19773 leader_election.cc:336] T 69934b9a49354c92b5b13e182247ef77 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Network error: Client connection negotiation failed: client connection to 127.18.182.193:38269: connect: Connection refused (error 111)
I20260925 13:56:36.191896 19773 leader_election.cc:304] T 69934b9a49354c92b5b13e182247ef77 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:36.192499 20891 raft_consensus.cc:2749] T 69934b9a49354c92b5b13e182247ef77 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:56:36.193225 19773 leader_election.cc:336] T ab1ba6ef73a046b59ffac150f7c73953 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Network error: Client connection negotiation failed: client connection to 127.18.182.193:38269: connect: Connection refused (error 111)
I20260925 13:56:36.193625 19773 leader_election.cc:304] T ab1ba6ef73a046b59ffac150f7c73953 P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:36.194207 20891 raft_consensus.cc:2749] T ab1ba6ef73a046b59ffac150f7c73953 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:56:36.204686 19938 ts_tablet_manager.cc:1434] T 43692c9ae2dc460d9e32efad41c263c1 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.041s user 0.005s sys 0.000s
I20260925 13:56:36.207048 19938 tablet_bootstrap.cc:492] T 8e5f54ef08054e7fb231c361615e2ab2 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
W20260925 13:56:36.210292 19773 leader_election.cc:336] T 80ded435c38e4a84a639cd9dcad25cde P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Network error: Client connection negotiation failed: client connection to 127.18.182.193:38269: connect: Connection refused (error 111)
I20260925 13:56:36.216694 19809 tablet_service.cc:1558] Processing DeleteTablet for tablet 43692c9ae2dc460d9e32efad41c263c1 with delete_type TABLET_DATA_DELETED (replaced by tablet a426f65001f3420fb5b3672c57ae53d1) from {username='slave'} at 127.0.0.1:51442
I20260925 13:56:36.222568 19938 tablet_bootstrap.cc:654] T 8e5f54ef08054e7fb231c361615e2ab2 P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:36.248548 21107 tablet_replica.cc:333] T 43692c9ae2dc460d9e32efad41c263c1 P f790493ba7ad403993b678f2df2b55be: stopping tablet replica
I20260925 13:56:36.249472 21107 raft_consensus.cc:2243] T 43692c9ae2dc460d9e32efad41c263c1 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:56:36.249931 21107 raft_consensus.cc:2272] T 43692c9ae2dc460d9e32efad41c263c1 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:56:36.251945 21107 ts_tablet_manager.cc:1916] T 43692c9ae2dc460d9e32efad41c263c1 P f790493ba7ad403993b678f2df2b55be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:56:36.278395 21107 ts_tablet_manager.cc:1929] T 43692c9ae2dc460d9e32efad41c263c1 P f790493ba7ad403993b678f2df2b55be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:56:36.278859 21107 log.cc:1199] T 43692c9ae2dc460d9e32efad41c263c1 P f790493ba7ad403993b678f2df2b55be: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/43692c9ae2dc460d9e32efad41c263c1
I20260925 13:56:36.279623 21107 ts_tablet_manager.cc:1950] T 43692c9ae2dc460d9e32efad41c263c1 P f790493ba7ad403993b678f2df2b55be: Deleting consensus metadata
I20260925 13:56:36.297993 19938 tablet_bootstrap.cc:492] T 8e5f54ef08054e7fb231c361615e2ab2 P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:36.298581 19938 ts_tablet_manager.cc:1403] T 8e5f54ef08054e7fb231c361615e2ab2 P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.092s user 0.008s sys 0.000s
I20260925 13:56:36.307508 19938 raft_consensus.cc:359] T 8e5f54ef08054e7fb231c361615e2ab2 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:36.308180 19938 raft_consensus.cc:385] T 8e5f54ef08054e7fb231c361615e2ab2 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:36.308460 19938 raft_consensus.cc:740] T 8e5f54ef08054e7fb231c361615e2ab2 P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:36.309099 19938 consensus_queue.cc:260] T 8e5f54ef08054e7fb231c361615e2ab2 P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:36.316176 19938 ts_tablet_manager.cc:1434] T 8e5f54ef08054e7fb231c361615e2ab2 P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.017s user 0.008s sys 0.000s
I20260925 13:56:36.316982 19938 tablet_bootstrap.cc:492] T 77f3683ff34342e8bc8ad5a4a349da6d P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
I20260925 13:56:36.330394 19938 tablet_bootstrap.cc:654] T 77f3683ff34342e8bc8ad5a4a349da6d P f790493ba7ad403993b678f2df2b55be: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:36.364996 19938 tablet_bootstrap.cc:492] T 77f3683ff34342e8bc8ad5a4a349da6d P f790493ba7ad403993b678f2df2b55be: No bootstrap required, opened a new log
I20260925 13:56:36.365406 19938 ts_tablet_manager.cc:1403] T 77f3683ff34342e8bc8ad5a4a349da6d P f790493ba7ad403993b678f2df2b55be: Time spent bootstrapping tablet: real 0.049s user 0.010s sys 0.000s
I20260925 13:56:36.367982 19938 raft_consensus.cc:359] T 77f3683ff34342e8bc8ad5a4a349da6d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:36.368646 19938 raft_consensus.cc:385] T 77f3683ff34342e8bc8ad5a4a349da6d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:36.369431 21085 raft_consensus.cc:493] T 28219970809145bc9d4e2a3289a9479a P f790493ba7ad403993b678f2df2b55be [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8f578869bc084a5d8a9a69f143954eb3)
I20260925 13:56:36.369872 21085 raft_consensus.cc:515] T 28219970809145bc9d4e2a3289a9479a P f790493ba7ad403993b678f2df2b55be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:36.371747 21085 leader_election.cc:290] T 28219970809145bc9d4e2a3289a9479a P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641), 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269)
I20260925 13:56:36.375737 19266 catalog_manager.cc:5063] TS f790493ba7ad403993b678f2df2b55be (127.18.182.195:46649): tablet 43692c9ae2dc460d9e32efad41c263c1 (table default.Table_9ae8fc862af64663b8f5f175a4085a03 [id=fa78293b520f449fa0e961b2a14a373b]) successfully deleted
I20260925 13:56:36.384421 20891 raft_consensus.cc:493] T f1c6e2137700448a8ebf859a9ca9085d P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f578869bc084a5d8a9a69f143954eb3)
I20260925 13:56:36.384822 20891 raft_consensus.cc:515] T f1c6e2137700448a8ebf859a9ca9085d P f790493ba7ad403993b678f2df2b55be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:36.386940 20891 leader_election.cc:290] T f1c6e2137700448a8ebf859a9ca9085d P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269), 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641)
I20260925 13:56:36.368940 19938 raft_consensus.cc:740] T 77f3683ff34342e8bc8ad5a4a349da6d P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f790493ba7ad403993b678f2df2b55be, State: Initialized, Role: FOLLOWER
I20260925 13:56:36.390851 19938 consensus_queue.cc:260] T 77f3683ff34342e8bc8ad5a4a349da6d P f790493ba7ad403993b678f2df2b55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f578869bc084a5d8a9a69f143954eb3" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 38269 } } peers { permanent_uuid: "7c1b52895a14482fb54a6758f5054f8f" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 38641 } } peers { permanent_uuid: "f790493ba7ad403993b678f2df2b55be" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 46649 } }
I20260925 13:56:36.392557 19938 ts_tablet_manager.cc:1434] T 77f3683ff34342e8bc8ad5a4a349da6d P f790493ba7ad403993b678f2df2b55be: Time spent starting tablet: real 0.027s user 0.006s sys 0.000s
I20260925 13:56:36.393558 19938 tablet_bootstrap.cc:492] T fae82e043ab74e12bc4e14e2b5536a92 P f790493ba7ad403993b678f2df2b55be: Bootstrap starting.
W20260925 13:56:36.396142 19773 leader_election.cc:336] T 28219970809145bc9d4e2a3289a9479a P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Network error: Client connection negotiation failed: client connection to 127.18.182.193:38269: connect: Connection refused (error 111)
W20260925 13:56:36.396602 19773 leader_election.cc:336] T f1c6e2137700448a8ebf859a9ca9085d P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8f578869bc084a5d8a9a69f143954eb3 (127.18.182.193:38269): Network error: Client connection negotiation failed: client connection to 127.18.182.193:38269: connect: Connection refused (error 111)
W20260925 13:56:36.428319 19266 connection.cc:573] server connection from 127.18.182.194:34125 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 13:56:36.429080 19773 connection.cc:573] client connection to 127.18.182.194:38641 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 13:56:36.435617 19773 leader_election.cc:336] T 80ded435c38e4a84a639cd9dcad25cde P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Network error: Client connection negotiation failed: client connection to 127.18.182.194:38641: connect: Connection refused (error 111)
I20260925 13:56:36.436125 19773 leader_election.cc:304] T 80ded435c38e4a84a639cd9dcad25cde P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:36.436903 20831 raft_consensus.cc:2749] T 80ded435c38e4a84a639cd9dcad25cde P f790493ba7ad403993b678f2df2b55be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:56:36.437072 19773 consensus_peers.cc:597] T 1215d875209a40d38cc527e2795d6c75 P f790493ba7ad403993b678f2df2b55be -> Peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Couldn't send request to peer 7c1b52895a14482fb54a6758f5054f8f. Status: Network error: Client connection negotiation failed: client connection to 127.18.182.194:38641: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260925 13:56:36.437471 19773 consensus_peers.cc:597] T f50ea68e3b3c4459b3f5e5b0df724724 P f790493ba7ad403993b678f2df2b55be -> Peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Couldn't send request to peer 7c1b52895a14482fb54a6758f5054f8f. Status: Network error: Client connection negotiation failed: client connection to 127.18.182.194:38641: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260925 13:56:36.437893 19773 consensus_peers.cc:597] T 7e544c93488541da9ebb588a465b27b7 P f790493ba7ad403993b678f2df2b55be -> Peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Couldn't send request to peer 7c1b52895a14482fb54a6758f5054f8f. Status: Network error: Client connection negotiation failed: client connection to 127.18.182.194:38641: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260925 13:56:36.438320 19773 leader_election.cc:336] T 28219970809145bc9d4e2a3289a9479a P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7c1b52895a14482fb54a6758f5054f8f (127.18.182.194:38641): Network error: Client connection negotiation failed: client connection to 127.18.182.194:38641: connect: Connection refused (error 111)
I20260925 13:56:36.438712 19773 leader_election.cc:304] T 28219970809145bc9d4e2a3289a9479a P f790493ba7ad403993b678f2df2b55be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f790493ba7ad403993b678f2df2b55be; no voters: 7c1b52895a14482fb54a6758f5054f8f, 8f578869bc084a5d8a9a69f143954eb3
I20260925 13:56:36.439430 20831 raft_consensus.cc:2749] T 28219970809145bc9d4e2a3289a9479a P f790493ba7ad403993b678f2df2b55be [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260925 13:56:36.442142 19163 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskYmJttM/build/tsan/bin/kudu with pid 19751
W20260925 13:56:36.712941 19266 connection.cc:573] client connection to 127.18.182.195:46649 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 13:56:36.713215 19163 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskYmJttM/build/tsan/bin/kudu with pid 19172
I20260925 13:56:36.791742 19163 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskYmJttM/build/tsan/bin/kudu with pid 19242
I20260925 13:56:36.863693 19163 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskYmJttM/build/tsan/bin/kudu with pid 20083
2026-09-25T13:56:36Z chronyd exiting
I20260925 13:56:36.918401 19163 master-stress-test.cc:587] Tables created: 170
I20260925 13:56:36.918624 19163 master-stress-test.cc:588] Tables altered: 80
I20260925 13:56:36.918807 19163 master-stress-test.cc:589] Tables deleted: 66
I20260925 13:56:36.918972 19163 master-stress-test.cc:590] Tablets replaced: 30
I20260925 13:56:36.919145 19163 master-stress-test.cc:591] Masters restarted: 1
[ OK ] CatalogManagerConfigurations/MasterStressTest.Test/0 (43526 ms)
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/3
2026-09-25T13:56:37Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:56:37Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskYmJttM/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskYmJttM/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/security.properties
2026-09-25 13:56:44: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskYmJttM/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskYmJttM/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260925 13:56:52.520751 19163 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskYmJttM/build/tsan/bin/kudu
/tmp/dist-test-taskYmJttM/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.182.254:43163
--webserver_interface=127.18.182.254
--webserver_port=0
--builtin_ntp_servers=127.18.182.212:40529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.182.254:43163,127.18.182.253:46177,127.18.182.252:39283
--hive_metastore_uris=thrift://127.0.0.1:39557
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260925 13:56:52.824246 21401 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:56:52.824882 21401 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:56:52.825333 21401 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260925 13:56:52.858549 21401 flags.cc:432] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260925 13:56:52.858830 21401 flags.cc:432] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260925 13:56:52.859122 21401 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:56:52.859404 21401 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:56:52.859664 21401 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:56:52.859920 21401 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:56:52.896189 21401 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.182.212:40529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:39557
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.18.182.254:43163,127.18.182.253:46177,127.18.182.252:39283
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.182.254:43163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.182.254
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 102683f65af844d269565345d7d8c6ebc55c92c5
build type FASTDEBUG
built by None at 25 Sep 2026 13:43:25 UTC on e7f111948823
build id 13994
TSAN enabled
I20260925 13:56:52.897632 21401 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:56:52.899303 21401 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260925 13:56:52.912099 21407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:56:52.912992 21408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:56:52.915256 21410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:56:54.133829 21409 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
I20260925 13:56:54.133863 21401 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260925 13:56:54.138743 21401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:56:54.141498 21401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:56:54.142831 21401 hybrid_clock.cc:648] HybridClock initialized: now 1790344614142802 us; error 68 us; skew 500 ppm
I20260925 13:56:54.143709 21401 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 13:56:54.149972 21401 webserver.cc:533] Webserver started at http://127.18.182.254:35865/ using document root <none> and password file <none>
I20260925 13:56:54.150874 21401 fs_manager.cc:362] Metadata directory not provided
I20260925 13:56:54.151110 21401 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:56:54.151608 21401 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:56:54.155601 21401 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-0/data/instance:
uuid: "e942d7f2e4864b5ca08e6e830085caa0"
format_stamp: "Formatted at 2026-09-25 13:56:54 on dist-test-slave-nlqf"
I20260925 13:56:54.156647 21401 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-0/wal/instance:
uuid: "e942d7f2e4864b5ca08e6e830085caa0"
format_stamp: "Formatted at 2026-09-25 13:56:54 on dist-test-slave-nlqf"
I20260925 13:56:54.163740 21401 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260925 13:56:54.169196 21418 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:56:54.170262 21401 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260925 13:56:54.170609 21401 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-0/data,/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-0/wal
uuid: "e942d7f2e4864b5ca08e6e830085caa0"
format_stamp: "Formatted at 2026-09-25 13:56:54 on dist-test-slave-nlqf"
I20260925 13:56:54.170981 21401 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:56:54.232611 21401 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260925 13:56:54.234202 21401 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:56:54.234733 21401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:56:54.305792 21401 rpc_server.cc:307] RPC server started. Bound to: 127.18.182.254:43163
I20260925 13:56:54.305891 21469 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.182.254:43163 every 8 connection(s)
I20260925 13:56:54.308598 21401 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-0/data/info.pb
I20260925 13:56:54.314271 21470 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:54.317936 19163 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskYmJttM/build/tsan/bin/kudu as pid 21401
I20260925 13:56:54.318601 19163 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-0/wal/instance
I20260925 13:56:54.322291 19163 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskYmJttM/build/tsan/bin/kudu
/tmp/dist-test-taskYmJttM/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.182.253:46177
--webserver_interface=127.18.182.253
--webserver_port=0
--builtin_ntp_servers=127.18.182.212:40529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.182.254:43163,127.18.182.253:46177,127.18.182.252:39283
--hive_metastore_uris=thrift://127.0.0.1:39557
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260925 13:56:54.329902 21470 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:56:54.369733 21470 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } has no permanent_uuid. Determining permanent_uuid...
W20260925 13:56:54.372185 21420 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.182.253:46177: connect: Connection refused (error 111)
W20260925 13:56:54.376523 21470 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.253:46177: Network error: Client connection negotiation failed: client connection to 127.18.182.253:46177: connect: Connection refused (error 111)
I20260925 13:56:54.426087 21470 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } attempt: 1
W20260925 13:56:54.430420 21470 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.253:46177: Network error: Client connection negotiation failed: client connection to 127.18.182.253:46177: connect: Connection refused (error 111)
I20260925 13:56:54.498927 21470 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } attempt: 2
W20260925 13:56:54.503737 21470 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.253:46177: Network error: Client connection negotiation failed: client connection to 127.18.182.253:46177: connect: Connection refused (error 111)
I20260925 13:56:54.595253 21470 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } attempt: 3
W20260925 13:56:54.599651 21470 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.253:46177: Network error: Client connection negotiation failed: client connection to 127.18.182.253:46177: connect: Connection refused (error 111)
W20260925 13:56:54.643286 21471 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:56:54.643883 21471 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:56:54.644254 21471 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260925 13:56:54.677191 21471 flags.cc:432] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260925 13:56:54.677454 21471 flags.cc:432] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260925 13:56:54.677659 21471 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:56:54.677861 21471 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:56:54.678061 21471 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:56:54.678260 21471 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:56:54.716105 21471 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.182.212:40529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:39557
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.18.182.254:43163,127.18.182.253:46177,127.18.182.252:39283
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.182.253:46177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.182.253
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 102683f65af844d269565345d7d8c6ebc55c92c5
build type FASTDEBUG
built by None at 25 Sep 2026 13:43:25 UTC on e7f111948823
build id 13994
TSAN enabled
I20260925 13:56:54.717548 21471 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:56:54.719466 21471 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260925 13:56:54.732828 21480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:56:54.743134 21470 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } attempt: 4
W20260925 13:56:54.749393 21470 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.253:46177: Network error: Client connection negotiation failed: client connection to 127.18.182.253:46177: connect: Connection refused (error 111)
I20260925 13:56:55.048960 21470 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } attempt: 5
W20260925 13:56:55.056391 21470 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.253:46177: Network error: Client connection negotiation failed: client connection to 127.18.182.253:46177: connect: Connection refused (error 111)
I20260925 13:56:55.604010 21470 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } attempt: 6
W20260925 13:56:55.615126 21470 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.253:46177: Network error: Client connection negotiation failed: client connection to 127.18.182.253:46177: connect: Connection refused (error 111)
W20260925 13:56:54.734776 21481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:56:55.939152 21482 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
W20260925 13:56:55.941555 21483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:56:55.944460 21471 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.212s user 0.449s sys 0.762s
W20260925 13:56:55.944772 21471 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.212s user 0.449s sys 0.762s
I20260925 13:56:55.945027 21471 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260925 13:56:55.946368 21471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:56:55.949131 21471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:56:55.950516 21471 hybrid_clock.cc:648] HybridClock initialized: now 1790344615950487 us; error 36 us; skew 500 ppm
I20260925 13:56:55.951473 21471 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 13:56:55.958763 21471 webserver.cc:533] Webserver started at http://127.18.182.253:33753/ using document root <none> and password file <none>
I20260925 13:56:55.959791 21471 fs_manager.cc:362] Metadata directory not provided
I20260925 13:56:55.960026 21471 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:56:55.960486 21471 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:56:55.964963 21471 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-1/data/instance:
uuid: "3ea1e11bc6d84114b7616815b6af4635"
format_stamp: "Formatted at 2026-09-25 13:56:55 on dist-test-slave-nlqf"
I20260925 13:56:55.966041 21471 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-1/wal/instance:
uuid: "3ea1e11bc6d84114b7616815b6af4635"
format_stamp: "Formatted at 2026-09-25 13:56:55 on dist-test-slave-nlqf"
I20260925 13:56:55.973997 21471 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.004s sys 0.005s
I20260925 13:56:55.980299 21492 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:56:55.981518 21471 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260925 13:56:55.981860 21471 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-1/data,/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-1/wal
uuid: "3ea1e11bc6d84114b7616815b6af4635"
format_stamp: "Formatted at 2026-09-25 13:56:55 on dist-test-slave-nlqf"
I20260925 13:56:55.982210 21471 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:56:56.051496 21471 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260925 13:56:56.053032 21471 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:56:56.053508 21471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:56:56.130751 21471 rpc_server.cc:307] RPC server started. Bound to: 127.18.182.253:46177
I20260925 13:56:56.130789 21543 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.182.253:46177 every 8 connection(s)
I20260925 13:56:56.133821 21471 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-1/data/info.pb
I20260925 13:56:56.136159 19163 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskYmJttM/build/tsan/bin/kudu as pid 21471
I20260925 13:56:56.136759 19163 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-1/wal/instance
I20260925 13:56:56.140766 19163 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskYmJttM/build/tsan/bin/kudu
/tmp/dist-test-taskYmJttM/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.182.252:39283
--webserver_interface=127.18.182.252
--webserver_port=0
--builtin_ntp_servers=127.18.182.212:40529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.182.254:43163,127.18.182.253:46177,127.18.182.252:39283
--hive_metastore_uris=thrift://127.0.0.1:39557
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260925 13:56:56.141993 21544 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:56.156343 21544 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:56:56.182404 21544 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:56:56.198390 21544 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } has no permanent_uuid. Determining permanent_uuid...
W20260925 13:56:56.200958 21496 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.182.252:39283: connect: Connection refused (error 111)
W20260925 13:56:56.205775 21544 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.252:39283: Network error: Client connection negotiation failed: client connection to 127.18.182.252:39283: connect: Connection refused (error 111)
I20260925 13:56:56.255390 21544 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } attempt: 1
W20260925 13:56:56.259644 21544 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.252:39283: Network error: Client connection negotiation failed: client connection to 127.18.182.252:39283: connect: Connection refused (error 111)
I20260925 13:56:56.328142 21544 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } attempt: 2
W20260925 13:56:56.332440 21544 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.252:39283: Network error: Client connection negotiation failed: client connection to 127.18.182.252:39283: connect: Connection refused (error 111)
I20260925 13:56:56.423899 21544 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } attempt: 3
W20260925 13:56:56.428200 21544 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.252:39283: Network error: Client connection negotiation failed: client connection to 127.18.182.252:39283: connect: Connection refused (error 111)
W20260925 13:56:56.469326 21545 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:56:56.469956 21545 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:56:56.470353 21545 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260925 13:56:56.503686 21545 flags.cc:432] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260925 13:56:56.503933 21545 flags.cc:432] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260925 13:56:56.504136 21545 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:56:56.504407 21545 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:56:56.504618 21545 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:56:56.504820 21545 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:56:56.543649 21545 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.182.212:40529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:39557
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.18.182.254:43163,127.18.182.253:46177,127.18.182.252:39283
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.182.252:39283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.182.252
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 102683f65af844d269565345d7d8c6ebc55c92c5
build type FASTDEBUG
built by None at 25 Sep 2026 13:43:25 UTC on e7f111948823
build id 13994
TSAN enabled
I20260925 13:56:56.545281 21545 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:56:56.547004 21545 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260925 13:56:56.557613 21555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:56:56.571667 21544 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } attempt: 4
W20260925 13:56:56.577850 21544 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.252:39283: Network error: Client connection negotiation failed: client connection to 127.18.182.252:39283: connect: Connection refused (error 111)
I20260925 13:56:56.675822 21470 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } attempt: 7
I20260925 13:56:56.707468 21470 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } has no permanent_uuid. Determining permanent_uuid...
W20260925 13:56:56.752035 21470 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.252:39283: Network error: Client connection negotiation failed: client connection to 127.18.182.252:39283: connect: Connection refused (error 111)
I20260925 13:56:56.810647 21470 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } attempt: 1
W20260925 13:56:56.819144 21470 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.252:39283: Network error: Client connection negotiation failed: client connection to 127.18.182.252:39283: connect: Connection refused (error 111)
I20260925 13:56:56.877300 21544 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } attempt: 5
W20260925 13:56:56.887461 21544 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.252:39283: Network error: Client connection negotiation failed: client connection to 127.18.182.252:39283: connect: Connection refused (error 111)
I20260925 13:56:56.900776 21470 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } attempt: 2
W20260925 13:56:56.910292 21470 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.252:39283: Network error: Client connection negotiation failed: client connection to 127.18.182.252:39283: connect: Connection refused (error 111)
I20260925 13:56:56.995875 21470 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } attempt: 3
W20260925 13:56:57.005558 21470 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.252:39283: Network error: Client connection negotiation failed: client connection to 127.18.182.252:39283: connect: Connection refused (error 111)
I20260925 13:56:57.146159 21470 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } attempt: 4
W20260925 13:56:57.154980 21470 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.252:39283: Network error: Client connection negotiation failed: client connection to 127.18.182.252:39283: connect: Connection refused (error 111)
I20260925 13:56:57.435119 21544 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } attempt: 6
I20260925 13:56:57.438975 21470 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } attempt: 5
W20260925 13:56:57.447439 21470 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.252:39283: Network error: Client connection negotiation failed: client connection to 127.18.182.252:39283: connect: Connection refused (error 111)
W20260925 13:56:57.475611 21544 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.252:39283: Network error: Client connection negotiation failed: client connection to 127.18.182.252:39283: connect: Connection refused (error 111)
I20260925 13:56:58.000015 21470 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } attempt: 6
W20260925 13:56:58.038262 21470 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.182.252:39283: Network error: Client connection negotiation failed: client connection to 127.18.182.252:39283: connect: Connection refused (error 111)
W20260925 13:56:57.961664 21554 debug-util.cc:398] Leaking SignalData structure 0x7b08000292a0 after lost signal to thread 21545
W20260925 13:56:58.141177 21545 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.583s user 0.584s sys 0.989s
W20260925 13:56:58.143030 21557 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1583 milliseconds
W20260925 13:56:58.143426 21545 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.586s user 0.584s sys 0.989s
W20260925 13:56:58.143541 21558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:56:58.143710 21545 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
W20260925 13:56:56.558635 21556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:56:58.148025 21545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:56:58.150406 21545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:56:58.151791 21545 hybrid_clock.cc:648] HybridClock initialized: now 1790344618151751 us; error 35 us; skew 500 ppm
I20260925 13:56:58.152611 21545 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 13:56:58.158826 21545 webserver.cc:533] Webserver started at http://127.18.182.252:41169/ using document root <none> and password file <none>
I20260925 13:56:58.159819 21545 fs_manager.cc:362] Metadata directory not provided
I20260925 13:56:58.160053 21545 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:56:58.160502 21545 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:56:58.164736 21545 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-2/data/instance:
uuid: "a9bf5a5925ac42769e474adc84a1b725"
format_stamp: "Formatted at 2026-09-25 13:56:58 on dist-test-slave-nlqf"
I20260925 13:56:58.165848 21545 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-2/wal/instance:
uuid: "a9bf5a5925ac42769e474adc84a1b725"
format_stamp: "Formatted at 2026-09-25 13:56:58 on dist-test-slave-nlqf"
I20260925 13:56:58.173115 21545 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.006s sys 0.002s
I20260925 13:56:58.178977 21571 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:56:58.180176 21545 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260925 13:56:58.180516 21545 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-2/data,/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-2/wal
uuid: "a9bf5a5925ac42769e474adc84a1b725"
format_stamp: "Formatted at 2026-09-25 13:56:58 on dist-test-slave-nlqf"
I20260925 13:56:58.180852 21545 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:56:58.233965 21545 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260925 13:56:58.235460 21545 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:56:58.235977 21545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:56:58.306905 21545 rpc_server.cc:307] RPC server started. Bound to: 127.18.182.252:39283
I20260925 13:56:58.306988 21622 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.182.252:39283 every 8 connection(s)
I20260925 13:56:58.309777 21545 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-2/data/info.pb
I20260925 13:56:58.312108 19163 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskYmJttM/build/tsan/bin/kudu as pid 21545
I20260925 13:56:58.312716 19163 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-2/wal/instance
I20260925 13:56:58.316568 21623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:56:58.331607 21623 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:56:58.352092 21623 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:56:58.363408 21623 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:56:58.382733 21623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Bootstrap starting.
I20260925 13:56:58.390574 21623 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:58.392791 21623 log.cc:826] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Log is configured to *not* fsync() on all Append() calls
I20260925 13:56:58.398363 21623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: No bootstrap required, opened a new log
I20260925 13:56:58.420079 21623 raft_consensus.cc:359] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:56:58.421128 21623 raft_consensus.cc:385] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:58.421428 21623 raft_consensus.cc:740] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9bf5a5925ac42769e474adc84a1b725, State: Initialized, Role: FOLLOWER
I20260925 13:56:58.422444 21623 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:56:58.425210 21630 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:56:58.425860 21630 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:56:58.426513 21623 sys_catalog.cc:565] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:56:58.442925 21630 raft_consensus.cc:493] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:56:58.443495 21630 raft_consensus.cc:515] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:56:58.445850 21630 leader_election.cc:290] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e942d7f2e4864b5ca08e6e830085caa0 (127.18.182.254:43163), 3ea1e11bc6d84114b7616815b6af4635 (127.18.182.253:46177)
W20260925 13:56:58.476537 21644 catalog_manager.cc:1607] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 13:56:58.477062 21644 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 13:56:58.491586 21577 tablet.cc:2433] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 13:56:58.502624 21644 catalog_manager.cc:1607] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 13:56:58.503146 21644 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 13:56:58.519141 21644 catalog_manager.cc:1607] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 13:56:58.519588 21644 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 13:56:58.460922 21545 master_runner.cc:187] Error getting master registration for 127.18.182.254:43163: OK, instance_id { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" instance_seqno: 1790344614286957 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 13:56:58.523885 21545 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260925 13:56:58.534984 21644 catalog_manager.cc:1607] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 13:56:58.535363 21644 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 13:56:58.536068 21544 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } attempt: 7
W20260925 13:56:58.550315 21644 catalog_manager.cc:1607] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 13:56:58.550889 21644 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 13:56:58.566577 21544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635: Bootstrap starting.
W20260925 13:56:58.567678 21644 catalog_manager.cc:1607] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 13:56:58.568506 21644 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 13:56:58.574901 21544 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:58.577642 21544 log.cc:826] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635: Log is configured to *not* fsync() on all Append() calls
I20260925 13:56:58.584089 21544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635: No bootstrap required, opened a new log
W20260925 13:56:58.585803 21644 catalog_manager.cc:1607] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 13:56:58.586135 21644 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 13:56:58.605010 21644 catalog_manager.cc:1607] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 13:56:58.605700 21644 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 13:56:58.614928 21544 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:56:58.616062 21544 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:58.616408 21544 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ea1e11bc6d84114b7616815b6af4635, State: Initialized, Role: FOLLOWER
I20260925 13:56:58.617498 21544 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:56:58.621392 21650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:56:58.623049 21544 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:56:58.623198 21519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9bf5a5925ac42769e474adc84a1b725" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea1e11bc6d84114b7616815b6af4635" is_pre_election: true
I20260925 13:56:58.623204 21650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:56:58.624205 21519 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9bf5a5925ac42769e474adc84a1b725 in term 0.
I20260925 13:56:58.625983 21573 leader_election.cc:304] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea1e11bc6d84114b7616815b6af4635, a9bf5a5925ac42769e474adc84a1b725; no voters:
I20260925 13:56:58.627223 21630 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:56:58.627642 21630 raft_consensus.cc:493] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:56:58.627960 21630 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 0 FOLLOWER]: Advancing to term 1
W20260925 13:56:58.629249 21644 catalog_manager.cc:1607] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 13:56:58.631217 21644 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 13:56:58.635098 21630 raft_consensus.cc:515] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:56:58.642563 21630 leader_election.cc:290] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [CANDIDATE]: Term 1 election: Requested vote from peers e942d7f2e4864b5ca08e6e830085caa0 (127.18.182.254:43163), 3ea1e11bc6d84114b7616815b6af4635 (127.18.182.253:46177)
W20260925 13:56:58.646067 21644 catalog_manager.cc:1607] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 13:56:58.646864 21644 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 13:56:58.648192 21519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9bf5a5925ac42769e474adc84a1b725" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea1e11bc6d84114b7616815b6af4635"
I20260925 13:56:58.648996 21519 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:58.656792 21519 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9bf5a5925ac42769e474adc84a1b725 in term 1.
I20260925 13:56:58.658627 21573 leader_election.cc:304] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea1e11bc6d84114b7616815b6af4635, a9bf5a5925ac42769e474adc84a1b725; no voters:
I20260925 13:56:58.660820 21630 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 1 FOLLOWER]: Leader election won for term 1
W20260925 13:56:58.663193 21644 catalog_manager.cc:1607] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 13:56:58.664386 21644 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 13:56:58.667953 21630 raft_consensus.cc:697] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 1 LEADER]: Becoming Leader. State: Replica: a9bf5a5925ac42769e474adc84a1b725, State: Running, Role: LEADER
I20260925 13:56:58.669289 21630 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:56:58.686302 21655 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9bf5a5925ac42769e474adc84a1b725. Latest consensus state: current_term: 1 leader_uuid: "a9bf5a5925ac42769e474adc84a1b725" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:56:58.687604 21655 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [sys.catalog]: This master's current role is: LEADER
W20260925 13:56:58.694006 21644 catalog_manager.cc:1607] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 13:56:58.694461 21644 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 13:56:58.695205 21658 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260925 13:56:58.703768 21658 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260925 13:56:58.747852 21519 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 1 FOLLOWER]: Refusing update from remote peer a9bf5a5925ac42769e474adc84a1b725: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:56:58.749871 21630 consensus_queue.cc:1048] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 13:56:58.782080 21669 catalog_manager.cc:818] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 13:56:58.787225 21650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9bf5a5925ac42769e474adc84a1b725. Latest consensus state: current_term: 1 leader_uuid: "a9bf5a5925ac42769e474adc84a1b725" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:56:58.788110 21650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:56:58.797084 21663 mvcc.cc:204] Tried to move back new op lower bound from 7333251558315036672 to 7333251558082727936. Current Snapshot: MvccSnapshot[applied={T|T < 7333251558315036672}]
I20260925 13:56:58.807542 21658 catalog_manager.cc:1396] Generated new cluster ID: d5d0bc5567a74d9cad4d92a9d1041fd9
I20260925 13:56:58.807830 21658 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260925 13:56:58.809170 21630 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a9bf5a5925ac42769e474adc84a1b725" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:56:58.809973 21630 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [sys.catalog]: This master's current role is: LEADER
W20260925 13:56:58.811714 21498 tablet.cc:2433] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 13:56:58.785835 21471 master_runner.cc:187] Error getting master registration for 127.18.182.254:43163: OK, instance_id { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" instance_seqno: 1790344614286957 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 13:56:58.823244 21471 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260925 13:56:58.826547 21669 catalog_manager.cc:1607] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 13:56:58.826866 21669 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 13:56:58.828912 21650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a9bf5a5925ac42769e474adc84a1b725" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:56:58.829552 21650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:56:58.855018 21669 catalog_manager.cc:1308] Loaded cluster ID: d5d0bc5567a74d9cad4d92a9d1041fd9
I20260925 13:56:58.855618 21669 catalog_manager.cc:1601] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635: loading cluster ID for follower catalog manager: success
I20260925 13:56:58.864231 21658 catalog_manager.cc:1419] Generated new certificate authority record
I20260925 13:56:58.866318 21658 catalog_manager.cc:1553] Loading token signing keys...
W20260925 13:56:58.874106 21669 catalog_manager.cc:1625] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260925 13:56:58.883157 21669 catalog_manager.cc:896] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260925 13:56:58.902153 21658 catalog_manager.cc:6219] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Generated new TSK 0
I20260925 13:56:58.903690 21658 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260925 13:56:58.908329 21658 catalog_manager.cc:1576] Loading latest processed Hive Metastore notification log event ID...
I20260925 13:56:58.912595 21658 catalog_manager.cc:6068] Last processed Hive Metastore notification event ID: -1
I20260925 13:56:58.923143 21669 catalog_manager.cc:1623] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635: acquiring CA information for follower catalog manager: success
I20260925 13:56:58.928791 21669 catalog_manager.cc:1651] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 13:56:58.956648 19163 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskYmJttM/build/tsan/bin/kudu
/tmp/dist-test-taskYmJttM/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.182.193:0
--local_ip_for_outbound_sockets=127.18.182.193
--webserver_interface=127.18.182.193
--webserver_port=0
--tserver_master_addrs=127.18.182.254:43163,127.18.182.253:46177,127.18.182.252:39283
--builtin_ntp_servers=127.18.182.212:40529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
W20260925 13:56:59.017176 21572 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.18.182.254:43163 timed out after 0.571s (SENT)
I20260925 13:56:59.071887 21470 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } attempt: 7
I20260925 13:56:59.137091 21470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0: Bootstrap starting.
I20260925 13:56:59.168154 21470 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0: Neither blocks nor log segments found. Creating new log.
I20260925 13:56:59.178689 21470 log.cc:826] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0: Log is configured to *not* fsync() on all Append() calls
I20260925 13:56:59.216356 21470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0: No bootstrap required, opened a new log
I20260925 13:56:59.408022 21680 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a9bf5a5925ac42769e474adc84a1b725)
I20260925 13:56:59.408639 21680 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:56:59.412907 21598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ea1e11bc6d84114b7616815b6af4635" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a9bf5a5925ac42769e474adc84a1b725" is_pre_election: true
I20260925 13:56:59.423563 21680 leader_election.cc:290] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e942d7f2e4864b5ca08e6e830085caa0 (127.18.182.254:43163), a9bf5a5925ac42769e474adc84a1b725 (127.18.182.252:39283)
I20260925 13:56:59.424611 21470 raft_consensus.cc:359] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:56:59.442150 21470 raft_consensus.cc:385] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:56:59.452253 21470 raft_consensus.cc:740] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e942d7f2e4864b5ca08e6e830085caa0, State: Initialized, Role: FOLLOWER
I20260925 13:56:59.482555 21470 consensus_queue.cc:260] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:56:59.579572 21684 sys_catalog.cc:455] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
W20260925 13:56:59.609503 21572 leader_election.cc:336] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer e942d7f2e4864b5ca08e6e830085caa0 (127.18.182.254:43163): Timed out: RequestConsensusVote RPC to 127.18.182.254:43163 timed out after 0.571s (SENT)
I20260925 13:56:59.604010 21684 sys_catalog.cc:458] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:56:59.615797 21443 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:56:59.620674 21442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9bf5a5925ac42769e474adc84a1b725" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e942d7f2e4864b5ca08e6e830085caa0" is_pre_election: true
W20260925 13:56:59.621809 21442 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42084 (request call id 5) took 584 ms (client timeout 571 ms). Trace:
W20260925 13:56:59.622162 21442 rpcz_store.cc:269] 0925 13:56:59.037189 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:59.037338 (+ 149us) service_pool.cc:224] Handling call
0925 13:56:59.621712 (+584374us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260925 13:56:59.623665 21444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9bf5a5925ac42769e474adc84a1b725" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e942d7f2e4864b5ca08e6e830085caa0"
W20260925 13:56:59.624545 21444 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42084 (request call id 3) took 984 ms (client timeout 511 ms). Trace:
W20260925 13:56:59.624871 21444 rpcz_store.cc:269] 0925 13:56:58.640063 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:58.640268 (+ 205us) service_pool.cc:224] Handling call
0925 13:56:59.624487 (+984219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260925 13:56:59.625747 21445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9bf5a5925ac42769e474adc84a1b725" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e942d7f2e4864b5ca08e6e830085caa0" is_pre_election: true
W20260925 13:56:59.626549 21445 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42084 (request call id 1) took 1180 ms (client timeout 571 ms). Trace:
W20260925 13:56:59.626885 21445 rpcz_store.cc:269] 0925 13:56:58.446829 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:58.447064 (+ 235us) service_pool.cc:224] Handling call
0925 13:56:59.626482 (+1179418us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260925 13:56:59.627766 21441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9bf5a5925ac42769e474adc84a1b725" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e942d7f2e4864b5ca08e6e830085caa0"
W20260925 13:56:59.628542 21441 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42084 (request call id 6) took 476 ms (client timeout 511 ms). Trace:
W20260925 13:56:59.628855 21441 rpcz_store.cc:269] 0925 13:56:59.152180 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:56:59.152348 (+ 168us) service_pool.cc:224] Handling call
0925 13:56:59.628493 (+476145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260925 13:56:59.629667 21440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ea1e11bc6d84114b7616815b6af4635" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e942d7f2e4864b5ca08e6e830085caa0" is_pre_election: true
I20260925 13:56:59.631768 21493 leader_election.cc:304] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ea1e11bc6d84114b7616815b6af4635; no voters: a9bf5a5925ac42769e474adc84a1b725, e942d7f2e4864b5ca08e6e830085caa0
I20260925 13:56:59.632663 21680 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:56:59.638376 21443 raft_consensus.cc:1275] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 1 FOLLOWER]: Refusing update from remote peer a9bf5a5925ac42769e474adc84a1b725: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:56:59.640084 21683 consensus_queue.cc:1048] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:56:59.644241 21470 sys_catalog.cc:565] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [sys.catalog]: configured and running, proceeding with master startup.
W20260925 13:56:59.694456 21675 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:56:59.695130 21675 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:56:59.696058 21675 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260925 13:56:59.713589 21684 sys_catalog.cc:455] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9bf5a5925ac42769e474adc84a1b725. Latest consensus state: current_term: 1 leader_uuid: "a9bf5a5925ac42769e474adc84a1b725" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:56:59.715319 21684 sys_catalog.cc:458] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:56:59.716462 21694 sys_catalog.cc:455] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a9bf5a5925ac42769e474adc84a1b725" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:56:59.717533 21694 sys_catalog.cc:458] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:56:59.757520 21706 catalog_manager.cc:818] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260925 13:56:59.759897 21675 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:56:59.760983 21675 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.182.193
I20260925 13:56:59.798398 21706 catalog_manager.cc:1308] Loaded cluster ID: d5d0bc5567a74d9cad4d92a9d1041fd9
I20260925 13:56:59.798671 21706 catalog_manager.cc:1601] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0: loading cluster ID for follower catalog manager: success
I20260925 13:56:59.806536 21706 catalog_manager.cc:1623] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0: acquiring CA information for follower catalog manager: success
I20260925 13:56:59.820320 21706 catalog_manager.cc:1651] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 13:56:59.832576 21675 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.182.212:40529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.182.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.182.193
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.18.182.254:43163,127.18.182.253:46177,127.18.182.252:39283
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.182.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 102683f65af844d269565345d7d8c6ebc55c92c5
build type FASTDEBUG
built by None at 25 Sep 2026 13:43:25 UTC on e7f111948823
build id 13994
TSAN enabled
I20260925 13:56:59.834249 21675 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:56:59.836541 21675 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260925 13:56:59.865761 21709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:57:00.148953 21694 raft_consensus.cc:493] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a9bf5a5925ac42769e474adc84a1b725)
I20260925 13:57:00.149499 21694 raft_consensus.cc:515] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:00.154125 21519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e942d7f2e4864b5ca08e6e830085caa0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ea1e11bc6d84114b7616815b6af4635" is_pre_election: true
I20260925 13:57:00.154163 21598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e942d7f2e4864b5ca08e6e830085caa0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a9bf5a5925ac42769e474adc84a1b725" is_pre_election: true
I20260925 13:57:00.156625 21422 leader_election.cc:304] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e942d7f2e4864b5ca08e6e830085caa0; no voters: 3ea1e11bc6d84114b7616815b6af4635, a9bf5a5925ac42769e474adc84a1b725
I20260925 13:57:00.157493 21684 raft_consensus.cc:2749] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:57:00.158200 21694 leader_election.cc:290] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ea1e11bc6d84114b7616815b6af4635 (127.18.182.253:46177), a9bf5a5925ac42769e474adc84a1b725 (127.18.182.252:39283)
I20260925 13:57:00.247294 21683 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a9bf5a5925ac42769e474adc84a1b725" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:57:00.250584 21683 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [sys.catalog]: This master's current role is: LEADER
I20260925 13:57:00.772377 21716 raft_consensus.cc:493] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a9bf5a5925ac42769e474adc84a1b725)
I20260925 13:57:00.773087 21716 raft_consensus.cc:515] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:00.779796 21519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e942d7f2e4864b5ca08e6e830085caa0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ea1e11bc6d84114b7616815b6af4635" is_pre_election: true
I20260925 13:57:00.780624 21598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e942d7f2e4864b5ca08e6e830085caa0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a9bf5a5925ac42769e474adc84a1b725" is_pre_election: true
I20260925 13:57:00.782780 21422 leader_election.cc:304] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e942d7f2e4864b5ca08e6e830085caa0; no voters: 3ea1e11bc6d84114b7616815b6af4635, a9bf5a5925ac42769e474adc84a1b725
I20260925 13:57:00.789271 21716 leader_election.cc:290] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ea1e11bc6d84114b7616815b6af4635 (127.18.182.253:46177), a9bf5a5925ac42769e474adc84a1b725 (127.18.182.252:39283)
I20260925 13:57:00.790153 21716 raft_consensus.cc:2749] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:57:00.964458 21718 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a9bf5a5925ac42769e474adc84a1b725)
I20260925 13:57:00.967582 21718 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:00.969787 21718 leader_election.cc:290] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e942d7f2e4864b5ca08e6e830085caa0 (127.18.182.254:43163), a9bf5a5925ac42769e474adc84a1b725 (127.18.182.252:39283)
I20260925 13:57:00.972615 21443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ea1e11bc6d84114b7616815b6af4635" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e942d7f2e4864b5ca08e6e830085caa0" is_pre_election: true
I20260925 13:57:00.973908 21598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ea1e11bc6d84114b7616815b6af4635" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a9bf5a5925ac42769e474adc84a1b725" is_pre_election: true
I20260925 13:57:00.976001 21493 leader_election.cc:304] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ea1e11bc6d84114b7616815b6af4635; no voters: a9bf5a5925ac42769e474adc84a1b725, e942d7f2e4864b5ca08e6e830085caa0
I20260925 13:57:00.985109 21718 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:57:01.564126 21719 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a9bf5a5925ac42769e474adc84a1b725)
I20260925 13:57:01.564819 21719 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:01.578027 21443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ea1e11bc6d84114b7616815b6af4635" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e942d7f2e4864b5ca08e6e830085caa0" is_pre_election: true
I20260925 13:57:01.578668 21598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ea1e11bc6d84114b7616815b6af4635" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a9bf5a5925ac42769e474adc84a1b725" is_pre_election: true
I20260925 13:57:01.580695 21496 leader_election.cc:304] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ea1e11bc6d84114b7616815b6af4635; no voters: a9bf5a5925ac42769e474adc84a1b725, e942d7f2e4864b5ca08e6e830085caa0
I20260925 13:57:01.582891 21719 leader_election.cc:290] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e942d7f2e4864b5ca08e6e830085caa0 (127.18.182.254:43163), a9bf5a5925ac42769e474adc84a1b725 (127.18.182.252:39283)
I20260925 13:57:01.583154 21720 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:57:01.748688 21721 raft_consensus.cc:493] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a9bf5a5925ac42769e474adc84a1b725)
I20260925 13:57:01.758496 21721 raft_consensus.cc:515] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:01.762804 21519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e942d7f2e4864b5ca08e6e830085caa0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ea1e11bc6d84114b7616815b6af4635" is_pre_election: true
I20260925 13:57:01.763211 21598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e942d7f2e4864b5ca08e6e830085caa0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a9bf5a5925ac42769e474adc84a1b725" is_pre_election: true
I20260925 13:57:01.765014 21721 leader_election.cc:290] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ea1e11bc6d84114b7616815b6af4635 (127.18.182.253:46177), a9bf5a5925ac42769e474adc84a1b725 (127.18.182.252:39283)
I20260925 13:57:01.766872 21420 leader_election.cc:304] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e942d7f2e4864b5ca08e6e830085caa0; no voters: 3ea1e11bc6d84114b7616815b6af4635, a9bf5a5925ac42769e474adc84a1b725
I20260925 13:57:01.767954 21721 raft_consensus.cc:2749] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 13:57:01.259378 21708 debug-util.cc:398] Leaking SignalData structure 0x7b0800037240 after lost signal to thread 21675
W20260925 13:57:01.855880 21708 kernel_stack_watchdog.cc:198] Thread 21675 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:642 for 388ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20260925 13:56:59.870330 21710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:57:01.857175 21675 thread.cc:641] GCE (cloud detector) Time spent creating pthread: real 1.989s user 0.650s sys 1.149s
W20260925 13:57:01.857517 21675 thread.cc:608] GCE (cloud detector) Time spent starting thread: real 1.989s user 0.650s sys 1.149s
I20260925 13:57:01.871716 21675 server_base.cc:1061] running on GCE node
W20260925 13:57:01.873363 21723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:57:01.874778 21675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:57:01.888537 21675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:57:01.892366 21675 hybrid_clock.cc:648] HybridClock initialized: now 1790344621892320 us; error 59 us; skew 500 ppm
I20260925 13:57:01.893473 21675 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 13:57:01.930766 21675 webserver.cc:533] Webserver started at http://127.18.182.193:43081/ using document root <none> and password file <none>
I20260925 13:57:01.932113 21675 fs_manager.cc:362] Metadata directory not provided
I20260925 13:57:01.932410 21675 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:57:01.932983 21675 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:57:01.939289 21675 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/data/instance:
uuid: "5e2731975dbf433aade4a859df9699e2"
format_stamp: "Formatted at 2026-09-25 13:57:01 on dist-test-slave-nlqf"
I20260925 13:57:01.940894 21675 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/instance:
uuid: "5e2731975dbf433aade4a859df9699e2"
format_stamp: "Formatted at 2026-09-25 13:57:01 on dist-test-slave-nlqf"
I20260925 13:57:01.961552 21675 fs_manager.cc:696] Time spent creating directory manager: real 0.020s user 0.016s sys 0.004s
I20260925 13:57:01.984521 21728 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:57:01.986130 21675 fs_manager.cc:730] Time spent opening block manager: real 0.020s user 0.013s sys 0.007s
I20260925 13:57:01.986616 21675 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal
uuid: "5e2731975dbf433aade4a859df9699e2"
format_stamp: "Formatted at 2026-09-25 13:57:01 on dist-test-slave-nlqf"
I20260925 13:57:01.987174 21675 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:57:02.090256 21675 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260925 13:57:02.093371 21675 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:57:02.094146 21675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:57:02.102180 21675 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260925 13:57:02.108495 21675 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:57:02.108844 21675 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260925 13:57:02.109452 21675 ts_tablet_manager.cc:616] Registered 0 tablets
I20260925 13:57:02.109733 21675 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:57:02.190932 21742 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a9bf5a5925ac42769e474adc84a1b725)
I20260925 13:57:02.199883 21742 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:02.203184 21443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ea1e11bc6d84114b7616815b6af4635" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e942d7f2e4864b5ca08e6e830085caa0" is_pre_election: true
I20260925 13:57:02.203662 21598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ea1e11bc6d84114b7616815b6af4635" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a9bf5a5925ac42769e474adc84a1b725" is_pre_election: true
I20260925 13:57:02.202764 21742 leader_election.cc:290] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e942d7f2e4864b5ca08e6e830085caa0 (127.18.182.254:43163), a9bf5a5925ac42769e474adc84a1b725 (127.18.182.252:39283)
I20260925 13:57:02.205948 21493 leader_election.cc:304] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ea1e11bc6d84114b7616815b6af4635; no voters: a9bf5a5925ac42769e474adc84a1b725, e942d7f2e4864b5ca08e6e830085caa0
I20260925 13:57:02.206645 21742 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:57:02.501212 21675 rpc_server.cc:307] RPC server started. Bound to: 127.18.182.193:39881
I20260925 13:57:02.501235 21841 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.182.193:39881 every 8 connection(s)
I20260925 13:57:02.504266 21675 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/data/info.pb
I20260925 13:57:02.508697 19163 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskYmJttM/build/tsan/bin/kudu as pid 21675
I20260925 13:57:02.509303 19163 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/instance
I20260925 13:57:02.529698 19163 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskYmJttM/build/tsan/bin/kudu
/tmp/dist-test-taskYmJttM/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.182.194:0
--local_ip_for_outbound_sockets=127.18.182.194
--webserver_interface=127.18.182.194
--webserver_port=0
--tserver_master_addrs=127.18.182.254:43163,127.18.182.253:46177,127.18.182.252:39283
--builtin_ntp_servers=127.18.182.212:40529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260925 13:57:02.606205 21843 heartbeater.cc:344] Connected to a master server at 127.18.182.253:46177
I20260925 13:57:02.606721 21843 heartbeater.cc:461] Registering TS with master...
I20260925 13:57:02.608094 21843 heartbeater.cc:507] Master 127.18.182.253:46177 requested a full tablet report, sending...
I20260925 13:57:02.611763 21509 ts_manager.cc:194] Registered new tserver with Master: 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881)
I20260925 13:57:02.615243 21842 heartbeater.cc:344] Connected to a master server at 127.18.182.252:39283
I20260925 13:57:02.615687 21842 heartbeater.cc:461] Registering TS with master...
I20260925 13:57:02.616784 21842 heartbeater.cc:507] Master 127.18.182.252:39283 requested a full tablet report, sending...
I20260925 13:57:02.619688 21588 ts_manager.cc:194] Registered new tserver with Master: 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881)
I20260925 13:57:02.622291 21588 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.182.193:57907
I20260925 13:57:02.635249 21848 heartbeater.cc:344] Connected to a master server at 127.18.182.254:43163
I20260925 13:57:02.635703 21848 heartbeater.cc:461] Registering TS with master...
I20260925 13:57:02.636490 21848 heartbeater.cc:507] Master 127.18.182.254:43163 requested a full tablet report, sending...
I20260925 13:57:02.640412 21435 ts_manager.cc:194] Registered new tserver with Master: 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881)
W20260925 13:57:03.080219 21849 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:57:03.080797 21849 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:57:03.081391 21849 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260925 13:57:03.119508 21849 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:57:03.120543 21849 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.182.194
I20260925 13:57:03.126520 21842 heartbeater.cc:499] Master 127.18.182.252:39283 was elected leader, sending a full tablet report...
I20260925 13:57:03.179767 21849 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.182.212:40529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.182.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.182.194
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.18.182.254:43163,127.18.182.253:46177,127.18.182.252:39283
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.182.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 102683f65af844d269565345d7d8c6ebc55c92c5
build type FASTDEBUG
built by None at 25 Sep 2026 13:43:25 UTC on e7f111948823
build id 13994
TSAN enabled
I20260925 13:57:03.181141 21849 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:57:03.182838 21849 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260925 13:57:03.190969 21858 raft_consensus.cc:493] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a9bf5a5925ac42769e474adc84a1b725)
I20260925 13:57:03.192068 21858 raft_consensus.cc:515] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:03.197201 21858 leader_election.cc:290] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ea1e11bc6d84114b7616815b6af4635 (127.18.182.253:46177), a9bf5a5925ac42769e474adc84a1b725 (127.18.182.252:39283)
W20260925 13:57:03.198302 21860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:57:03.198701 21598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e942d7f2e4864b5ca08e6e830085caa0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a9bf5a5925ac42769e474adc84a1b725" is_pre_election: true
I20260925 13:57:03.199483 21862 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a9bf5a5925ac42769e474adc84a1b725)
I20260925 13:57:03.199932 21862 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:03.202034 21862 leader_election.cc:290] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e942d7f2e4864b5ca08e6e830085caa0 (127.18.182.254:43163), a9bf5a5925ac42769e474adc84a1b725 (127.18.182.252:39283)
I20260925 13:57:03.203637 21443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ea1e11bc6d84114b7616815b6af4635" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e942d7f2e4864b5ca08e6e830085caa0" is_pre_election: true
I20260925 13:57:03.204630 21443 raft_consensus.cc:2468] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ea1e11bc6d84114b7616815b6af4635 in term 1.
I20260925 13:57:03.207005 21493 leader_election.cc:304] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea1e11bc6d84114b7616815b6af4635, e942d7f2e4864b5ca08e6e830085caa0; no voters:
I20260925 13:57:03.209123 21519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e942d7f2e4864b5ca08e6e830085caa0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ea1e11bc6d84114b7616815b6af4635" is_pre_election: true
I20260925 13:57:03.209650 21519 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e942d7f2e4864b5ca08e6e830085caa0 in term 1.
I20260925 13:57:03.209718 21598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ea1e11bc6d84114b7616815b6af4635" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a9bf5a5925ac42769e474adc84a1b725" is_pre_election: true
I20260925 13:57:03.212252 21862 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 13:57:03.212663 21862 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a9bf5a5925ac42769e474adc84a1b725)
I20260925 13:57:03.214859 21420 leader_election.cc:304] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ea1e11bc6d84114b7616815b6af4635, e942d7f2e4864b5ca08e6e830085caa0; no voters: a9bf5a5925ac42769e474adc84a1b725
I20260925 13:57:03.215788 21858 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 13:57:03.216137 21858 raft_consensus.cc:493] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a9bf5a5925ac42769e474adc84a1b725)
I20260925 13:57:03.213069 21862 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 1 FOLLOWER]: Advancing to term 2
I20260925 13:57:03.216470 21858 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 1 FOLLOWER]: Advancing to term 2
I20260925 13:57:03.224180 21858 raft_consensus.cc:515] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:03.227073 21519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e942d7f2e4864b5ca08e6e830085caa0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ea1e11bc6d84114b7616815b6af4635"
I20260925 13:57:03.241110 21858 leader_election.cc:290] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [CANDIDATE]: Term 2 election: Requested vote from peers 3ea1e11bc6d84114b7616815b6af4635 (127.18.182.253:46177), a9bf5a5925ac42769e474adc84a1b725 (127.18.182.252:39283)
I20260925 13:57:03.241039 21598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e942d7f2e4864b5ca08e6e830085caa0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a9bf5a5925ac42769e474adc84a1b725"
I20260925 13:57:03.246701 21443 raft_consensus.cc:1240] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 2 FOLLOWER]: Rejecting Update request from peer a9bf5a5925ac42769e474adc84a1b725 for earlier term 1. Current term is 2. Ops: []
I20260925 13:57:03.257300 21862 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:03.266583 21519 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e942d7f2e4864b5ca08e6e830085caa0 in current term 2: Already voted for candidate 3ea1e11bc6d84114b7616815b6af4635 in this term.
I20260925 13:57:03.267772 21443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ea1e11bc6d84114b7616815b6af4635" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e942d7f2e4864b5ca08e6e830085caa0"
I20260925 13:57:03.268716 21443 raft_consensus.cc:2393] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ea1e11bc6d84114b7616815b6af4635 in current term 2: Already voted for candidate e942d7f2e4864b5ca08e6e830085caa0 in this term.
I20260925 13:57:03.270113 21518 raft_consensus.cc:1240] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 2 FOLLOWER]: Rejecting Update request from peer a9bf5a5925ac42769e474adc84a1b725 for earlier term 1. Current term is 2. Ops: []
I20260925 13:57:03.260385 21865 consensus_queue.cc:1059] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260925 13:57:03.271418 21864 raft_consensus.cc:3055] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 1 LEADER]: Stepping down as leader of term 1
I20260925 13:57:03.272472 21420 leader_election.cc:304] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e942d7f2e4864b5ca08e6e830085caa0; no voters: 3ea1e11bc6d84114b7616815b6af4635, a9bf5a5925ac42769e474adc84a1b725
I20260925 13:57:03.272814 21864 raft_consensus.cc:740] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a9bf5a5925ac42769e474adc84a1b725, State: Running, Role: LEADER
I20260925 13:57:03.273772 21858 raft_consensus.cc:2749] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260925 13:57:03.276809 21862 leader_election.cc:290] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [CANDIDATE]: Term 2 election: Requested vote from peers e942d7f2e4864b5ca08e6e830085caa0 (127.18.182.254:43163), a9bf5a5925ac42769e474adc84a1b725 (127.18.182.252:39283)
I20260925 13:57:03.278115 21598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ea1e11bc6d84114b7616815b6af4635" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a9bf5a5925ac42769e474adc84a1b725"
I20260925 13:57:03.275700 21864 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:03.280702 21864 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 1 FOLLOWER]: Advancing to term 2
W20260925 13:57:03.281155 21866 consensus_queue.cc:1175] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3ea1e11bc6d84114b7616815b6af4635" responder_term: 2 status { last_received { term: 1 index: 4 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer a9bf5a5925ac42769e474adc84a1b725 for earlier term 1. Current term is 2. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260925 13:57:03.312362 21598 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea1e11bc6d84114b7616815b6af4635 in term 2.
I20260925 13:57:03.320480 21496 leader_election.cc:304] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ea1e11bc6d84114b7616815b6af4635, a9bf5a5925ac42769e474adc84a1b725; no voters: e942d7f2e4864b5ca08e6e830085caa0
I20260925 13:57:03.322022 21862 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 13:57:03.316553 21644 catalog_manager.cc:1651] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 13:57:03.334534 21862 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 2 LEADER]: Becoming Leader. State: Replica: 3ea1e11bc6d84114b7616815b6af4635, State: Running, Role: LEADER
I20260925 13:57:03.337157 21862 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:03.353669 21868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ea1e11bc6d84114b7616815b6af4635. Latest consensus state: current_term: 2 leader_uuid: "3ea1e11bc6d84114b7616815b6af4635" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:57:03.354892 21868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [sys.catalog]: This master's current role is: LEADER
I20260925 13:57:03.364426 21870 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260925 13:57:03.374583 21870 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260925 13:57:03.380626 21870 catalog_manager.cc:1308] Loaded cluster ID: d5d0bc5567a74d9cad4d92a9d1041fd9
I20260925 13:57:03.380923 21870 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260925 13:57:03.387656 21870 catalog_manager.cc:1553] Loading token signing keys...
I20260925 13:57:03.392802 21870 catalog_manager.cc:6230] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635: Loaded TSK: 0
I20260925 13:57:03.394438 21870 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260925 13:57:03.400226 21870 catalog_manager.cc:1576] Loading latest processed Hive Metastore notification log event ID...
I20260925 13:57:03.404886 21870 catalog_manager.cc:6068] Last processed Hive Metastore notification event ID: -1
I20260925 13:57:03.849289 21872 raft_consensus.cc:493] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:03.850055 21872 raft_consensus.cc:515] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:03.853250 21443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9bf5a5925ac42769e474adc84a1b725" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e942d7f2e4864b5ca08e6e830085caa0" is_pre_election: true
I20260925 13:57:03.853281 21871 raft_consensus.cc:493] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:03.856304 21519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9bf5a5925ac42769e474adc84a1b725" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ea1e11bc6d84114b7616815b6af4635" is_pre_election: true
I20260925 13:57:03.858207 21872 leader_election.cc:290] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e942d7f2e4864b5ca08e6e830085caa0 (127.18.182.254:43163), 3ea1e11bc6d84114b7616815b6af4635 (127.18.182.253:46177)
I20260925 13:57:03.857446 21871 raft_consensus.cc:515] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:03.859025 21443 raft_consensus.cc:2468] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9bf5a5925ac42769e474adc84a1b725 in term 2.
I20260925 13:57:03.864542 21572 leader_election.cc:304] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a9bf5a5925ac42769e474adc84a1b725, e942d7f2e4864b5ca08e6e830085caa0; no voters: 3ea1e11bc6d84114b7616815b6af4635
I20260925 13:57:03.865649 21872 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260925 13:57:03.865689 21440 raft_consensus.cc:1275] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 2 FOLLOWER]: Refusing update from remote peer 3ea1e11bc6d84114b7616815b6af4635: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260925 13:57:03.865943 21872 raft_consensus.cc:493] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:03.866225 21872 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 2 FOLLOWER]: Advancing to term 3
I20260925 13:57:03.869571 21871 leader_election.cc:290] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ea1e11bc6d84114b7616815b6af4635 (127.18.182.253:46177), a9bf5a5925ac42769e474adc84a1b725 (127.18.182.252:39283)
I20260925 13:57:03.872663 21872 raft_consensus.cc:515] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:03.875789 21598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e942d7f2e4864b5ca08e6e830085caa0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a9bf5a5925ac42769e474adc84a1b725" is_pre_election: true
I20260925 13:57:03.876888 21598 raft_consensus.cc:2393] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e942d7f2e4864b5ca08e6e830085caa0 in current term 3: Already voted for candidate a9bf5a5925ac42769e474adc84a1b725 in this term.
I20260925 13:57:03.883817 21443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9bf5a5925ac42769e474adc84a1b725" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e942d7f2e4864b5ca08e6e830085caa0"
I20260925 13:57:03.884322 21443 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 2 FOLLOWER]: Advancing to term 3
I20260925 13:57:03.885571 21519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e942d7f2e4864b5ca08e6e830085caa0" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ea1e11bc6d84114b7616815b6af4635" is_pre_election: true
I20260925 13:57:03.890679 21422 leader_election.cc:304] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e942d7f2e4864b5ca08e6e830085caa0; no voters: 3ea1e11bc6d84114b7616815b6af4635, a9bf5a5925ac42769e474adc84a1b725
I20260925 13:57:03.891868 21443 raft_consensus.cc:2468] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9bf5a5925ac42769e474adc84a1b725 in term 3.
I20260925 13:57:03.892485 21871 raft_consensus.cc:2749] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260925 13:57:03.893771 21572 leader_election.cc:304] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9bf5a5925ac42769e474adc84a1b725, e942d7f2e4864b5ca08e6e830085caa0; no voters:
W20260925 13:57:03.895393 21466 debug-util.cc:398] Leaking SignalData structure 0x7b08000a0860 after lost signal to thread 21402
W20260925 13:57:03.896041 21466 debug-util.cc:398] Leaking SignalData structure 0x7b08000917a0 after lost signal to thread 21469
I20260925 13:57:03.899983 21873 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 3 FOLLOWER]: Leader election won for term 3
I20260925 13:57:03.906993 21868 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260925 13:57:03.913158 21872 leader_election.cc:290] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [CANDIDATE]: Term 3 election: Requested vote from peers e942d7f2e4864b5ca08e6e830085caa0 (127.18.182.254:43163), 3ea1e11bc6d84114b7616815b6af4635 (127.18.182.253:46177)
I20260925 13:57:03.916330 21519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9bf5a5925ac42769e474adc84a1b725" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3ea1e11bc6d84114b7616815b6af4635"
I20260925 13:57:03.926256 21441 raft_consensus.cc:1240] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 3 FOLLOWER]: Rejecting Update request from peer 3ea1e11bc6d84114b7616815b6af4635 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260925 13:57:03.928682 21868 consensus_queue.cc:1059] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260925 13:57:03.936041 21876 raft_consensus.cc:3055] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 2 LEADER]: Stepping down as leader of term 2
I20260925 13:57:03.936467 21876 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ea1e11bc6d84114b7616815b6af4635, State: Running, Role: LEADER
I20260925 13:57:03.937455 21876 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:03.938848 21873 raft_consensus.cc:697] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 3 LEADER]: Becoming Leader. State: Replica: a9bf5a5925ac42769e474adc84a1b725, State: Running, Role: LEADER
I20260925 13:57:03.939903 21873 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:03.943993 21876 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 2 FOLLOWER]: Advancing to term 3
I20260925 13:57:03.962018 21598 raft_consensus.cc:1240] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 3 LEADER]: Rejecting Update request from peer 3ea1e11bc6d84114b7616815b6af4635 for earlier term 2. Current term is 3. Ops: []
I20260925 13:57:03.963156 21872 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9bf5a5925ac42769e474adc84a1b725. Latest consensus state: current_term: 3 leader_uuid: "a9bf5a5925ac42769e474adc84a1b725" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:57:03.963891 21872 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [sys.catalog]: This master's current role is: LEADER
I20260925 13:57:03.987164 21878 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260925 13:57:04.000341 21878 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260925 13:57:04.004532 21878 catalog_manager.cc:1308] Loaded cluster ID: d5d0bc5567a74d9cad4d92a9d1041fd9
I20260925 13:57:04.005760 21878 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260925 13:57:04.013872 21878 catalog_manager.cc:1553] Loading token signing keys...
I20260925 13:57:04.019207 21878 catalog_manager.cc:6230] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Loaded TSK: 0
I20260925 13:57:04.021880 21878 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260925 13:57:04.027065 21878 catalog_manager.cc:1576] Loading latest processed Hive Metastore notification log event ID...
I20260925 13:57:04.033102 21878 catalog_manager.cc:6068] Last processed Hive Metastore notification event ID: -1
I20260925 13:57:04.125021 21843 heartbeater.cc:499] Master 127.18.182.253:46177 was elected leader, sending a full tablet report...
I20260925 13:57:04.374924 21440 raft_consensus.cc:1275] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 3 FOLLOWER]: Refusing update from remote peer a9bf5a5925ac42769e474adc84a1b725: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260925 13:57:04.377625 21872 consensus_queue.cc:1048] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260925 13:57:04.402311 21871 sys_catalog.cc:455] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9bf5a5925ac42769e474adc84a1b725. Latest consensus state: current_term: 3 leader_uuid: "a9bf5a5925ac42769e474adc84a1b725" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:57:04.403146 21871 sys_catalog.cc:458] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:57:04.439276 21871 sys_catalog.cc:455] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "a9bf5a5925ac42769e474adc84a1b725" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:57:04.440323 21871 sys_catalog.cc:458] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:57:04.450484 21519 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 3 FOLLOWER]: Refusing update from remote peer a9bf5a5925ac42769e474adc84a1b725: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260925 13:57:04.450984 21519 pending_rounds.cc:85] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635: Aborting all ops after (but not including) 4
I20260925 13:57:04.451395 21519 pending_rounds.cc:107] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260925 13:57:04.451756 21519 raft_consensus.cc:2889] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260925 13:57:04.454286 21873 consensus_queue.cc:1048] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260925 13:57:04.462224 21873 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "a9bf5a5925ac42769e474adc84a1b725" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:57:04.463253 21873 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [sys.catalog]: This master's current role is: LEADER
I20260925 13:57:04.489725 21876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9bf5a5925ac42769e474adc84a1b725. Latest consensus state: current_term: 3 leader_uuid: "a9bf5a5925ac42769e474adc84a1b725" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:57:04.490448 21876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:57:04.491161 21884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "a9bf5a5925ac42769e474adc84a1b725" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:57:04.492713 21884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:57:04.641925 21842 heartbeater.cc:499] Master 127.18.182.252:39283 was elected leader, sending a full tablet report...
W20260925 13:57:04.600776 21859 debug-util.cc:398] Leaking SignalData structure 0x7b0800037240 after lost signal to thread 21849
W20260925 13:57:03.203097 21861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:57:04.796289 21849 thread.cc:641] GCE (cloud detector) Time spent creating pthread: real 1.595s user 0.472s sys 0.842s
W20260925 13:57:04.796854 21849 thread.cc:608] GCE (cloud detector) Time spent starting thread: real 1.596s user 0.472s sys 0.842s
W20260925 13:57:04.805058 21887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:57:04.805157 21849 server_base.cc:1061] running on GCE node
I20260925 13:57:04.806324 21849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:57:04.808987 21849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:57:04.810406 21849 hybrid_clock.cc:648] HybridClock initialized: now 1790344624810348 us; error 59 us; skew 500 ppm
I20260925 13:57:04.811196 21849 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 13:57:04.818511 21849 webserver.cc:533] Webserver started at http://127.18.182.194:45107/ using document root <none> and password file <none>
I20260925 13:57:04.819499 21849 fs_manager.cc:362] Metadata directory not provided
I20260925 13:57:04.819777 21849 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:57:04.820343 21849 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:57:04.825037 21849 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/data/instance:
uuid: "145e520e0a5a445d943a53cf8c27efee"
format_stamp: "Formatted at 2026-09-25 13:57:04 on dist-test-slave-nlqf"
I20260925 13:57:04.826165 21849 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/instance:
uuid: "145e520e0a5a445d943a53cf8c27efee"
format_stamp: "Formatted at 2026-09-25 13:57:04 on dist-test-slave-nlqf"
I20260925 13:57:04.834018 21849 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.005s sys 0.005s
I20260925 13:57:04.840078 21892 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:57:04.841652 21849 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260925 13:57:04.842075 21849 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal
uuid: "145e520e0a5a445d943a53cf8c27efee"
format_stamp: "Formatted at 2026-09-25 13:57:04 on dist-test-slave-nlqf"
I20260925 13:57:04.842525 21849 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:57:04.882143 21873 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "a9bf5a5925ac42769e474adc84a1b725" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:57:04.882905 21873 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [sys.catalog]: This master's current role is: LEADER
I20260925 13:57:04.913466 21871 raft_consensus.cc:493] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a9bf5a5925ac42769e474adc84a1b725)
I20260925 13:57:04.913890 21871 raft_consensus.cc:515] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:04.915779 21871 leader_election.cc:290] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ea1e11bc6d84114b7616815b6af4635 (127.18.182.253:46177), a9bf5a5925ac42769e474adc84a1b725 (127.18.182.252:39283)
I20260925 13:57:04.916987 21519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e942d7f2e4864b5ca08e6e830085caa0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ea1e11bc6d84114b7616815b6af4635" is_pre_election: true
I20260925 13:57:04.918782 21598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e942d7f2e4864b5ca08e6e830085caa0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a9bf5a5925ac42769e474adc84a1b725" is_pre_election: true
I20260925 13:57:04.920181 21422 leader_election.cc:304] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e942d7f2e4864b5ca08e6e830085caa0; no voters: 3ea1e11bc6d84114b7616815b6af4635, a9bf5a5925ac42769e474adc84a1b725
I20260925 13:57:04.920787 21871 raft_consensus.cc:2749] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260925 13:57:04.930642 21849 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260925 13:57:04.933099 21849 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:57:04.933794 21849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:57:04.937973 21849 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260925 13:57:04.944716 21849 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:57:04.945035 21849 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:57:04.945353 21849 ts_tablet_manager.cc:616] Registered 0 tablets
I20260925 13:57:04.945582 21849 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:57:05.139995 21849 rpc_server.cc:307] RPC server started. Bound to: 127.18.182.194:44291
I20260925 13:57:05.140230 22004 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.182.194:44291 every 8 connection(s)
I20260925 13:57:05.143817 21849 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/data/info.pb
I20260925 13:57:05.144394 19163 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskYmJttM/build/tsan/bin/kudu as pid 21849
I20260925 13:57:05.145131 19163 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/instance
I20260925 13:57:05.167128 19163 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskYmJttM/build/tsan/bin/kudu
/tmp/dist-test-taskYmJttM/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.182.195:0
--local_ip_for_outbound_sockets=127.18.182.195
--webserver_interface=127.18.182.195
--webserver_port=0
--tserver_master_addrs=127.18.182.254:43163,127.18.182.253:46177,127.18.182.252:39283
--builtin_ntp_servers=127.18.182.212:40529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260925 13:57:05.224079 22005 heartbeater.cc:344] Connected to a master server at 127.18.182.252:39283
I20260925 13:57:05.224745 22005 heartbeater.cc:461] Registering TS with master...
I20260925 13:57:05.226050 22005 heartbeater.cc:507] Master 127.18.182.252:39283 requested a full tablet report, sending...
I20260925 13:57:05.226562 22008 heartbeater.cc:344] Connected to a master server at 127.18.182.253:46177
I20260925 13:57:05.226938 22008 heartbeater.cc:461] Registering TS with master...
I20260925 13:57:05.227865 22008 heartbeater.cc:507] Master 127.18.182.253:46177 requested a full tablet report, sending...
I20260925 13:57:05.230214 21588 ts_manager.cc:194] Registered new tserver with Master: 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:05.232705 21588 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.182.194:37579
I20260925 13:57:05.235533 21509 ts_manager.cc:194] Registered new tserver with Master: 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:05.243096 22009 heartbeater.cc:344] Connected to a master server at 127.18.182.254:43163
I20260925 13:57:05.243500 22009 heartbeater.cc:461] Registering TS with master...
I20260925 13:57:05.244234 22009 heartbeater.cc:507] Master 127.18.182.254:43163 requested a full tablet report, sending...
I20260925 13:57:05.246917 21435 ts_manager.cc:194] Registered new tserver with Master: 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:05.387068 22016 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a9bf5a5925ac42769e474adc84a1b725)
I20260925 13:57:05.387681 22016 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:05.390012 22016 leader_election.cc:290] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e942d7f2e4864b5ca08e6e830085caa0 (127.18.182.254:43163), a9bf5a5925ac42769e474adc84a1b725 (127.18.182.252:39283)
I20260925 13:57:05.390905 21441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ea1e11bc6d84114b7616815b6af4635" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e942d7f2e4864b5ca08e6e830085caa0" is_pre_election: true
I20260925 13:57:05.391674 21598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ea1e11bc6d84114b7616815b6af4635" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a9bf5a5925ac42769e474adc84a1b725" is_pre_election: true
I20260925 13:57:05.393159 21496 leader_election.cc:304] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ea1e11bc6d84114b7616815b6af4635; no voters: a9bf5a5925ac42769e474adc84a1b725, e942d7f2e4864b5ca08e6e830085caa0
I20260925 13:57:05.394091 22016 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260925 13:57:05.568219 22012 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:57:05.568733 22012 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:57:05.569301 22012 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260925 13:57:05.606657 22012 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:57:05.607636 22012 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.182.195
I20260925 13:57:05.647994 22012 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.182.212:40529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.182.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.182.195
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.18.182.254:43163,127.18.182.253:46177,127.18.182.252:39283
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.182.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 102683f65af844d269565345d7d8c6ebc55c92c5
build type FASTDEBUG
built by None at 25 Sep 2026 13:43:25 UTC on e7f111948823
build id 13994
TSAN enabled
I20260925 13:57:05.649360 22012 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:57:05.650969 22012 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260925 13:57:05.666270 22022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:57:05.741611 22005 heartbeater.cc:499] Master 127.18.182.252:39283 was elected leader, sending a full tablet report...
I20260925 13:57:06.400067 22028 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a9bf5a5925ac42769e474adc84a1b725)
I20260925 13:57:06.407634 22028 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:06.414973 22028 leader_election.cc:290] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e942d7f2e4864b5ca08e6e830085caa0 (127.18.182.254:43163), a9bf5a5925ac42769e474adc84a1b725 (127.18.182.252:39283)
I20260925 13:57:06.416271 21441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ea1e11bc6d84114b7616815b6af4635" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e942d7f2e4864b5ca08e6e830085caa0" is_pre_election: true
I20260925 13:57:06.417999 21598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ea1e11bc6d84114b7616815b6af4635" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a9bf5a5925ac42769e474adc84a1b725" is_pre_election: true
I20260925 13:57:06.421100 21496 leader_election.cc:304] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ea1e11bc6d84114b7616815b6af4635; no voters: a9bf5a5925ac42769e474adc84a1b725, e942d7f2e4864b5ca08e6e830085caa0
I20260925 13:57:06.422636 22028 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260925 13:57:06.494531 22030 raft_consensus.cc:493] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a9bf5a5925ac42769e474adc84a1b725)
I20260925 13:57:06.495139 22030 raft_consensus.cc:515] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:06.501493 21519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e942d7f2e4864b5ca08e6e830085caa0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ea1e11bc6d84114b7616815b6af4635" is_pre_election: true
I20260925 13:57:06.505106 21598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e942d7f2e4864b5ca08e6e830085caa0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a9bf5a5925ac42769e474adc84a1b725" is_pre_election: true
I20260925 13:57:06.501094 22030 leader_election.cc:290] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ea1e11bc6d84114b7616815b6af4635 (127.18.182.253:46177), a9bf5a5925ac42769e474adc84a1b725 (127.18.182.252:39283)
I20260925 13:57:06.507532 21422 leader_election.cc:304] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e942d7f2e4864b5ca08e6e830085caa0; no voters: 3ea1e11bc6d84114b7616815b6af4635, a9bf5a5925ac42769e474adc84a1b725
I20260925 13:57:06.508931 22030 raft_consensus.cc:2749] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260925 13:57:07.085300 22031 raft_consensus.cc:493] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a9bf5a5925ac42769e474adc84a1b725)
I20260925 13:57:07.089812 22031 raft_consensus.cc:515] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:07.092689 21519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e942d7f2e4864b5ca08e6e830085caa0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3ea1e11bc6d84114b7616815b6af4635" is_pre_election: true
I20260925 13:57:07.095924 21598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e942d7f2e4864b5ca08e6e830085caa0" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a9bf5a5925ac42769e474adc84a1b725" is_pre_election: true
I20260925 13:57:07.097564 21422 leader_election.cc:304] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e942d7f2e4864b5ca08e6e830085caa0; no voters: 3ea1e11bc6d84114b7616815b6af4635, a9bf5a5925ac42769e474adc84a1b725
I20260925 13:57:07.103329 22031 leader_election.cc:290] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ea1e11bc6d84114b7616815b6af4635 (127.18.182.253:46177), a9bf5a5925ac42769e474adc84a1b725 (127.18.182.252:39283)
I20260925 13:57:07.104370 22031 raft_consensus.cc:2749] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260925 13:57:07.400242 22033 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a9bf5a5925ac42769e474adc84a1b725)
I20260925 13:57:07.403074 22033 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:07.408684 21441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ea1e11bc6d84114b7616815b6af4635" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e942d7f2e4864b5ca08e6e830085caa0" is_pre_election: true
I20260925 13:57:07.414980 21598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ea1e11bc6d84114b7616815b6af4635" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a9bf5a5925ac42769e474adc84a1b725" is_pre_election: true
I20260925 13:57:07.416975 21496 leader_election.cc:304] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ea1e11bc6d84114b7616815b6af4635; no voters: a9bf5a5925ac42769e474adc84a1b725, e942d7f2e4864b5ca08e6e830085caa0
I20260925 13:57:07.418732 22034 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260925 13:57:07.419472 22033 leader_election.cc:290] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e942d7f2e4864b5ca08e6e830085caa0 (127.18.182.254:43163), a9bf5a5925ac42769e474adc84a1b725 (127.18.182.252:39283)
W20260925 13:57:07.069301 22021 debug-util.cc:398] Leaking SignalData structure 0x7b0800037240 after lost signal to thread 22012
W20260925 13:57:07.663724 22012 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.996s user 0.615s sys 1.270s
W20260925 13:57:05.668002 22023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:57:07.664314 22012 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.997s user 0.615s sys 1.270s
W20260925 13:57:07.665158 22024 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1996 milliseconds
I20260925 13:57:07.666136 22012 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
W20260925 13:57:07.666191 22025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:57:07.669128 22012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:57:07.671089 22012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:57:07.672394 22012 hybrid_clock.cc:648] HybridClock initialized: now 1790344627672371 us; error 46 us; skew 500 ppm
I20260925 13:57:07.673864 22012 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 13:57:07.680503 22012 webserver.cc:533] Webserver started at http://127.18.182.195:41107/ using document root <none> and password file <none>
I20260925 13:57:07.681461 22012 fs_manager.cc:362] Metadata directory not provided
I20260925 13:57:07.681701 22012 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:57:07.682164 22012 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:57:07.686723 22012 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/data/instance:
uuid: "ee02f77594864349982492862d668a36"
format_stamp: "Formatted at 2026-09-25 13:57:07 on dist-test-slave-nlqf"
I20260925 13:57:07.687929 22012 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/instance:
uuid: "ee02f77594864349982492862d668a36"
format_stamp: "Formatted at 2026-09-25 13:57:07 on dist-test-slave-nlqf"
I20260925 13:57:07.695469 22012 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.006s sys 0.002s
I20260925 13:57:07.701037 22039 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:57:07.702193 22012 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260925 13:57:07.702512 22012 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal
uuid: "ee02f77594864349982492862d668a36"
format_stamp: "Formatted at 2026-09-25 13:57:07 on dist-test-slave-nlqf"
I20260925 13:57:07.702813 22012 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:57:07.754690 22012 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260925 13:57:07.756407 22012 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:57:07.756861 22012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:57:07.759969 22012 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260925 13:57:07.764917 22012 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:57:07.765153 22012 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:57:07.765407 22012 ts_tablet_manager.cc:616] Registered 0 tablets
I20260925 13:57:07.765575 22012 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:57:07.900795 22012 rpc_server.cc:307] RPC server started. Bound to: 127.18.182.195:40873
I20260925 13:57:07.900909 22151 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.182.195:40873 every 8 connection(s)
I20260925 13:57:07.903661 22012 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/data/info.pb
I20260925 13:57:07.912492 19163 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskYmJttM/build/tsan/bin/kudu as pid 22012
I20260925 13:57:07.913071 19163 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/instance
I20260925 13:57:07.953622 22152 heartbeater.cc:344] Connected to a master server at 127.18.182.252:39283
I20260925 13:57:07.953680 22154 heartbeater.cc:344] Connected to a master server at 127.18.182.254:43163
I20260925 13:57:07.953882 22153 heartbeater.cc:344] Connected to a master server at 127.18.182.253:46177
I20260925 13:57:07.954190 22152 heartbeater.cc:461] Registering TS with master...
I20260925 13:57:07.954191 22154 heartbeater.cc:461] Registering TS with master...
I20260925 13:57:07.954212 22153 heartbeater.cc:461] Registering TS with master...
I20260925 13:57:07.955639 22152 heartbeater.cc:507] Master 127.18.182.252:39283 requested a full tablet report, sending...
I20260925 13:57:07.955683 22153 heartbeater.cc:507] Master 127.18.182.253:46177 requested a full tablet report, sending...
I20260925 13:57:07.955701 22154 heartbeater.cc:507] Master 127.18.182.254:43163 requested a full tablet report, sending...
I20260925 13:57:07.958683 21587 ts_manager.cc:194] Registered new tserver with Master: ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:07.958971 21509 ts_manager.cc:194] Registered new tserver with Master: ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:07.961210 21435 ts_manager.cc:194] Registered new tserver with Master: ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:07.961192 21587 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.18.182.195:53791
I20260925 13:57:07.979010 19163 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260925 13:57:08.028777 21588 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35994:
name: "default.Table_366914d3ae844485b147c75792717126"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:57:08.028556 21587 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35994:
name: "default.Table_492c567ddda14e77851270f34471730e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:57:08.031575 21586 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35994:
name: "default.Table_63ea1ad4e7904f8791170dd4da08d74e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:57:08.034507 21585 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35994:
name: "default.Table_376a1428b63b4da3ad04f4491ce69d3b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:57:08.041072 21588 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_366914d3ae844485b147c75792717126 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 13:57:08.053769 21585 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_376a1428b63b4da3ad04f4491ce69d3b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 13:57:08.055070 21586 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_63ea1ad4e7904f8791170dd4da08d74e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 13:57:08.053797 21587 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_492c567ddda14e77851270f34471730e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:57:08.074193 19163 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskYmJttM/build/tsan/bin/kudu with pid 21471
W20260925 13:57:08.186682 21843 heartbeater.cc:646] Failed to heartbeat to 127.18.182.253:46177 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.182.253:46177: connect: Connection refused (error 111)
W20260925 13:57:08.263237 22008 heartbeater.cc:646] Failed to heartbeat to 127.18.182.253:46177 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.182.253:46177: connect: Connection refused (error 111)
W20260925 13:57:08.416914 21573 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.182.253:46177: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260925 13:57:08.433379 21573 consensus_peers.cc:597] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 -> Peer 3ea1e11bc6d84114b7616815b6af4635 (127.18.182.253:46177): Couldn't send request to peer 3ea1e11bc6d84114b7616815b6af4635. Status: Network error: Client connection negotiation failed: client connection to 127.18.182.253:46177: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 13:57:08.464478 22152 heartbeater.cc:499] Master 127.18.182.252:39283 was elected leader, sending a full tablet report...
W20260925 13:57:08.483932 22153 heartbeater.cc:646] Failed to heartbeat to 127.18.182.253:46177 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.182.253:46177: connect: Connection refused (error 111)
I20260925 13:57:08.567672 21588 catalog_manager.cc:2213] created HMS catalog entry for table [id=74ee67384ecf4220b7d08fdf0637aefb]
I20260925 13:57:08.612255 21585 catalog_manager.cc:2213] created HMS catalog entry for table [id=9d18288ef656432994079ec83988ec3f]
I20260925 13:57:08.672737 21587 catalog_manager.cc:2213] created HMS catalog entry for table [id=754c40502e614c5bb18230c852aa0016]
I20260925 13:57:08.716966 21585 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35994:
table { table_name: "default.Table_366914d3ae844485b147c75792717126" } new_table_name: "default.Table_51d65314582841419e34561a4a8d6bbb" modify_external_catalogs: true
I20260925 13:57:08.724412 21588 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35994:
table { table_name: "default.Table_376a1428b63b4da3ad04f4491ce69d3b" } new_table_name: "default.Table_01a6dec392d24e9f89578f6f4274c704" modify_external_catalogs: true
I20260925 13:57:08.756287 21587 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35994:
table { table_name: "default.Table_492c567ddda14e77851270f34471730e" } modify_external_catalogs: true
I20260925 13:57:08.757128 21587 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35994:
table { table_name: "default.Table_492c567ddda14e77851270f34471730e" } modify_external_catalogs: true
I20260925 13:57:08.775122 21586 catalog_manager.cc:2213] created HMS catalog entry for table [id=af7e917a89ce4ca88be6ee9f143481cf]
I20260925 13:57:08.811527 21940 tablet_service.cc:1511] Processing CreateTablet for tablet d267def3f0864435a0d2c40fd7052eab (DEFAULT_TABLE table=default.table_376a1428b63b4da3ad04f4491ce69d3b [id=9d18288ef656432994079ec83988ec3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:08.813377 21940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d267def3f0864435a0d2c40fd7052eab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:08.814338 21777 tablet_service.cc:1511] Processing CreateTablet for tablet d267def3f0864435a0d2c40fd7052eab (DEFAULT_TABLE table=default.table_376a1428b63b4da3ad04f4491ce69d3b [id=9d18288ef656432994079ec83988ec3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:08.816215 21777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d267def3f0864435a0d2c40fd7052eab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:08.816520 21939 tablet_service.cc:1511] Processing CreateTablet for tablet 4edff719362a4c3287bd7775c020d27b (DEFAULT_TABLE table=default.table_366914d3ae844485b147c75792717126 [id=74ee67384ecf4220b7d08fdf0637aefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:08.817574 21939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4edff719362a4c3287bd7775c020d27b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:08.816587 21776 tablet_service.cc:1511] Processing CreateTablet for tablet 4edff719362a4c3287bd7775c020d27b (DEFAULT_TABLE table=default.table_366914d3ae844485b147c75792717126 [id=74ee67384ecf4220b7d08fdf0637aefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:08.818527 21776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4edff719362a4c3287bd7775c020d27b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:08.846069 21938 tablet_service.cc:1511] Processing CreateTablet for tablet 0972853b828c41c882c68f4326c1ea63 (DEFAULT_TABLE table=default.table_492c567ddda14e77851270f34471730e [id=754c40502e614c5bb18230c852aa0016]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:08.846694 21775 tablet_service.cc:1511] Processing CreateTablet for tablet 0972853b828c41c882c68f4326c1ea63 (DEFAULT_TABLE table=default.table_492c567ddda14e77851270f34471730e [id=754c40502e614c5bb18230c852aa0016]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:08.847575 21938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0972853b828c41c882c68f4326c1ea63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:08.847865 21775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0972853b828c41c882c68f4326c1ea63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:08.851899 22087 tablet_service.cc:1511] Processing CreateTablet for tablet d267def3f0864435a0d2c40fd7052eab (DEFAULT_TABLE table=default.table_376a1428b63b4da3ad04f4491ce69d3b [id=9d18288ef656432994079ec83988ec3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:08.854179 22087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d267def3f0864435a0d2c40fd7052eab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:08.865518 22086 tablet_service.cc:1511] Processing CreateTablet for tablet 4edff719362a4c3287bd7775c020d27b (DEFAULT_TABLE table=default.table_366914d3ae844485b147c75792717126 [id=74ee67384ecf4220b7d08fdf0637aefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:08.866645 22086 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4edff719362a4c3287bd7775c020d27b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:08.869120 22085 tablet_service.cc:1511] Processing CreateTablet for tablet 0972853b828c41c882c68f4326c1ea63 (DEFAULT_TABLE table=default.table_492c567ddda14e77851270f34471730e [id=754c40502e614c5bb18230c852aa0016]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:08.870261 22085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0972853b828c41c882c68f4326c1ea63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:08.876201 21583 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35994:
name: "default.Table_8b5db7d66475449d9772db568c9ded56"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:57:08.876595 21586 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35994:
table { table_name: "default.Table_63ea1ad4e7904f8791170dd4da08d74e" } modify_external_catalogs: true
I20260925 13:57:08.877305 21586 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35994:
table { table_name: "default.Table_63ea1ad4e7904f8791170dd4da08d74e" } modify_external_catalogs: true
I20260925 13:57:08.885109 21584 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35994:
name: "default.Table_b17685eaeada404790103bd89711435e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:57:08.907696 22210 tablet_bootstrap.cc:492] T d267def3f0864435a0d2c40fd7052eab P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:08.916087 21776 tablet_service.cc:1511] Processing CreateTablet for tablet ffeba794dc43421fab68478796f67e92 (DEFAULT_TABLE table=default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:08.917094 21776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffeba794dc43421fab68478796f67e92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:08.917001 22084 tablet_service.cc:1511] Processing CreateTablet for tablet ffeba794dc43421fab68478796f67e92 (DEFAULT_TABLE table=default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:08.918126 22084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffeba794dc43421fab68478796f67e92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:08.922237 21582 master_service.cc:960] ReplaceTablet: received request to replace tablet d267def3f0864435a0d2c40fd7052eab from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:08.923554 21937 tablet_service.cc:1511] Processing CreateTablet for tablet ffeba794dc43421fab68478796f67e92 (DEFAULT_TABLE table=default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:08.928064 21937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffeba794dc43421fab68478796f67e92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:08.943782 21582 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet d267def3f0864435a0d2c40fd7052eab
I20260925 13:57:08.947384 21777 tablet_service.cc:1558] Processing DeleteTablet for tablet d267def3f0864435a0d2c40fd7052eab with delete_type TABLET_DATA_DELETED (replaced by tablet 5c593c8aa09c4423b6c7d7bbcf94f4ab) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:08.948678 21582 catalog_manager.cc:6934] ReplaceTablet: tablet d267def3f0864435a0d2c40fd7052eab deleted and replaced by tablet 5c593c8aa09c4423b6c7d7bbcf94f4ab
I20260925 13:57:08.949448 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet d267def3f0864435a0d2c40fd7052eab with delete_type TABLET_DATA_DELETED (replaced by tablet 5c593c8aa09c4423b6c7d7bbcf94f4ab) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:08.950414 22211 tablet_bootstrap.cc:492] T d267def3f0864435a0d2c40fd7052eab P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:08.951668 21938 tablet_service.cc:1558] Processing DeleteTablet for tablet d267def3f0864435a0d2c40fd7052eab with delete_type TABLET_DATA_DELETED (replaced by tablet 5c593c8aa09c4423b6c7d7bbcf94f4ab) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:08.957523 21581 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35994:
name: "default.Table_bd714c618f7a4f16a69a69d5932b2014"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:57:08.960167 21575 catalog_manager.cc:5043] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): delete failed for tablet d267def3f0864435a0d2c40fd7052eab because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d267def3f0864435a0d2c40fd7052eab already in progress: creating tablet
W20260925 13:57:08.961277 21572 catalog_manager.cc:5043] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): delete failed for tablet d267def3f0864435a0d2c40fd7052eab because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d267def3f0864435a0d2c40fd7052eab already in progress: creating tablet
I20260925 13:57:08.981480 22210 tablet_bootstrap.cc:654] T d267def3f0864435a0d2c40fd7052eab P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:08.982126 21582 master_service.cc:960] ReplaceTablet: received request to replace tablet 4edff719362a4c3287bd7775c020d27b from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:08.984295 22210 log.cc:826] T d267def3f0864435a0d2c40fd7052eab P 5e2731975dbf433aade4a859df9699e2: Log is configured to *not* fsync() on all Append() calls
I20260925 13:57:08.990209 19163 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskYmJttM/build/tsan/bin/kudu
/tmp/dist-test-taskYmJttM/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.182.253:46177
--webserver_interface=127.18.182.253
--webserver_port=33753
--builtin_ntp_servers=127.18.182.212:40529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.182.254:43163,127.18.182.253:46177,127.18.182.252:39283
--hive_metastore_uris=thrift://127.0.0.1:39557
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260925 13:57:08.999783 22212 tablet_bootstrap.cc:492] T 4edff719362a4c3287bd7775c020d27b P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:09.010627 22211 tablet_bootstrap.cc:654] T d267def3f0864435a0d2c40fd7052eab P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:09.014161 21776 tablet_service.cc:1511] Processing CreateTablet for tablet 5c593c8aa09c4423b6c7d7bbcf94f4ab (DEFAULT_TABLE table=default.table_376a1428b63b4da3ad04f4491ce69d3b [id=9d18288ef656432994079ec83988ec3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:09.015280 21776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c593c8aa09c4423b6c7d7bbcf94f4ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:09.016726 21938 tablet_service.cc:1511] Processing CreateTablet for tablet 5c593c8aa09c4423b6c7d7bbcf94f4ab (DEFAULT_TABLE table=default.table_376a1428b63b4da3ad04f4491ce69d3b [id=9d18288ef656432994079ec83988ec3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:09.017776 21938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c593c8aa09c4423b6c7d7bbcf94f4ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:09.023766 22210 tablet_bootstrap.cc:492] T d267def3f0864435a0d2c40fd7052eab P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:09.024322 22210 ts_tablet_manager.cc:1403] T d267def3f0864435a0d2c40fd7052eab P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.117s user 0.008s sys 0.008s
I20260925 13:57:09.024631 22201 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "a9bf5a5925ac42769e474adc84a1b725" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:57:09.025477 22201 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [sys.catalog]: This master's current role is: LEADER
I20260925 13:57:09.036726 22212 tablet_bootstrap.cc:654] T 4edff719362a4c3287bd7775c020d27b P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:09.038920 21582 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet 4edff719362a4c3287bd7775c020d27b
I20260925 13:57:09.040853 21582 catalog_manager.cc:6934] ReplaceTablet: tablet 4edff719362a4c3287bd7775c020d27b deleted and replaced by tablet ccefd36f4e2a48f9b3d66cb147a022b5
W20260925 13:57:09.045163 21575 catalog_manager.cc:5030] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): delete failed for tablet d267def3f0864435a0d2c40fd7052eab because the tablet was not found. No further retry: Not found: Tablet not found: d267def3f0864435a0d2c40fd7052eab
I20260925 13:57:09.045107 22083 tablet_service.cc:1511] Processing CreateTablet for tablet 5c593c8aa09c4423b6c7d7bbcf94f4ab (DEFAULT_TABLE table=default.table_376a1428b63b4da3ad04f4491ce69d3b [id=9d18288ef656432994079ec83988ec3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:09.046278 22083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c593c8aa09c4423b6c7d7bbcf94f4ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:09.047147 21937 tablet_service.cc:1558] Processing DeleteTablet for tablet 4edff719362a4c3287bd7775c020d27b with delete_type TABLET_DATA_DELETED (replaced by tablet ccefd36f4e2a48f9b3d66cb147a022b5) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:09.047595 22087 tablet_service.cc:1558] Processing DeleteTablet for tablet 4edff719362a4c3287bd7775c020d27b with delete_type TABLET_DATA_DELETED (replaced by tablet ccefd36f4e2a48f9b3d66cb147a022b5) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:09.048115 21777 tablet_service.cc:1558] Processing DeleteTablet for tablet 4edff719362a4c3287bd7775c020d27b with delete_type TABLET_DATA_DELETED (replaced by tablet ccefd36f4e2a48f9b3d66cb147a022b5) from {username='slave'} at 127.0.0.1:34008
W20260925 13:57:09.049340 21575 catalog_manager.cc:5043] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): delete failed for tablet 4edff719362a4c3287bd7775c020d27b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4edff719362a4c3287bd7775c020d27b already in progress: creating tablet
W20260925 13:57:09.049772 21572 catalog_manager.cc:5043] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): delete failed for tablet 4edff719362a4c3287bd7775c020d27b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4edff719362a4c3287bd7775c020d27b already in progress: creating tablet
W20260925 13:57:09.050151 21575 catalog_manager.cc:5043] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): delete failed for tablet 4edff719362a4c3287bd7775c020d27b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4edff719362a4c3287bd7775c020d27b already in progress: creating tablet
I20260925 13:57:09.039196 22212 log.cc:826] T 4edff719362a4c3287bd7775c020d27b P ee02f77594864349982492862d668a36: Log is configured to *not* fsync() on all Append() calls
I20260925 13:57:09.073922 22211 log.cc:826] T d267def3f0864435a0d2c40fd7052eab P 145e520e0a5a445d943a53cf8c27efee: Log is configured to *not* fsync() on all Append() calls
I20260925 13:57:09.082518 22212 tablet_bootstrap.cc:492] T 4edff719362a4c3287bd7775c020d27b P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:09.083647 22212 ts_tablet_manager.cc:1403] T 4edff719362a4c3287bd7775c020d27b P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.098s user 0.008s sys 0.010s
W20260925 13:57:09.089334 21573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.253:46177, user_credentials={real_user=slave}} blocked reactor thread for 61684.5us
I20260925 13:57:09.090099 21777 tablet_service.cc:1511] Processing CreateTablet for tablet ccefd36f4e2a48f9b3d66cb147a022b5 (DEFAULT_TABLE table=default.table_366914d3ae844485b147c75792717126 [id=74ee67384ecf4220b7d08fdf0637aefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:09.091254 21777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccefd36f4e2a48f9b3d66cb147a022b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:09.092728 21582 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35994:
name: "default.Table_8008d16a92cf4c0dafb258a1122bffb5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:57:09.106503 21938 tablet_service.cc:1511] Processing CreateTablet for tablet ccefd36f4e2a48f9b3d66cb147a022b5 (DEFAULT_TABLE table=default.table_366914d3ae844485b147c75792717126 [id=74ee67384ecf4220b7d08fdf0637aefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:09.107028 22211 tablet_bootstrap.cc:492] T d267def3f0864435a0d2c40fd7052eab P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:09.107578 21938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccefd36f4e2a48f9b3d66cb147a022b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:09.112164 22083 tablet_service.cc:1511] Processing CreateTablet for tablet ccefd36f4e2a48f9b3d66cb147a022b5 (DEFAULT_TABLE table=default.table_366914d3ae844485b147c75792717126 [id=74ee67384ecf4220b7d08fdf0637aefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:09.113241 22083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccefd36f4e2a48f9b3d66cb147a022b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:09.107568 22211 ts_tablet_manager.cc:1403] T d267def3f0864435a0d2c40fd7052eab P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.158s user 0.015s sys 0.009s
I20260925 13:57:09.123816 22210 raft_consensus.cc:359] T d267def3f0864435a0d2c40fd7052eab P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.134202 22210 raft_consensus.cc:385] T d267def3f0864435a0d2c40fd7052eab P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:09.134619 22210 raft_consensus.cc:740] T d267def3f0864435a0d2c40fd7052eab P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:09.135747 22210 consensus_queue.cc:260] T d267def3f0864435a0d2c40fd7052eab P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.152441 22210 ts_tablet_manager.cc:1434] T d267def3f0864435a0d2c40fd7052eab P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.128s user 0.031s sys 0.005s
I20260925 13:57:09.153391 22210 tablet_bootstrap.cc:492] T 0972853b828c41c882c68f4326c1ea63 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
W20260925 13:57:09.154578 21580 catalog_manager.cc:5671] Ignoring report from unknown tablet d267def3f0864435a0d2c40fd7052eab
W20260925 13:57:09.161752 21580 catalog_manager.cc:5671] Ignoring report from unknown tablet d267def3f0864435a0d2c40fd7052eab
I20260925 13:57:09.161458 22212 raft_consensus.cc:359] T 4edff719362a4c3287bd7775c020d27b P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.150535 22211 raft_consensus.cc:359] T d267def3f0864435a0d2c40fd7052eab P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.164597 22211 raft_consensus.cc:385] T d267def3f0864435a0d2c40fd7052eab P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:09.164978 22211 raft_consensus.cc:740] T d267def3f0864435a0d2c40fd7052eab P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:09.165995 22211 consensus_queue.cc:260] T d267def3f0864435a0d2c40fd7052eab P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.166685 22210 tablet_bootstrap.cc:654] T 0972853b828c41c882c68f4326c1ea63 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:09.168138 21580 master_service.cc:960] ReplaceTablet: received request to replace tablet 4edff719362a4c3287bd7775c020d27b from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:09.170481 22212 raft_consensus.cc:385] T 4edff719362a4c3287bd7775c020d27b P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:09.170969 22212 raft_consensus.cc:740] T 4edff719362a4c3287bd7775c020d27b P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:09.172432 22212 consensus_queue.cc:260] T 4edff719362a4c3287bd7775c020d27b P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.177261 22211 ts_tablet_manager.cc:1434] T d267def3f0864435a0d2c40fd7052eab P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.058s user 0.029s sys 0.008s
W20260925 13:57:09.179260 21580 catalog_manager.cc:5671] Ignoring report from unknown tablet d267def3f0864435a0d2c40fd7052eab
I20260925 13:57:09.178329 22211 tablet_bootstrap.cc:492] T 4edff719362a4c3287bd7775c020d27b P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
W20260925 13:57:09.184412 21579 catalog_manager.cc:5671] Ignoring report from unknown tablet d267def3f0864435a0d2c40fd7052eab
I20260925 13:57:09.189502 22211 tablet_bootstrap.cc:654] T 4edff719362a4c3287bd7775c020d27b P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:09.193084 22210 tablet_bootstrap.cc:492] T 0972853b828c41c882c68f4326c1ea63 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:09.193557 22210 ts_tablet_manager.cc:1403] T 0972853b828c41c882c68f4326c1ea63 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.040s user 0.003s sys 0.011s
I20260925 13:57:09.196440 22210 raft_consensus.cc:359] T 0972853b828c41c882c68f4326c1ea63 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.199141 22210 raft_consensus.cc:385] T 0972853b828c41c882c68f4326c1ea63 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:09.202155 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet 4edff719362a4c3287bd7775c020d27b with delete_type TABLET_DATA_DELETED (replaced by tablet ccefd36f4e2a48f9b3d66cb147a022b5) from {username='slave'} at 127.0.0.1:50052
W20260925 13:57:09.203945 21575 catalog_manager.cc:5043] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): delete failed for tablet 4edff719362a4c3287bd7775c020d27b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4edff719362a4c3287bd7775c020d27b already in progress: creating tablet
I20260925 13:57:09.205478 22212 ts_tablet_manager.cc:1434] T 4edff719362a4c3287bd7775c020d27b P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.122s user 0.024s sys 0.014s
I20260925 13:57:09.212030 22212 tablet_bootstrap.cc:492] T ffeba794dc43421fab68478796f67e92 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:09.213979 22210 raft_consensus.cc:740] T 0972853b828c41c882c68f4326c1ea63 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:09.215004 22210 consensus_queue.cc:260] T 0972853b828c41c882c68f4326c1ea63 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.215675 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet 4edff719362a4c3287bd7775c020d27b with delete_type TABLET_DATA_DELETED (replaced by tablet ccefd36f4e2a48f9b3d66cb147a022b5) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:09.216488 22214 tablet_replica.cc:333] T 4edff719362a4c3287bd7775c020d27b P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:09.217293 22214 raft_consensus.cc:2243] T 4edff719362a4c3287bd7775c020d27b P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:09.217875 22214 raft_consensus.cc:2272] T 4edff719362a4c3287bd7775c020d27b P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:09.222491 22214 ts_tablet_manager.cc:1916] T 4edff719362a4c3287bd7775c020d27b P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:09.227016 22210 ts_tablet_manager.cc:1434] T 0972853b828c41c882c68f4326c1ea63 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.033s user 0.005s sys 0.000s
I20260925 13:57:09.227984 22210 tablet_bootstrap.cc:492] T 4edff719362a4c3287bd7775c020d27b P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:09.230684 22212 tablet_bootstrap.cc:654] T ffeba794dc43421fab68478796f67e92 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:09.238996 22210 tablet_bootstrap.cc:654] T 4edff719362a4c3287bd7775c020d27b P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
W20260925 13:57:09.251796 22013 tablet.cc:2433] T d267def3f0864435a0d2c40fd7052eab P 145e520e0a5a445d943a53cf8c27efee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 13:57:09.253259 22211 tablet_bootstrap.cc:492] T 4edff719362a4c3287bd7775c020d27b P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:09.253679 22211 ts_tablet_manager.cc:1403] T 4edff719362a4c3287bd7775c020d27b P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.076s user 0.000s sys 0.016s
I20260925 13:57:09.255168 22214 ts_tablet_manager.cc:1929] T 4edff719362a4c3287bd7775c020d27b P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:09.255707 22214 log.cc:1199] T 4edff719362a4c3287bd7775c020d27b P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/4edff719362a4c3287bd7775c020d27b
I20260925 13:57:09.256669 22214 ts_tablet_manager.cc:1950] T 4edff719362a4c3287bd7775c020d27b P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:09.256559 22211 raft_consensus.cc:359] T 4edff719362a4c3287bd7775c020d27b P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.257236 22211 raft_consensus.cc:385] T 4edff719362a4c3287bd7775c020d27b P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:09.257501 22211 raft_consensus.cc:740] T 4edff719362a4c3287bd7775c020d27b P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:09.258234 22211 consensus_queue.cc:260] T 4edff719362a4c3287bd7775c020d27b P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.264000 22211 ts_tablet_manager.cc:1434] T 4edff719362a4c3287bd7775c020d27b P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.010s user 0.007s sys 0.001s
I20260925 13:57:09.264276 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet 4edff719362a4c3287bd7775c020d27b (table default.table_366914d3ae844485b147c75792717126 [id=74ee67384ecf4220b7d08fdf0637aefb]) successfully deleted
I20260925 13:57:09.268801 22212 tablet_bootstrap.cc:492] T ffeba794dc43421fab68478796f67e92 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:09.269306 22212 ts_tablet_manager.cc:1403] T ffeba794dc43421fab68478796f67e92 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.057s user 0.008s sys 0.008s
I20260925 13:57:09.270102 22211 tablet_bootstrap.cc:492] T 0972853b828c41c882c68f4326c1ea63 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:09.273093 22212 raft_consensus.cc:359] T ffeba794dc43421fab68478796f67e92 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.273738 22212 raft_consensus.cc:385] T ffeba794dc43421fab68478796f67e92 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:09.274013 22212 raft_consensus.cc:740] T ffeba794dc43421fab68478796f67e92 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:09.280632 22211 tablet_bootstrap.cc:654] T 0972853b828c41c882c68f4326c1ea63 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:09.282248 21938 tablet_service.cc:1558] Processing DeleteTablet for tablet 4edff719362a4c3287bd7775c020d27b with delete_type TABLET_DATA_DELETED (replaced by tablet ccefd36f4e2a48f9b3d66cb147a022b5) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:09.283142 22215 tablet_replica.cc:333] T 4edff719362a4c3287bd7775c020d27b P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:09.283087 22212 consensus_queue.cc:260] T ffeba794dc43421fab68478796f67e92 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.283972 22215 raft_consensus.cc:2243] T 4edff719362a4c3287bd7775c020d27b P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:09.284528 22215 raft_consensus.cc:2272] T 4edff719362a4c3287bd7775c020d27b P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:09.287328 22215 ts_tablet_manager.cc:1916] T 4edff719362a4c3287bd7775c020d27b P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:09.296988 22210 tablet_bootstrap.cc:492] T 4edff719362a4c3287bd7775c020d27b P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:09.297415 22210 ts_tablet_manager.cc:1403] T 4edff719362a4c3287bd7775c020d27b P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.070s user 0.009s sys 0.010s
I20260925 13:57:09.298870 21580 master_service.cc:960] ReplaceTablet: received request to replace tablet ffeba794dc43421fab68478796f67e92 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:09.299976 22210 raft_consensus.cc:359] T 4edff719362a4c3287bd7775c020d27b P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.300611 22210 raft_consensus.cc:385] T 4edff719362a4c3287bd7775c020d27b P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:09.300882 22210 raft_consensus.cc:740] T 4edff719362a4c3287bd7775c020d27b P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:09.301570 22210 consensus_queue.cc:260] T 4edff719362a4c3287bd7775c020d27b P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.309396 22215 ts_tablet_manager.cc:1929] T 4edff719362a4c3287bd7775c020d27b P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:09.309774 22215 log.cc:1199] T 4edff719362a4c3287bd7775c020d27b P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/4edff719362a4c3287bd7775c020d27b
I20260925 13:57:09.318524 22211 tablet_bootstrap.cc:492] T 0972853b828c41c882c68f4326c1ea63 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:09.318950 22211 ts_tablet_manager.cc:1403] T 0972853b828c41c882c68f4326c1ea63 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.049s user 0.007s sys 0.004s
I20260925 13:57:09.319620 22210 ts_tablet_manager.cc:1434] T 4edff719362a4c3287bd7775c020d27b P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.022s user 0.006s sys 0.001s
I20260925 13:57:09.320483 22210 tablet_bootstrap.cc:492] T ffeba794dc43421fab68478796f67e92 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:09.326243 22210 tablet_bootstrap.cc:654] T ffeba794dc43421fab68478796f67e92 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:09.328018 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet ffeba794dc43421fab68478796f67e92 with delete_type TABLET_DATA_DELETED (replaced by tablet aa509c1466e24caba7ca5caf1395bd86) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:09.328502 22211 raft_consensus.cc:359] T 0972853b828c41c882c68f4326c1ea63 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.329543 22212 ts_tablet_manager.cc:1434] T ffeba794dc43421fab68478796f67e92 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.060s user 0.005s sys 0.000s
I20260925 13:57:09.336037 22211 raft_consensus.cc:385] T 0972853b828c41c882c68f4326c1ea63 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:09.335707 22212 tablet_bootstrap.cc:492] T 0972853b828c41c882c68f4326c1ea63 P ee02f77594864349982492862d668a36: Bootstrap starting.
W20260925 13:57:09.338382 21575 catalog_manager.cc:5043] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): delete failed for tablet ffeba794dc43421fab68478796f67e92 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ffeba794dc43421fab68478796f67e92 already in progress: creating tablet
I20260925 13:57:09.338649 22211 raft_consensus.cc:740] T 0972853b828c41c882c68f4326c1ea63 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:09.339403 22211 consensus_queue.cc:260] T 0972853b828c41c882c68f4326c1ea63 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.342037 22211 ts_tablet_manager.cc:1434] T 0972853b828c41c882c68f4326c1ea63 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.023s user 0.007s sys 0.000s
I20260925 13:57:09.343019 21777 tablet_service.cc:1558] Processing DeleteTablet for tablet 4edff719362a4c3287bd7775c020d27b with delete_type TABLET_DATA_DELETED (replaced by tablet ccefd36f4e2a48f9b3d66cb147a022b5) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:09.344056 22223 raft_consensus.cc:493] T d267def3f0864435a0d2c40fd7052eab P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:09.344583 22223 raft_consensus.cc:515] T d267def3f0864435a0d2c40fd7052eab P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.345373 22213 tablet_replica.cc:333] T 4edff719362a4c3287bd7775c020d27b P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:09.346200 22213 raft_consensus.cc:2243] T 4edff719362a4c3287bd7775c020d27b P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:09.346343 21580 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet ffeba794dc43421fab68478796f67e92
I20260925 13:57:09.346705 22213 raft_consensus.cc:2272] T 4edff719362a4c3287bd7775c020d27b P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:09.348680 21777 tablet_service.cc:1558] Processing DeleteTablet for tablet ffeba794dc43421fab68478796f67e92 with delete_type TABLET_DATA_DELETED (replaced by tablet aa509c1466e24caba7ca5caf1395bd86) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:09.349399 22223 leader_election.cc:290] T d267def3f0864435a0d2c40fd7052eab P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:09.351642 21938 tablet_service.cc:1558] Processing DeleteTablet for tablet ffeba794dc43421fab68478796f67e92 with delete_type TABLET_DATA_DELETED (replaced by tablet aa509c1466e24caba7ca5caf1395bd86) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:09.352289 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet ffeba794dc43421fab68478796f67e92 with delete_type TABLET_DATA_DELETED (replaced by tablet aa509c1466e24caba7ca5caf1395bd86) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:09.352979 22214 tablet_replica.cc:333] T ffeba794dc43421fab68478796f67e92 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:09.353606 22214 raft_consensus.cc:2243] T ffeba794dc43421fab68478796f67e92 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:09.354084 22214 raft_consensus.cc:2272] T ffeba794dc43421fab68478796f67e92 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:09.354735 22213 ts_tablet_manager.cc:1916] T 4edff719362a4c3287bd7775c020d27b P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:09.356637 22211 tablet_bootstrap.cc:492] T ffeba794dc43421fab68478796f67e92 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:09.357689 22214 ts_tablet_manager.cc:1916] T ffeba794dc43421fab68478796f67e92 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:09.310662 22215 ts_tablet_manager.cc:1950] T 4edff719362a4c3287bd7775c020d27b P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:09.364007 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet 4edff719362a4c3287bd7775c020d27b (table default.table_366914d3ae844485b147c75792717126 [id=74ee67384ecf4220b7d08fdf0637aefb]) successfully deleted
I20260925 13:57:09.367482 21580 catalog_manager.cc:6934] ReplaceTablet: tablet ffeba794dc43421fab68478796f67e92 deleted and replaced by tablet aa509c1466e24caba7ca5caf1395bd86
I20260925 13:57:09.368553 22211 tablet_bootstrap.cc:654] T ffeba794dc43421fab68478796f67e92 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
W20260925 13:57:09.371168 21575 catalog_manager.cc:5043] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): delete failed for tablet ffeba794dc43421fab68478796f67e92 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ffeba794dc43421fab68478796f67e92 already in progress: creating tablet
W20260925 13:57:09.384446 21852 tablet.cc:2433] T 0972853b828c41c882c68f4326c1ea63 P 5e2731975dbf433aade4a859df9699e2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 13:57:09.392117 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d267def3f0864435a0d2c40fd7052eab" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:09.393105 21579 master_service.cc:960] ReplaceTablet: received request to replace tablet ffeba794dc43421fab68478796f67e92 from {username='slave'} at 127.0.0.1:35994
W20260925 13:57:09.393903 21732 leader_election.cc:343] T d267def3f0864435a0d2c40fd7052eab P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:57:09.398960 22212 tablet_bootstrap.cc:654] T 0972853b828c41c882c68f4326c1ea63 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:09.425875 21777 tablet_service.cc:1511] Processing CreateTablet for tablet aa509c1466e24caba7ca5caf1395bd86 (DEFAULT_TABLE table=default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:09.426929 21777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa509c1466e24caba7ca5caf1395bd86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:09.434116 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d267def3f0864435a0d2c40fd7052eab" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:09.434984 21960 raft_consensus.cc:2468] T d267def3f0864435a0d2c40fd7052eab P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:09.436414 21732 leader_election.cc:304] T d267def3f0864435a0d2c40fd7052eab P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters: ee02f77594864349982492862d668a36
I20260925 13:57:09.436893 22211 tablet_bootstrap.cc:492] T ffeba794dc43421fab68478796f67e92 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:09.437180 22210 tablet_bootstrap.cc:492] T ffeba794dc43421fab68478796f67e92 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:09.437289 22211 ts_tablet_manager.cc:1403] T ffeba794dc43421fab68478796f67e92 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.081s user 0.011s sys 0.005s
I20260925 13:57:09.437338 22223 raft_consensus.cc:2804] T d267def3f0864435a0d2c40fd7052eab P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:09.437589 22210 ts_tablet_manager.cc:1403] T ffeba794dc43421fab68478796f67e92 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.117s user 0.016s sys 0.005s
I20260925 13:57:09.437709 22223 raft_consensus.cc:493] T d267def3f0864435a0d2c40fd7052eab P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:09.438037 22223 raft_consensus.cc:3060] T d267def3f0864435a0d2c40fd7052eab P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:09.440333 22210 raft_consensus.cc:359] T ffeba794dc43421fab68478796f67e92 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.440989 22210 raft_consensus.cc:385] T ffeba794dc43421fab68478796f67e92 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:09.441304 22210 raft_consensus.cc:740] T ffeba794dc43421fab68478796f67e92 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:09.444469 22223 raft_consensus.cc:515] T d267def3f0864435a0d2c40fd7052eab P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.446390 22223 leader_election.cc:290] T d267def3f0864435a0d2c40fd7052eab P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Requested vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:09.449605 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d267def3f0864435a0d2c40fd7052eab" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:09.449692 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d267def3f0864435a0d2c40fd7052eab" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
I20260925 13:57:09.450304 21960 raft_consensus.cc:3060] T d267def3f0864435a0d2c40fd7052eab P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
W20260925 13:57:09.450923 21732 leader_election.cc:343] T d267def3f0864435a0d2c40fd7052eab P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:57:09.447041 22210 consensus_queue.cc:260] T ffeba794dc43421fab68478796f67e92 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.445415 22213 ts_tablet_manager.cc:1929] T 4edff719362a4c3287bd7775c020d27b P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:09.446897 22211 raft_consensus.cc:359] T ffeba794dc43421fab68478796f67e92 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.455688 22213 log.cc:1199] T 4edff719362a4c3287bd7775c020d27b P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/4edff719362a4c3287bd7775c020d27b
I20260925 13:57:09.456055 22211 raft_consensus.cc:385] T ffeba794dc43421fab68478796f67e92 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:09.452950 22210 ts_tablet_manager.cc:1434] T ffeba794dc43421fab68478796f67e92 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.015s user 0.003s sys 0.002s
I20260925 13:57:09.458209 22213 ts_tablet_manager.cc:1950] T 4edff719362a4c3287bd7775c020d27b P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:09.458552 22211 raft_consensus.cc:740] T ffeba794dc43421fab68478796f67e92 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:09.459736 22211 consensus_queue.cc:260] T ffeba794dc43421fab68478796f67e92 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.458767 22210 tablet_bootstrap.cc:492] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:09.468762 21960 raft_consensus.cc:2468] T d267def3f0864435a0d2c40fd7052eab P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:09.469967 21732 leader_election.cc:304] T d267def3f0864435a0d2c40fd7052eab P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters: ee02f77594864349982492862d668a36
I20260925 13:57:09.471009 22223 raft_consensus.cc:2804] T d267def3f0864435a0d2c40fd7052eab P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:09.471328 22210 tablet_bootstrap.cc:654] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:09.480918 22211 ts_tablet_manager.cc:1434] T ffeba794dc43421fab68478796f67e92 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.043s user 0.008s sys 0.000s
I20260925 13:57:09.483247 22223 raft_consensus.cc:697] T d267def3f0864435a0d2c40fd7052eab P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Becoming Leader. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Running, Role: LEADER
I20260925 13:57:09.488207 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet 4edff719362a4c3287bd7775c020d27b (table default.table_366914d3ae844485b147c75792717126 [id=74ee67384ecf4220b7d08fdf0637aefb]) successfully deleted
I20260925 13:57:09.487987 22223 consensus_queue.cc:237] T d267def3f0864435a0d2c40fd7052eab P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.493043 22083 tablet_service.cc:1511] Processing CreateTablet for tablet aa509c1466e24caba7ca5caf1395bd86 (DEFAULT_TABLE table=default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:09.493499 21938 tablet_service.cc:1511] Processing CreateTablet for tablet aa509c1466e24caba7ca5caf1395bd86 (DEFAULT_TABLE table=default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:09.494160 22083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa509c1466e24caba7ca5caf1395bd86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:09.494490 21938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa509c1466e24caba7ca5caf1395bd86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:09.507469 22213 tablet_replica.cc:333] T ffeba794dc43421fab68478796f67e92 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:09.510784 22211 tablet_bootstrap.cc:492] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:09.513275 22213 raft_consensus.cc:2243] T ffeba794dc43421fab68478796f67e92 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:09.517838 22213 raft_consensus.cc:2272] T ffeba794dc43421fab68478796f67e92 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:09.523659 22213 ts_tablet_manager.cc:1916] T ffeba794dc43421fab68478796f67e92 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:09.532255 21938 tablet_service.cc:1558] Processing DeleteTablet for tablet ffeba794dc43421fab68478796f67e92 with delete_type TABLET_DATA_DELETED (replaced by tablet aa509c1466e24caba7ca5caf1395bd86) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:09.532994 22215 tablet_replica.cc:333] T ffeba794dc43421fab68478796f67e92 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:09.533694 22215 raft_consensus.cc:2243] T ffeba794dc43421fab68478796f67e92 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:09.534123 22215 raft_consensus.cc:2272] T ffeba794dc43421fab68478796f67e92 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:09.537004 22215 ts_tablet_manager.cc:1916] T ffeba794dc43421fab68478796f67e92 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 13:57:09.551186 21580 catalog_manager.cc:5671] Ignoring report from unknown tablet d267def3f0864435a0d2c40fd7052eab
I20260925 13:57:09.562970 22211 tablet_bootstrap.cc:654] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:09.571316 22214 ts_tablet_manager.cc:1929] T ffeba794dc43421fab68478796f67e92 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:09.571763 22214 log.cc:1199] T ffeba794dc43421fab68478796f67e92 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/ffeba794dc43421fab68478796f67e92
I20260925 13:57:09.572440 22214 ts_tablet_manager.cc:1950] T ffeba794dc43421fab68478796f67e92 P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:09.575277 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet ffeba794dc43421fab68478796f67e92 (table default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]) successfully deleted
I20260925 13:57:09.580740 22212 tablet_bootstrap.cc:492] T 0972853b828c41c882c68f4326c1ea63 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:09.584594 22212 ts_tablet_manager.cc:1403] T 0972853b828c41c882c68f4326c1ea63 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.249s user 0.016s sys 0.006s
I20260925 13:57:09.587931 22215 ts_tablet_manager.cc:1929] T ffeba794dc43421fab68478796f67e92 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:09.594081 21580 master_service.cc:960] ReplaceTablet: received request to replace tablet ccefd36f4e2a48f9b3d66cb147a022b5 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:09.614104 21579 master_service.cc:960] ReplaceTablet: received request to replace tablet 0972853b828c41c882c68f4326c1ea63 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:09.617249 22210 tablet_bootstrap.cc:492] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:09.617542 21580 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet ccefd36f4e2a48f9b3d66cb147a022b5
I20260925 13:57:09.617671 22210 ts_tablet_manager.cc:1403] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.159s user 0.010s sys 0.020s
I20260925 13:57:09.619257 21776 tablet_service.cc:1558] Processing DeleteTablet for tablet ccefd36f4e2a48f9b3d66cb147a022b5 with delete_type TABLET_DATA_DELETED (replaced by tablet cd30e39a98fd4d7fbe793bdbc02eea13) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:09.620371 22210 raft_consensus.cc:359] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.621061 22210 raft_consensus.cc:385] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:09.621193 21580 catalog_manager.cc:6934] ReplaceTablet: tablet ccefd36f4e2a48f9b3d66cb147a022b5 deleted and replaced by tablet cd30e39a98fd4d7fbe793bdbc02eea13
I20260925 13:57:09.621387 22210 raft_consensus.cc:740] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:09.621641 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet ccefd36f4e2a48f9b3d66cb147a022b5 with delete_type TABLET_DATA_DELETED (replaced by tablet cd30e39a98fd4d7fbe793bdbc02eea13) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:09.624544 22210 consensus_queue.cc:260] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:09.626437 21575 catalog_manager.cc:5043] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): delete failed for tablet ccefd36f4e2a48f9b3d66cb147a022b5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ccefd36f4e2a48f9b3d66cb147a022b5 already in progress: creating tablet
I20260925 13:57:09.619899 21938 tablet_service.cc:1558] Processing DeleteTablet for tablet ccefd36f4e2a48f9b3d66cb147a022b5 with delete_type TABLET_DATA_DELETED (replaced by tablet cd30e39a98fd4d7fbe793bdbc02eea13) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:09.628379 22215 log.cc:1199] T ffeba794dc43421fab68478796f67e92 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/ffeba794dc43421fab68478796f67e92
I20260925 13:57:09.629184 22215 ts_tablet_manager.cc:1950] T ffeba794dc43421fab68478796f67e92 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:09.631182 22224 raft_consensus.cc:493] T 0972853b828c41c882c68f4326c1ea63 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:09.634615 22224 raft_consensus.cc:515] T 0972853b828c41c882c68f4326c1ea63 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.637462 22210 ts_tablet_manager.cc:1434] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.019s user 0.006s sys 0.002s
I20260925 13:57:09.640095 22224 leader_election.cc:290] T 0972853b828c41c882c68f4326c1ea63 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:09.642637 22210 tablet_bootstrap.cc:492] T ccefd36f4e2a48f9b3d66cb147a022b5 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:09.657609 22210 tablet_bootstrap.cc:654] T ccefd36f4e2a48f9b3d66cb147a022b5 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:09.660629 22212 raft_consensus.cc:359] T 0972853b828c41c882c68f4326c1ea63 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.661422 22212 raft_consensus.cc:385] T 0972853b828c41c882c68f4326c1ea63 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:09.661715 22212 raft_consensus.cc:740] T 0972853b828c41c882c68f4326c1ea63 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:09.662895 21579 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet 0972853b828c41c882c68f4326c1ea63
I20260925 13:57:09.665390 21777 tablet_service.cc:1558] Processing DeleteTablet for tablet 0972853b828c41c882c68f4326c1ea63 with delete_type TABLET_DATA_DELETED (replaced by tablet d4774462e5a8486fba564154d1861e6c) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:09.667876 21938 tablet_service.cc:1558] Processing DeleteTablet for tablet 0972853b828c41c882c68f4326c1ea63 with delete_type TABLET_DATA_DELETED (replaced by tablet d4774462e5a8486fba564154d1861e6c) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:09.670974 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet 0972853b828c41c882c68f4326c1ea63 with delete_type TABLET_DATA_DELETED (replaced by tablet d4774462e5a8486fba564154d1861e6c) from {username='slave'} at 127.0.0.1:50052
W20260925 13:57:09.676517 21575 catalog_manager.cc:5043] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): delete failed for tablet 0972853b828c41c882c68f4326c1ea63 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0972853b828c41c882c68f4326c1ea63 already in progress: creating tablet
I20260925 13:57:09.677470 22210 tablet_bootstrap.cc:492] T ccefd36f4e2a48f9b3d66cb147a022b5 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:09.677927 22210 ts_tablet_manager.cc:1403] T ccefd36f4e2a48f9b3d66cb147a022b5 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.035s user 0.005s sys 0.004s
I20260925 13:57:09.678524 21579 catalog_manager.cc:6934] ReplaceTablet: tablet 0972853b828c41c882c68f4326c1ea63 deleted and replaced by tablet d4774462e5a8486fba564154d1861e6c
I20260925 13:57:09.680688 22210 raft_consensus.cc:359] T ccefd36f4e2a48f9b3d66cb147a022b5 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.681362 22210 raft_consensus.cc:385] T ccefd36f4e2a48f9b3d66cb147a022b5 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:09.681650 22210 raft_consensus.cc:740] T ccefd36f4e2a48f9b3d66cb147a022b5 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:09.682271 22210 consensus_queue.cc:260] T ccefd36f4e2a48f9b3d66cb147a022b5 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.685837 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet ffeba794dc43421fab68478796f67e92 (table default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]) successfully deleted
I20260925 13:57:09.687254 22210 ts_tablet_manager.cc:1434] T ccefd36f4e2a48f9b3d66cb147a022b5 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.009s user 0.006s sys 0.001s
I20260925 13:57:09.691009 22210 tablet_bootstrap.cc:492] T aa509c1466e24caba7ca5caf1395bd86 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:09.689792 22212 consensus_queue.cc:260] T 0972853b828c41c882c68f4326c1ea63 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:09.700359 21575 catalog_manager.cc:5043] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): delete failed for tablet ccefd36f4e2a48f9b3d66cb147a022b5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ccefd36f4e2a48f9b3d66cb147a022b5 already in progress: creating tablet
I20260925 13:57:09.704872 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet 0972853b828c41c882c68f4326c1ea63 with delete_type TABLET_DATA_DELETED (replaced by tablet d4774462e5a8486fba564154d1861e6c) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:09.706454 22211 tablet_bootstrap.cc:492] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
W20260925 13:57:09.706588 21575 catalog_manager.cc:5043] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): delete failed for tablet 0972853b828c41c882c68f4326c1ea63 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0972853b828c41c882c68f4326c1ea63 already in progress: creating tablet
I20260925 13:57:09.707023 22211 ts_tablet_manager.cc:1403] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.196s user 0.014s sys 0.000s
I20260925 13:57:09.709805 22212 ts_tablet_manager.cc:1434] T 0972853b828c41c882c68f4326c1ea63 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.090s user 0.004s sys 0.002s
I20260925 13:57:09.710486 22215 tablet_replica.cc:333] T 0972853b828c41c882c68f4326c1ea63 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:09.715790 22215 raft_consensus.cc:2243] T 0972853b828c41c882c68f4326c1ea63 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:09.716794 22215 raft_consensus.cc:2272] T 0972853b828c41c882c68f4326c1ea63 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:09.720949 22210 tablet_bootstrap.cc:654] T aa509c1466e24caba7ca5caf1395bd86 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:09.721778 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet 0972853b828c41c882c68f4326c1ea63 with delete_type TABLET_DATA_DELETED (replaced by tablet d4774462e5a8486fba564154d1861e6c) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:09.722648 22214 tablet_replica.cc:333] T 0972853b828c41c882c68f4326c1ea63 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:09.723300 22214 raft_consensus.cc:2243] T 0972853b828c41c882c68f4326c1ea63 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:09.723811 22214 raft_consensus.cc:2272] T 0972853b828c41c882c68f4326c1ea63 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:09.725368 22214 ts_tablet_manager.cc:1916] T 0972853b828c41c882c68f4326c1ea63 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:09.735246 22210 tablet_bootstrap.cc:492] T aa509c1466e24caba7ca5caf1395bd86 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:09.735661 22210 ts_tablet_manager.cc:1403] T aa509c1466e24caba7ca5caf1395bd86 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.045s user 0.011s sys 0.000s
I20260925 13:57:09.738199 22210 raft_consensus.cc:359] T aa509c1466e24caba7ca5caf1395bd86 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.738829 22210 raft_consensus.cc:385] T aa509c1466e24caba7ca5caf1395bd86 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:09.739100 22210 raft_consensus.cc:740] T aa509c1466e24caba7ca5caf1395bd86 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:09.739711 22210 consensus_queue.cc:260] T aa509c1466e24caba7ca5caf1395bd86 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.742448 22210 ts_tablet_manager.cc:1434] T aa509c1466e24caba7ca5caf1395bd86 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.007s user 0.004s sys 0.001s
I20260925 13:57:09.709880 22211 raft_consensus.cc:359] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.747280 22212 tablet_bootstrap.cc:492] T d267def3f0864435a0d2c40fd7052eab P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:09.748051 22211 raft_consensus.cc:385] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:09.752143 22211 raft_consensus.cc:740] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:09.757306 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0972853b828c41c882c68f4326c1ea63" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
W20260925 13:57:09.759724 21896 leader_election.cc:343] T 0972853b828c41c882c68f4326c1ea63 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Illegal state: Tablet not RUNNING: STOPPED
I20260925 13:57:09.761020 22215 ts_tablet_manager.cc:1916] T 0972853b828c41c882c68f4326c1ea63 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:09.779094 22212 tablet_bootstrap.cc:654] T d267def3f0864435a0d2c40fd7052eab P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:09.780676 22211 consensus_queue.cc:260] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.786482 21938 tablet_service.cc:1511] Processing CreateTablet for tablet d4774462e5a8486fba564154d1861e6c (DEFAULT_TABLE table=default.table_492c567ddda14e77851270f34471730e [id=754c40502e614c5bb18230c852aa0016]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:09.790975 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0972853b828c41c882c68f4326c1ea63" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2" is_pre_election: true
I20260925 13:57:09.792716 21797 raft_consensus.cc:2468] T 0972853b828c41c882c68f4326c1ea63 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 0.
I20260925 13:57:09.796423 21893 leader_election.cc:304] T 0972853b828c41c882c68f4326c1ea63 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters: ee02f77594864349982492862d668a36
I20260925 13:57:09.796563 21777 tablet_service.cc:1511] Processing CreateTablet for tablet d4774462e5a8486fba564154d1861e6c (DEFAULT_TABLE table=default.table_492c567ddda14e77851270f34471730e [id=754c40502e614c5bb18230c852aa0016]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 13:57:09.797331 21893 raft_consensus.cc:2686] T 0972853b828c41c882c68f4326c1ea63 P 145e520e0a5a445d943a53cf8c27efee: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260925 13:57:09.798030 21777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4774462e5a8486fba564154d1861e6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:09.798650 21938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4774462e5a8486fba564154d1861e6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:09.807694 22211 ts_tablet_manager.cc:1434] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.100s user 0.006s sys 0.001s
I20260925 13:57:09.812935 22213 ts_tablet_manager.cc:1929] T ffeba794dc43421fab68478796f67e92 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:09.813689 22213 log.cc:1199] T ffeba794dc43421fab68478796f67e92 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/ffeba794dc43421fab68478796f67e92
I20260925 13:57:09.833894 21579 master_service.cc:960] ReplaceTablet: received request to replace tablet ccefd36f4e2a48f9b3d66cb147a022b5 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:09.834708 22210 tablet_bootstrap.cc:492] T d4774462e5a8486fba564154d1861e6c P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:09.837109 22211 tablet_bootstrap.cc:492] T ccefd36f4e2a48f9b3d66cb147a022b5 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:09.840168 22083 tablet_service.cc:1511] Processing CreateTablet for tablet d4774462e5a8486fba564154d1861e6c (DEFAULT_TABLE table=default.table_492c567ddda14e77851270f34471730e [id=754c40502e614c5bb18230c852aa0016]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:09.841288 22083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4774462e5a8486fba564154d1861e6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:09.844830 21776 tablet_service.cc:1511] Processing CreateTablet for tablet cd30e39a98fd4d7fbe793bdbc02eea13 (DEFAULT_TABLE table=default.table_366914d3ae844485b147c75792717126 [id=74ee67384ecf4220b7d08fdf0637aefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:09.846043 21776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd30e39a98fd4d7fbe793bdbc02eea13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:09.848289 21937 tablet_service.cc:1511] Processing CreateTablet for tablet cd30e39a98fd4d7fbe793bdbc02eea13 (DEFAULT_TABLE table=default.table_366914d3ae844485b147c75792717126 [id=74ee67384ecf4220b7d08fdf0637aefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:09.849488 21937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd30e39a98fd4d7fbe793bdbc02eea13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:09.852837 22087 tablet_service.cc:1511] Processing CreateTablet for tablet cd30e39a98fd4d7fbe793bdbc02eea13 (DEFAULT_TABLE table=default.table_366914d3ae844485b147c75792717126 [id=74ee67384ecf4220b7d08fdf0637aefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:09.853935 22087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd30e39a98fd4d7fbe793bdbc02eea13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:09.814661 22213 ts_tablet_manager.cc:1950] T ffeba794dc43421fab68478796f67e92 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:09.863579 22211 tablet_bootstrap.cc:654] T ccefd36f4e2a48f9b3d66cb147a022b5 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:09.865134 22212 tablet_bootstrap.cc:492] T d267def3f0864435a0d2c40fd7052eab P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:09.865579 22212 ts_tablet_manager.cc:1403] T d267def3f0864435a0d2c40fd7052eab P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.118s user 0.010s sys 0.000s
I20260925 13:57:09.868358 22212 raft_consensus.cc:359] T d267def3f0864435a0d2c40fd7052eab P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.869005 22212 raft_consensus.cc:385] T d267def3f0864435a0d2c40fd7052eab P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:09.869094 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet ffeba794dc43421fab68478796f67e92 (table default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]) successfully deleted
I20260925 13:57:09.869292 22212 raft_consensus.cc:740] T d267def3f0864435a0d2c40fd7052eab P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:09.872327 22213 tablet_replica.cc:333] T ccefd36f4e2a48f9b3d66cb147a022b5 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:09.873286 22213 raft_consensus.cc:2243] T ccefd36f4e2a48f9b3d66cb147a022b5 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:09.873799 22213 raft_consensus.cc:2272] T ccefd36f4e2a48f9b3d66cb147a022b5 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:09.876044 22214 ts_tablet_manager.cc:1929] T 0972853b828c41c882c68f4326c1ea63 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:09.876437 22214 log.cc:1199] T 0972853b828c41c882c68f4326c1ea63 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/0972853b828c41c882c68f4326c1ea63
I20260925 13:57:09.877116 22214 ts_tablet_manager.cc:1950] T 0972853b828c41c882c68f4326c1ea63 P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:09.877956 22210 tablet_bootstrap.cc:654] T d4774462e5a8486fba564154d1861e6c P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:09.880033 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet 0972853b828c41c882c68f4326c1ea63 (table default.table_492c567ddda14e77851270f34471730e [id=754c40502e614c5bb18230c852aa0016]) successfully deleted
I20260925 13:57:09.883930 22212 consensus_queue.cc:260] T d267def3f0864435a0d2c40fd7052eab P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.885895 22212 ts_tablet_manager.cc:1434] T d267def3f0864435a0d2c40fd7052eab P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.020s user 0.007s sys 0.000s
I20260925 13:57:09.886807 22212 tablet_bootstrap.cc:492] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P ee02f77594864349982492862d668a36: Bootstrap starting.
W20260925 13:57:09.889884 21579 catalog_manager.cc:5671] Ignoring report from unknown tablet d267def3f0864435a0d2c40fd7052eab
I20260925 13:57:09.892383 22215 ts_tablet_manager.cc:1929] T 0972853b828c41c882c68f4326c1ea63 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260925 13:57:09.897377 21579 catalog_manager.cc:5671] Ignoring report from unknown tablet d267def3f0864435a0d2c40fd7052eab
I20260925 13:57:09.892803 22215 log.cc:1199] T 0972853b828c41c882c68f4326c1ea63 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/0972853b828c41c882c68f4326c1ea63
I20260925 13:57:09.913385 21579 master_service.cc:960] ReplaceTablet: received request to replace tablet 0972853b828c41c882c68f4326c1ea63 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:09.921770 22210 tablet_bootstrap.cc:492] T d4774462e5a8486fba564154d1861e6c P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:09.922091 22212 tablet_bootstrap.cc:654] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:09.922199 22210 ts_tablet_manager.cc:1403] T d4774462e5a8486fba564154d1861e6c P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.088s user 0.009s sys 0.003s
I20260925 13:57:09.924691 22210 raft_consensus.cc:359] T d4774462e5a8486fba564154d1861e6c P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.925366 22210 raft_consensus.cc:385] T d4774462e5a8486fba564154d1861e6c P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:09.925637 22210 raft_consensus.cc:740] T d4774462e5a8486fba564154d1861e6c P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:09.927161 22215 ts_tablet_manager.cc:1950] T 0972853b828c41c882c68f4326c1ea63 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:09.929171 22211 tablet_bootstrap.cc:492] T ccefd36f4e2a48f9b3d66cb147a022b5 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:09.929601 22211 ts_tablet_manager.cc:1403] T ccefd36f4e2a48f9b3d66cb147a022b5 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.093s user 0.005s sys 0.006s
I20260925 13:57:09.932761 22211 raft_consensus.cc:359] T ccefd36f4e2a48f9b3d66cb147a022b5 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.921772 22213 ts_tablet_manager.cc:1916] T ccefd36f4e2a48f9b3d66cb147a022b5 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:09.933420 22211 raft_consensus.cc:385] T ccefd36f4e2a48f9b3d66cb147a022b5 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:09.933696 22211 raft_consensus.cc:740] T ccefd36f4e2a48f9b3d66cb147a022b5 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
W20260925 13:57:09.934428 22156 tablet.cc:2433] T d267def3f0864435a0d2c40fd7052eab P ee02f77594864349982492862d668a36: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 13:57:09.934342 22211 consensus_queue.cc:260] T ccefd36f4e2a48f9b3d66cb147a022b5 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.936216 22210 consensus_queue.cc:260] T d4774462e5a8486fba564154d1861e6c P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.952630 22211 ts_tablet_manager.cc:1434] T ccefd36f4e2a48f9b3d66cb147a022b5 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.023s user 0.007s sys 0.001s
I20260925 13:57:09.952564 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet 0972853b828c41c882c68f4326c1ea63 (table default.table_492c567ddda14e77851270f34471730e [id=754c40502e614c5bb18230c852aa0016]) successfully deleted
I20260925 13:57:09.956169 22211 tablet_bootstrap.cc:492] T aa509c1466e24caba7ca5caf1395bd86 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:09.965755 21937 tablet_service.cc:1558] Processing DeleteTablet for tablet ccefd36f4e2a48f9b3d66cb147a022b5 with delete_type TABLET_DATA_DELETED (replaced by tablet cd30e39a98fd4d7fbe793bdbc02eea13) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:09.966421 22210 ts_tablet_manager.cc:1434] T d4774462e5a8486fba564154d1861e6c P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.044s user 0.001s sys 0.005s
I20260925 13:57:09.967442 22210 tablet_bootstrap.cc:492] T cd30e39a98fd4d7fbe793bdbc02eea13 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:09.968559 22211 tablet_bootstrap.cc:654] T aa509c1466e24caba7ca5caf1395bd86 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:09.973241 22215 tablet_replica.cc:333] T ccefd36f4e2a48f9b3d66cb147a022b5 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:09.974254 22215 raft_consensus.cc:2243] T ccefd36f4e2a48f9b3d66cb147a022b5 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:09.974828 22215 raft_consensus.cc:2272] T ccefd36f4e2a48f9b3d66cb147a022b5 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:09.978264 22212 tablet_bootstrap.cc:492] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:09.978721 22212 ts_tablet_manager.cc:1403] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.092s user 0.007s sys 0.003s
I20260925 13:57:09.982156 22212 raft_consensus.cc:359] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:09.982844 22212 raft_consensus.cc:385] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:09.983126 22212 raft_consensus.cc:740] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:09.984107 22215 ts_tablet_manager.cc:1916] T ccefd36f4e2a48f9b3d66cb147a022b5 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:09.983958 22212 consensus_queue.cc:260] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:10.001398 22212 ts_tablet_manager.cc:1434] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.022s user 0.007s sys 0.000s
I20260925 13:57:10.016644 22212 tablet_bootstrap.cc:492] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:10.018893 22211 tablet_bootstrap.cc:492] T aa509c1466e24caba7ca5caf1395bd86 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:10.019291 22211 ts_tablet_manager.cc:1403] T aa509c1466e24caba7ca5caf1395bd86 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.063s user 0.012s sys 0.003s
I20260925 13:57:10.021603 22211 raft_consensus.cc:359] T aa509c1466e24caba7ca5caf1395bd86 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:10.022271 22211 raft_consensus.cc:385] T aa509c1466e24caba7ca5caf1395bd86 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:10.022564 22211 raft_consensus.cc:740] T aa509c1466e24caba7ca5caf1395bd86 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:10.024631 22213 ts_tablet_manager.cc:1929] T ccefd36f4e2a48f9b3d66cb147a022b5 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:10.026186 22211 consensus_queue.cc:260] T aa509c1466e24caba7ca5caf1395bd86 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:10.025696 22223 raft_consensus.cc:493] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:10.035722 22223 raft_consensus.cc:515] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:10.037925 22223 leader_election.cc:290] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:10.030628 22210 tablet_bootstrap.cc:654] T cd30e39a98fd4d7fbe793bdbc02eea13 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:10.039407 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c593c8aa09c4423b6c7d7bbcf94f4ab" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:10.040241 22107 raft_consensus.cc:2468] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:10.040789 22213 log.cc:1199] T ccefd36f4e2a48f9b3d66cb147a022b5 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/ccefd36f4e2a48f9b3d66cb147a022b5
I20260925 13:57:10.041785 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c593c8aa09c4423b6c7d7bbcf94f4ab" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:10.042359 21960 raft_consensus.cc:2468] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:10.042232 21732 leader_election.cc:304] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:10.043733 22223 raft_consensus.cc:2804] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:10.044044 22223 raft_consensus.cc:493] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:10.044319 22223 raft_consensus.cc:3060] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:10.045728 21580 master_service.cc:960] ReplaceTablet: received request to replace tablet ccefd36f4e2a48f9b3d66cb147a022b5 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:10.048420 22211 ts_tablet_manager.cc:1434] T aa509c1466e24caba7ca5caf1395bd86 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.029s user 0.006s sys 0.000s
I20260925 13:57:10.050836 22213 ts_tablet_manager.cc:1950] T ccefd36f4e2a48f9b3d66cb147a022b5 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:10.054690 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet ccefd36f4e2a48f9b3d66cb147a022b5 (table default.table_366914d3ae844485b147c75792717126 [id=74ee67384ecf4220b7d08fdf0637aefb]) successfully deleted
I20260925 13:57:10.057117 22215 ts_tablet_manager.cc:1929] T ccefd36f4e2a48f9b3d66cb147a022b5 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:10.051512 22223 raft_consensus.cc:515] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:10.057497 22215 log.cc:1199] T ccefd36f4e2a48f9b3d66cb147a022b5 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/ccefd36f4e2a48f9b3d66cb147a022b5
I20260925 13:57:10.058156 22215 ts_tablet_manager.cc:1950] T ccefd36f4e2a48f9b3d66cb147a022b5 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:10.060230 22211 tablet_bootstrap.cc:492] T d4774462e5a8486fba564154d1861e6c P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:10.060057 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c593c8aa09c4423b6c7d7bbcf94f4ab" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
I20260925 13:57:10.060591 21960 raft_consensus.cc:3060] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:10.061440 22223 leader_election.cc:290] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Requested vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:10.062058 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c593c8aa09c4423b6c7d7bbcf94f4ab" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:10.062615 22107 raft_consensus.cc:3060] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:10.062151 22213 tablet_replica.cc:333] T 0972853b828c41c882c68f4326c1ea63 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:10.066428 22213 raft_consensus.cc:2243] T 0972853b828c41c882c68f4326c1ea63 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:10.067425 21960 raft_consensus.cc:2468] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:10.068682 21732 leader_election.cc:304] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:10.070588 22107 raft_consensus.cc:2468] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:10.071074 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet ccefd36f4e2a48f9b3d66cb147a022b5 (table default.table_366914d3ae844485b147c75792717126 [id=74ee67384ecf4220b7d08fdf0637aefb]) successfully deleted
I20260925 13:57:10.071224 22223 raft_consensus.cc:2804] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:10.073269 22223 raft_consensus.cc:697] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Becoming Leader. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Running, Role: LEADER
I20260925 13:57:10.074124 22223 consensus_queue.cc:237] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:10.076277 22212 tablet_bootstrap.cc:654] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:10.087131 22211 tablet_bootstrap.cc:654] T d4774462e5a8486fba564154d1861e6c P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:10.075093 22213 raft_consensus.cc:2272] T 0972853b828c41c882c68f4326c1ea63 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:10.090298 22213 ts_tablet_manager.cc:1916] T 0972853b828c41c882c68f4326c1ea63 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:10.092063 22212 tablet_bootstrap.cc:492] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:10.092511 22212 ts_tablet_manager.cc:1403] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.076s user 0.009s sys 0.002s
I20260925 13:57:10.078951 22210 tablet_bootstrap.cc:492] T cd30e39a98fd4d7fbe793bdbc02eea13 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:10.094977 22210 ts_tablet_manager.cc:1403] T cd30e39a98fd4d7fbe793bdbc02eea13 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.128s user 0.004s sys 0.006s
I20260925 13:57:10.097218 22210 raft_consensus.cc:359] T cd30e39a98fd4d7fbe793bdbc02eea13 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:10.097872 22210 raft_consensus.cc:385] T cd30e39a98fd4d7fbe793bdbc02eea13 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:10.098152 22210 raft_consensus.cc:740] T cd30e39a98fd4d7fbe793bdbc02eea13 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:10.096103 22212 raft_consensus.cc:359] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:10.099197 22212 raft_consensus.cc:385] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:10.098851 22210 consensus_queue.cc:260] T cd30e39a98fd4d7fbe793bdbc02eea13 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:10.099500 22212 raft_consensus.cc:740] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:10.100780 22210 ts_tablet_manager.cc:1434] T cd30e39a98fd4d7fbe793bdbc02eea13 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260925 13:57:10.100351 22212 consensus_queue.cc:260] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:10.107920 21579 catalog_manager.cc:5671] Ignoring report from unknown tablet ccefd36f4e2a48f9b3d66cb147a022b5
I20260925 13:57:10.115124 22211 tablet_bootstrap.cc:492] T d4774462e5a8486fba564154d1861e6c P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:10.115576 22211 ts_tablet_manager.cc:1403] T d4774462e5a8486fba564154d1861e6c P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.056s user 0.012s sys 0.000s
I20260925 13:57:10.118441 22211 raft_consensus.cc:359] T d4774462e5a8486fba564154d1861e6c P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:10.119108 22211 raft_consensus.cc:385] T d4774462e5a8486fba564154d1861e6c P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:10.119405 22211 raft_consensus.cc:740] T d4774462e5a8486fba564154d1861e6c P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:10.120733 21585 catalog_manager.cc:3541] renamed table default.table_366914d3ae844485b147c75792717126 [id=74ee67384ecf4220b7d08fdf0637aefb] in HMS: new name default.table_51d65314582841419e34561a4a8d6bbb
I20260925 13:57:10.125312 22212 ts_tablet_manager.cc:1434] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.032s user 0.008s sys 0.000s
W20260925 13:57:10.129256 21579 catalog_manager.cc:5671] Ignoring report from unknown tablet ccefd36f4e2a48f9b3d66cb147a022b5
I20260925 13:57:10.140617 22226 raft_consensus.cc:493] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:10.141127 22226 raft_consensus.cc:515] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:10.151984 21580 master_service.cc:960] ReplaceTablet: received request to replace tablet 0972853b828c41c882c68f4326c1ea63 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:10.152122 22226 leader_election.cc:290] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:10.167932 22213 ts_tablet_manager.cc:1929] T 0972853b828c41c882c68f4326c1ea63 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:10.159235 21579 catalog_manager.cc:5838] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 5e2731975dbf433aade4a859df9699e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e2731975dbf433aade4a859df9699e2 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "5e2731975dbf433aade4a859df9699e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: UNKNOWN } } }
I20260925 13:57:10.179555 22213 log.cc:1199] T 0972853b828c41c882c68f4326c1ea63 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/0972853b828c41c882c68f4326c1ea63
I20260925 13:57:10.180622 22213 ts_tablet_manager.cc:1950] T 0972853b828c41c882c68f4326c1ea63 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:10.185704 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet 0972853b828c41c882c68f4326c1ea63 (table default.table_492c567ddda14e77851270f34471730e [id=754c40502e614c5bb18230c852aa0016]) successfully deleted
I20260925 13:57:10.191092 22212 tablet_bootstrap.cc:492] T aa509c1466e24caba7ca5caf1395bd86 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:10.139887 22211 consensus_queue.cc:260] T d4774462e5a8486fba564154d1861e6c P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:10.207813 22211 ts_tablet_manager.cc:1434] T d4774462e5a8486fba564154d1861e6c P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.092s user 0.006s sys 0.000s
I20260925 13:57:10.232476 22211 tablet_bootstrap.cc:492] T cd30e39a98fd4d7fbe793bdbc02eea13 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
W20260925 13:57:10.246445 21643 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cd30e39a98fd4d7fbe793bdbc02eea13 consensus state has no leader
I20260925 13:57:10.254873 22212 tablet_bootstrap.cc:654] T aa509c1466e24caba7ca5caf1395bd86 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:10.251643 21579 catalog_manager.cc:5938] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881) does not have the latest schema for tablet cd30e39a98fd4d7fbe793bdbc02eea13 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 1 got 0
I20260925 13:57:10.258392 21580 master_service.cc:960] ReplaceTablet: received request to replace tablet d267def3f0864435a0d2c40fd7052eab from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:10.267853 22211 tablet_bootstrap.cc:654] T cd30e39a98fd4d7fbe793bdbc02eea13 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
W20260925 13:57:10.269846 22043 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
W20260925 13:57:10.278437 22040 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
I20260925 13:57:10.278888 22040 leader_election.cc:304] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:10.280120 22226 raft_consensus.cc:2749] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:57:10.278613 22212 tablet_bootstrap.cc:492] T aa509c1466e24caba7ca5caf1395bd86 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:10.282018 22212 ts_tablet_manager.cc:1403] T aa509c1466e24caba7ca5caf1395bd86 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.091s user 0.010s sys 0.003s
I20260925 13:57:10.284536 22212 raft_consensus.cc:359] T aa509c1466e24caba7ca5caf1395bd86 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:10.285245 22212 raft_consensus.cc:385] T aa509c1466e24caba7ca5caf1395bd86 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:10.285534 22212 raft_consensus.cc:740] T aa509c1466e24caba7ca5caf1395bd86 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:10.285702 22211 tablet_bootstrap.cc:492] T cd30e39a98fd4d7fbe793bdbc02eea13 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:10.286146 22211 ts_tablet_manager.cc:1403] T cd30e39a98fd4d7fbe793bdbc02eea13 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.054s user 0.013s sys 0.003s
I20260925 13:57:10.286432 22212 consensus_queue.cc:260] T aa509c1466e24caba7ca5caf1395bd86 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:10.288417 22211 raft_consensus.cc:359] T cd30e39a98fd4d7fbe793bdbc02eea13 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:10.289044 22211 raft_consensus.cc:385] T cd30e39a98fd4d7fbe793bdbc02eea13 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:10.289292 22211 raft_consensus.cc:740] T cd30e39a98fd4d7fbe793bdbc02eea13 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:10.290464 22211 consensus_queue.cc:260] T cd30e39a98fd4d7fbe793bdbc02eea13 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:10.301882 21580 catalog_manager.cc:5938] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291) does not have the latest schema for tablet cd30e39a98fd4d7fbe793bdbc02eea13 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 1 got 0
W20260925 13:57:10.303781 21572 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet cd30e39a98fd4d7fbe793bdbc02eea13 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet cd30e39a98fd4d7fbe793bdbc02eea13 consensus state has no leader
I20260925 13:57:10.309708 22211 ts_tablet_manager.cc:1434] T cd30e39a98fd4d7fbe793bdbc02eea13 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.023s user 0.005s sys 0.000s
I20260925 13:57:10.309999 22212 ts_tablet_manager.cc:1434] T aa509c1466e24caba7ca5caf1395bd86 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.028s user 0.005s sys 0.000s
I20260925 13:57:10.314478 21580 catalog_manager.cc:5938] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291) does not have the latest schema for tablet cd30e39a98fd4d7fbe793bdbc02eea13 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 1 got 0
I20260925 13:57:10.332675 22212 tablet_bootstrap.cc:492] T d4774462e5a8486fba564154d1861e6c P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:10.338465 22212 tablet_bootstrap.cc:654] T d4774462e5a8486fba564154d1861e6c P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:10.342449 22223 consensus_queue.cc:1048] T d267def3f0864435a0d2c40fd7052eab P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 13:57:10.349629 22223 raft_consensus.cc:493] T aa509c1466e24caba7ca5caf1395bd86 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:10.350081 22223 raft_consensus.cc:515] T aa509c1466e24caba7ca5caf1395bd86 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:10.354657 21959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa509c1466e24caba7ca5caf1395bd86" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:10.355340 21959 raft_consensus.cc:2468] T aa509c1466e24caba7ca5caf1395bd86 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:10.356637 21732 leader_election.cc:304] T aa509c1466e24caba7ca5caf1395bd86 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:10.357911 22252 raft_consensus.cc:2804] T aa509c1466e24caba7ca5caf1395bd86 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:10.358240 22252 raft_consensus.cc:493] T aa509c1466e24caba7ca5caf1395bd86 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:10.358556 22252 raft_consensus.cc:3060] T aa509c1466e24caba7ca5caf1395bd86 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:10.361186 21580 master_service.cc:960] ReplaceTablet: received request to replace tablet cd30e39a98fd4d7fbe793bdbc02eea13 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:10.365401 22252 raft_consensus.cc:515] T aa509c1466e24caba7ca5caf1395bd86 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:10.369298 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa509c1466e24caba7ca5caf1395bd86" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:10.369901 22107 raft_consensus.cc:2468] T aa509c1466e24caba7ca5caf1395bd86 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:10.372998 22223 leader_election.cc:290] T aa509c1466e24caba7ca5caf1395bd86 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:10.376627 21959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa509c1466e24caba7ca5caf1395bd86" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
I20260925 13:57:10.377192 21959 raft_consensus.cc:3060] T aa509c1466e24caba7ca5caf1395bd86 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:10.384070 21580 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet cd30e39a98fd4d7fbe793bdbc02eea13
I20260925 13:57:10.384372 21959 raft_consensus.cc:2468] T aa509c1466e24caba7ca5caf1395bd86 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:10.385497 21732 leader_election.cc:304] T aa509c1466e24caba7ca5caf1395bd86 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:10.386355 21776 tablet_service.cc:1558] Processing DeleteTablet for tablet cd30e39a98fd4d7fbe793bdbc02eea13 with delete_type TABLET_DATA_DELETED (replaced by tablet 71ff569d68154b61bcc14937b2e37fe5) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:10.388046 21937 tablet_service.cc:1558] Processing DeleteTablet for tablet cd30e39a98fd4d7fbe793bdbc02eea13 with delete_type TABLET_DATA_DELETED (replaced by tablet 71ff569d68154b61bcc14937b2e37fe5) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:10.388763 22215 tablet_replica.cc:333] T cd30e39a98fd4d7fbe793bdbc02eea13 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:10.389560 22215 raft_consensus.cc:2243] T cd30e39a98fd4d7fbe793bdbc02eea13 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:10.390025 22215 raft_consensus.cc:2272] T cd30e39a98fd4d7fbe793bdbc02eea13 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:10.391681 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet cd30e39a98fd4d7fbe793bdbc02eea13 with delete_type TABLET_DATA_DELETED (replaced by tablet 71ff569d68154b61bcc14937b2e37fe5) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:10.391836 22213 tablet_replica.cc:333] T cd30e39a98fd4d7fbe793bdbc02eea13 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:10.392094 21580 catalog_manager.cc:6934] ReplaceTablet: tablet cd30e39a98fd4d7fbe793bdbc02eea13 deleted and replaced by tablet 71ff569d68154b61bcc14937b2e37fe5
I20260925 13:57:10.392609 22213 raft_consensus.cc:2243] T cd30e39a98fd4d7fbe793bdbc02eea13 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:10.393078 22213 raft_consensus.cc:2272] T cd30e39a98fd4d7fbe793bdbc02eea13 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:10.393839 22215 ts_tablet_manager.cc:1916] T cd30e39a98fd4d7fbe793bdbc02eea13 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 13:57:10.395627 21575 catalog_manager.cc:5043] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): delete failed for tablet cd30e39a98fd4d7fbe793bdbc02eea13 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cd30e39a98fd4d7fbe793bdbc02eea13 already in progress: creating tablet
I20260925 13:57:10.396636 22213 ts_tablet_manager.cc:1916] T cd30e39a98fd4d7fbe793bdbc02eea13 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:10.406502 22223 raft_consensus.cc:2804] T aa509c1466e24caba7ca5caf1395bd86 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:10.414526 21776 tablet_service.cc:1511] Processing CreateTablet for tablet 71ff569d68154b61bcc14937b2e37fe5 (DEFAULT_TABLE table=default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:10.415738 21776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71ff569d68154b61bcc14937b2e37fe5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:10.417075 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa509c1466e24caba7ca5caf1395bd86" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:10.417541 22107 raft_consensus.cc:3060] T aa509c1466e24caba7ca5caf1395bd86 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:10.423828 22107 raft_consensus.cc:2468] T aa509c1466e24caba7ca5caf1395bd86 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:10.425769 22213 ts_tablet_manager.cc:1929] T cd30e39a98fd4d7fbe793bdbc02eea13 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:10.426085 22213 log.cc:1199] T cd30e39a98fd4d7fbe793bdbc02eea13 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/cd30e39a98fd4d7fbe793bdbc02eea13
I20260925 13:57:10.426676 22213 ts_tablet_manager.cc:1950] T cd30e39a98fd4d7fbe793bdbc02eea13 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:10.426774 21937 tablet_service.cc:1511] Processing CreateTablet for tablet 71ff569d68154b61bcc14937b2e37fe5 (DEFAULT_TABLE table=default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:10.429028 22083 tablet_service.cc:1511] Processing CreateTablet for tablet 71ff569d68154b61bcc14937b2e37fe5 (DEFAULT_TABLE table=default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:10.430102 22083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71ff569d68154b61bcc14937b2e37fe5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:10.431887 21937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71ff569d68154b61bcc14937b2e37fe5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:10.437127 22210 tablet_bootstrap.cc:492] T 71ff569d68154b61bcc14937b2e37fe5 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:10.442619 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet cd30e39a98fd4d7fbe793bdbc02eea13 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]) successfully deleted
I20260925 13:57:10.443327 22210 tablet_bootstrap.cc:654] T 71ff569d68154b61bcc14937b2e37fe5 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:10.455161 22252 leader_election.cc:290] T aa509c1466e24caba7ca5caf1395bd86 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Requested vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:10.458787 22210 tablet_bootstrap.cc:492] T 71ff569d68154b61bcc14937b2e37fe5 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:10.461781 22210 ts_tablet_manager.cc:1403] T 71ff569d68154b61bcc14937b2e37fe5 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.025s user 0.004s sys 0.008s
I20260925 13:57:10.466517 22210 raft_consensus.cc:359] T 71ff569d68154b61bcc14937b2e37fe5 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:10.467168 22210 raft_consensus.cc:385] T 71ff569d68154b61bcc14937b2e37fe5 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:10.467676 22210 raft_consensus.cc:740] T 71ff569d68154b61bcc14937b2e37fe5 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:10.468779 22210 consensus_queue.cc:260] T 71ff569d68154b61bcc14937b2e37fe5 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:10.473222 22210 ts_tablet_manager.cc:1434] T 71ff569d68154b61bcc14937b2e37fe5 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20260925 13:57:10.481986 22223 raft_consensus.cc:697] T aa509c1466e24caba7ca5caf1395bd86 P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Becoming Leader. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Running, Role: LEADER
I20260925 13:57:10.483080 22223 consensus_queue.cc:237] T aa509c1466e24caba7ca5caf1395bd86 P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:10.488427 22107 raft_consensus.cc:3060] T d267def3f0864435a0d2c40fd7052eab P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
W20260925 13:57:10.525682 21580 catalog_manager.cc:5671] Ignoring report from unknown tablet d267def3f0864435a0d2c40fd7052eab
I20260925 13:57:10.547518 22211 tablet_bootstrap.cc:492] T 71ff569d68154b61bcc14937b2e37fe5 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:10.555579 22211 tablet_bootstrap.cc:654] T 71ff569d68154b61bcc14937b2e37fe5 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:10.558998 22263 raft_consensus.cc:493] T d4774462e5a8486fba564154d1861e6c P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:10.559592 22263 raft_consensus.cc:515] T d4774462e5a8486fba564154d1861e6c P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:10.562002 22263 leader_election.cc:290] T d4774462e5a8486fba564154d1861e6c P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:10.563899 21959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4774462e5a8486fba564154d1861e6c" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:10.564746 21959 raft_consensus.cc:2468] T d4774462e5a8486fba564154d1861e6c P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:10.567704 22106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4774462e5a8486fba564154d1861e6c" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:10.567643 21732 leader_election.cc:304] T d4774462e5a8486fba564154d1861e6c P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
W20260925 13:57:10.572468 21732 leader_election.cc:343] T d4774462e5a8486fba564154d1861e6c P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:57:10.573303 22263 raft_consensus.cc:2804] T d4774462e5a8486fba564154d1861e6c P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:10.579164 22263 raft_consensus.cc:493] T d4774462e5a8486fba564154d1861e6c P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:10.580359 22263 raft_consensus.cc:3060] T d4774462e5a8486fba564154d1861e6c P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
W20260925 13:57:10.593907 21580 catalog_manager.cc:5671] Ignoring report from unknown tablet d267def3f0864435a0d2c40fd7052eab
I20260925 13:57:10.612402 21588 catalog_manager.cc:3541] renamed table default.table_376a1428b63b4da3ad04f4491ce69d3b [id=9d18288ef656432994079ec83988ec3f] in HMS: new name default.table_01a6dec392d24e9f89578f6f4274c704
W20260925 13:57:10.620879 22221 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:57:10.621780 22221 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:57:10.622610 22221 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260925 13:57:10.634397 21580 catalog_manager.cc:5838] T aa509c1466e24caba7ca5caf1395bd86 P 5e2731975dbf433aade4a859df9699e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e2731975dbf433aade4a859df9699e2 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "5e2731975dbf433aade4a859df9699e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: UNKNOWN } } }
I20260925 13:57:10.644632 22211 tablet_bootstrap.cc:492] T 71ff569d68154b61bcc14937b2e37fe5 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:10.650339 22211 ts_tablet_manager.cc:1403] T 71ff569d68154b61bcc14937b2e37fe5 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.103s user 0.008s sys 0.004s
I20260925 13:57:10.660084 22212 tablet_bootstrap.cc:492] T d4774462e5a8486fba564154d1861e6c P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:10.660605 22212 ts_tablet_manager.cc:1403] T d4774462e5a8486fba564154d1861e6c P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.328s user 0.059s sys 0.079s
I20260925 13:57:10.668998 21579 master_service.cc:960] ReplaceTablet: received request to replace tablet 71ff569d68154b61bcc14937b2e37fe5 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:10.670186 22262 consensus_queue.cc:1048] T d267def3f0864435a0d2c40fd7052eab P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 13:57:10.672508 22211 raft_consensus.cc:359] T 71ff569d68154b61bcc14937b2e37fe5 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:10.673103 22211 raft_consensus.cc:385] T 71ff569d68154b61bcc14937b2e37fe5 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:10.673347 22211 raft_consensus.cc:740] T 71ff569d68154b61bcc14937b2e37fe5 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:10.673996 22211 consensus_queue.cc:260] T 71ff569d68154b61bcc14937b2e37fe5 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:10.675740 22215 ts_tablet_manager.cc:1929] T cd30e39a98fd4d7fbe793bdbc02eea13 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:10.676090 22215 log.cc:1199] T cd30e39a98fd4d7fbe793bdbc02eea13 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/cd30e39a98fd4d7fbe793bdbc02eea13
I20260925 13:57:10.676741 22215 ts_tablet_manager.cc:1950] T cd30e39a98fd4d7fbe793bdbc02eea13 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:10.677585 22106 raft_consensus.cc:1275] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Refusing update from remote peer 5e2731975dbf433aade4a859df9699e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:57:10.663121 22212 raft_consensus.cc:359] T d4774462e5a8486fba564154d1861e6c P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:10.678094 22212 raft_consensus.cc:385] T d4774462e5a8486fba564154d1861e6c P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:10.678355 22212 raft_consensus.cc:740] T d4774462e5a8486fba564154d1861e6c P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:10.680099 21960 raft_consensus.cc:1275] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Refusing update from remote peer 5e2731975dbf433aade4a859df9699e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260925 13:57:10.685307 21583 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_8b5db7d66475449d9772db568c9ded56 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:57:10.679111 22212 consensus_queue.cc:260] T d4774462e5a8486fba564154d1861e6c P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:10.688850 22212 ts_tablet_manager.cc:1434] T d4774462e5a8486fba564154d1861e6c P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.028s user 0.004s sys 0.000s
W20260925 13:57:10.690125 22221 flags.cc:432] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260925 13:57:10.690944 22221 flags.cc:432] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260925 13:57:10.691315 22221 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:57:10.691725 22221 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:57:10.692094 22221 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:57:10.692479 22221 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:57:10.701025 22211 ts_tablet_manager.cc:1434] T 71ff569d68154b61bcc14937b2e37fe5 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.050s user 0.005s sys 0.000s
I20260925 13:57:10.702265 22212 tablet_bootstrap.cc:492] T cd30e39a98fd4d7fbe793bdbc02eea13 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:10.702508 22263 raft_consensus.cc:515] T d4774462e5a8486fba564154d1861e6c P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:10.707783 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet cd30e39a98fd4d7fbe793bdbc02eea13 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]) successfully deleted
I20260925 13:57:10.708603 22212 tablet_bootstrap.cc:654] T cd30e39a98fd4d7fbe793bdbc02eea13 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:10.710196 22263 leader_election.cc:290] T d4774462e5a8486fba564154d1861e6c P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Requested vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:10.710744 22263 consensus_queue.cc:1048] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:10.712184 22263 consensus_queue.cc:1048] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:10.715173 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4774462e5a8486fba564154d1861e6c" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
I20260925 13:57:10.715813 21960 raft_consensus.cc:3060] T d4774462e5a8486fba564154d1861e6c P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:10.723287 22106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4774462e5a8486fba564154d1861e6c" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:10.724469 22106 raft_consensus.cc:3060] T d4774462e5a8486fba564154d1861e6c P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:10.739012 22106 raft_consensus.cc:2468] T d4774462e5a8486fba564154d1861e6c P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:10.746011 21960 raft_consensus.cc:2468] T d4774462e5a8486fba564154d1861e6c P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:10.747871 21732 leader_election.cc:304] T d4774462e5a8486fba564154d1861e6c P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:10.757653 21579 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet 71ff569d68154b61bcc14937b2e37fe5
I20260925 13:57:10.760026 21579 catalog_manager.cc:6934] ReplaceTablet: tablet 71ff569d68154b61bcc14937b2e37fe5 deleted and replaced by tablet 5eb86772ca2e4306a5c164342514b7b0
I20260925 13:57:10.773579 21937 tablet_service.cc:1558] Processing DeleteTablet for tablet 71ff569d68154b61bcc14937b2e37fe5 with delete_type TABLET_DATA_DELETED (replaced by tablet 5eb86772ca2e4306a5c164342514b7b0) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:10.774314 22215 tablet_replica.cc:333] T 71ff569d68154b61bcc14937b2e37fe5 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:10.775029 22215 raft_consensus.cc:2243] T 71ff569d68154b61bcc14937b2e37fe5 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:10.775519 22215 raft_consensus.cc:2272] T 71ff569d68154b61bcc14937b2e37fe5 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:10.780567 22215 ts_tablet_manager.cc:1916] T 71ff569d68154b61bcc14937b2e37fe5 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:10.787552 21776 tablet_service.cc:1558] Processing DeleteTablet for tablet 71ff569d68154b61bcc14937b2e37fe5 with delete_type TABLET_DATA_DELETED (replaced by tablet 5eb86772ca2e4306a5c164342514b7b0) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:10.788416 22213 tablet_replica.cc:333] T 71ff569d68154b61bcc14937b2e37fe5 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:10.789628 22213 raft_consensus.cc:2243] T 71ff569d68154b61bcc14937b2e37fe5 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:10.790175 22213 raft_consensus.cc:2272] T 71ff569d68154b61bcc14937b2e37fe5 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:10.793314 22213 ts_tablet_manager.cc:1916] T 71ff569d68154b61bcc14937b2e37fe5 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:10.801129 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet 71ff569d68154b61bcc14937b2e37fe5 with delete_type TABLET_DATA_DELETED (replaced by tablet 5eb86772ca2e4306a5c164342514b7b0) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:10.802425 22221 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.182.212:40529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:39557
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.18.182.254:43163,127.18.182.253:46177,127.18.182.252:39283
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.182.253:46177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.182.253
--webserver_port=33753
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 102683f65af844d269565345d7d8c6ebc55c92c5
build type FASTDEBUG
built by None at 25 Sep 2026 13:43:25 UTC on e7f111948823
build id 13994
TSAN enabled
W20260925 13:57:10.803366 21575 catalog_manager.cc:5043] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): delete failed for tablet 71ff569d68154b61bcc14937b2e37fe5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 71ff569d68154b61bcc14937b2e37fe5 already in progress: creating tablet
I20260925 13:57:10.804430 22221 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:57:10.806764 22221 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260925 13:57:10.819927 21579 catalog_manager.cc:5671] Ignoring report from unknown tablet d267def3f0864435a0d2c40fd7052eab
W20260925 13:57:10.830016 21584 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_b17685eaeada404790103bd89711435e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 13:57:10.836670 21581 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_bd714c618f7a4f16a69a69d5932b2014 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 13:57:10.842595 21582 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_8008d16a92cf4c0dafb258a1122bffb5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:57:10.849972 21585 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:35994 (request call id 5) took 2177 ms. Trace:
I20260925 13:57:10.850509 21585 rpcz_store.cc:276] 0925 13:57:08.672601 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:08.672686 (+ 85us) service_pool.cc:224] Handling call
0925 13:57:08.716843 (+ 44157us) spinlock_profiling.cc:243] Waited 42.7 ms on lock 0x7b54000203d4. stack: 00007fd96d084323 00007fd96d084220 00007fd96cc28574 0000000000516964 00000000005168f9 00007fd9741825c0 00007fd9742216fa 00007fd975f92d8c 00007fd976092be6 00007fd973c46634 00007fd973c465c2 00007fd973c464f1 00007fd973c4646c 00007fd973c45722 00007fd971ee709c 00007fd971ee64d6
0925 13:57:08.826890 (+110047us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 13:57:10.846710 (+2019820us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 13:57:10.848709 (+ 1999us) catalog_manager.cc:3764] Apply alter schema
0925 13:57:10.849865 (+ 1156us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":91,"Hive Metastore.run_cpu_time_us":2056,"Hive Metastore.run_wall_time_us":1292380,"spinlock_wait_cycles":98210560}
I20260925 13:57:10.860997 21579 master_service.cc:960] ReplaceTablet: received request to replace tablet 71ff569d68154b61bcc14937b2e37fe5 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:10.879671 22262 raft_consensus.cc:2804] T d4774462e5a8486fba564154d1861e6c P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:10.880370 22262 raft_consensus.cc:697] T d4774462e5a8486fba564154d1861e6c P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Becoming Leader. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Running, Role: LEADER
I20260925 13:57:10.881238 22262 consensus_queue.cc:237] T d4774462e5a8486fba564154d1861e6c P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:10.885629 21732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.195:40873, user_credentials={real_user=slave}} blocked reactor thread for 137820us
I20260925 13:57:10.890229 22267 mvcc.cc:204] Tried to move back new op lower bound from 7333251607242973184 to 7333251604861972480. Current Snapshot: MvccSnapshot[applied={T|T < 7333251607242973184}]
W20260925 13:57:10.925554 21580 catalog_manager.cc:5671] Ignoring report from unknown tablet d267def3f0864435a0d2c40fd7052eab
I20260925 13:57:10.944309 22268 mvcc.cc:204] Tried to move back new op lower bound from 7333251607242973184 to 7333251604861972480. Current Snapshot: MvccSnapshot[applied={T|T < 7333251607242973184}]
I20260925 13:57:10.944309 22279 mvcc.cc:204] Tried to move back new op lower bound from 7333251607242973184 to 7333251604861972480. Current Snapshot: MvccSnapshot[applied={T|T < 7333251607242973184}]
I20260925 13:57:10.944410 21776 tablet_service.cc:1511] Processing CreateTablet for tablet 5eb86772ca2e4306a5c164342514b7b0 (DEFAULT_TABLE table=default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:10.945667 21776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5eb86772ca2e4306a5c164342514b7b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:10.878945 21585 catalog_manager.cc:5938] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291) does not have the latest schema for tablet 5c593c8aa09c4423b6c7d7bbcf94f4ab (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]). Expected version 1 got 0
I20260925 13:57:10.960879 22287 tablet.cc:1751] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 145e520e0a5a445d943a53cf8c27efee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:57:10.966972 22286 tablet.cc:1751] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P ee02f77594864349982492862d668a36: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:57:10.981887 21937 tablet_service.cc:1511] Processing CreateTablet for tablet 5eb86772ca2e4306a5c164342514b7b0 (DEFAULT_TABLE table=default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:10.988376 21937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5eb86772ca2e4306a5c164342514b7b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:10.985875 22212 tablet_bootstrap.cc:492] T cd30e39a98fd4d7fbe793bdbc02eea13 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:10.991775 22212 ts_tablet_manager.cc:1403] T cd30e39a98fd4d7fbe793bdbc02eea13 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.290s user 0.053s sys 0.114s
I20260925 13:57:10.994614 22212 raft_consensus.cc:359] T cd30e39a98fd4d7fbe793bdbc02eea13 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:10.995304 22212 raft_consensus.cc:385] T cd30e39a98fd4d7fbe793bdbc02eea13 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:10.995657 22212 raft_consensus.cc:740] T cd30e39a98fd4d7fbe793bdbc02eea13 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:10.996454 22212 consensus_queue.cc:260] T cd30e39a98fd4d7fbe793bdbc02eea13 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:10.999065 22212 ts_tablet_manager.cc:1434] T cd30e39a98fd4d7fbe793bdbc02eea13 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20260925 13:57:11.001741 22212 tablet_bootstrap.cc:492] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:11.001389 22285 tablet.cc:1751] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 5e2731975dbf433aade4a859df9699e2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:57:11.007759 22213 ts_tablet_manager.cc:1929] T 71ff569d68154b61bcc14937b2e37fe5 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:11.008394 22213 log.cc:1199] T 71ff569d68154b61bcc14937b2e37fe5 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/71ff569d68154b61bcc14937b2e37fe5
I20260925 13:57:11.012492 22212 tablet_bootstrap.cc:654] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
W20260925 13:57:11.030941 21573 consensus_peers.cc:597] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 -> Peer 3ea1e11bc6d84114b7616815b6af4635 (127.18.182.253:46177): Couldn't send request to peer 3ea1e11bc6d84114b7616815b6af4635. Status: Network error: Client connection negotiation failed: client connection to 127.18.182.253:46177: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260925 13:57:11.032004 21580 catalog_manager.cc:5938] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873) does not have the latest schema for tablet 5c593c8aa09c4423b6c7d7bbcf94f4ab (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]). Expected version 1 got 0
W20260925 13:57:11.035175 21732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.194:44291, user_credentials={real_user=slave}} blocked reactor thread for 49455.2us
I20260925 13:57:11.049640 21585 catalog_manager.cc:5838] T d4774462e5a8486fba564154d1861e6c P 5e2731975dbf433aade4a859df9699e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e2731975dbf433aade4a859df9699e2 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "5e2731975dbf433aade4a859df9699e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: UNKNOWN } } }
I20260925 13:57:11.058017 22083 tablet_service.cc:1511] Processing CreateTablet for tablet 5eb86772ca2e4306a5c164342514b7b0 (DEFAULT_TABLE table=default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:11.059141 22083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5eb86772ca2e4306a5c164342514b7b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:11.069259 22211 tablet_bootstrap.cc:492] T 5eb86772ca2e4306a5c164342514b7b0 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:11.074435 22211 tablet_bootstrap.cc:654] T 5eb86772ca2e4306a5c164342514b7b0 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:11.091737 22211 tablet_bootstrap.cc:492] T 5eb86772ca2e4306a5c164342514b7b0 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:11.092239 22211 ts_tablet_manager.cc:1403] T 5eb86772ca2e4306a5c164342514b7b0 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.023s user 0.007s sys 0.004s
I20260925 13:57:11.094615 22211 raft_consensus.cc:359] T 5eb86772ca2e4306a5c164342514b7b0 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:11.103745 21579 catalog_manager.cc:6391] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f] alter complete (version 1)
I20260925 13:57:11.105937 21579 master_service.cc:960] ReplaceTablet: received request to replace tablet 71ff569d68154b61bcc14937b2e37fe5 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:11.106644 22087 tablet_service.cc:1558] Processing DeleteTablet for tablet cd30e39a98fd4d7fbe793bdbc02eea13 with delete_type TABLET_DATA_DELETED (replaced by tablet 71ff569d68154b61bcc14937b2e37fe5) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:11.107694 22214 tablet_replica.cc:333] T cd30e39a98fd4d7fbe793bdbc02eea13 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:11.108474 22214 raft_consensus.cc:2243] T cd30e39a98fd4d7fbe793bdbc02eea13 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:11.108906 22214 raft_consensus.cc:2272] T cd30e39a98fd4d7fbe793bdbc02eea13 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:11.111932 22214 ts_tablet_manager.cc:1916] T cd30e39a98fd4d7fbe793bdbc02eea13 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:11.119928 22213 ts_tablet_manager.cc:1950] T 71ff569d68154b61bcc14937b2e37fe5 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:11.122877 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet 71ff569d68154b61bcc14937b2e37fe5 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]) successfully deleted
I20260925 13:57:11.127224 22212 tablet_bootstrap.cc:492] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:11.127616 22212 ts_tablet_manager.cc:1403] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.126s user 0.021s sys 0.028s
I20260925 13:57:11.130116 22212 raft_consensus.cc:359] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:11.130736 22212 raft_consensus.cc:385] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:11.130977 22212 raft_consensus.cc:740] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:11.131925 22212 consensus_queue.cc:260] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:11.134302 22211 raft_consensus.cc:385] T 5eb86772ca2e4306a5c164342514b7b0 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:11.134646 22211 raft_consensus.cc:740] T 5eb86772ca2e4306a5c164342514b7b0 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:11.131516 22215 ts_tablet_manager.cc:1929] T 71ff569d68154b61bcc14937b2e37fe5 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:11.136090 22215 log.cc:1199] T 71ff569d68154b61bcc14937b2e37fe5 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/71ff569d68154b61bcc14937b2e37fe5
I20260925 13:57:11.136866 22215 ts_tablet_manager.cc:1950] T 71ff569d68154b61bcc14937b2e37fe5 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:11.139124 22293 tablet_bootstrap.cc:492] T 5eb86772ca2e4306a5c164342514b7b0 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
W20260925 13:57:11.146440 21579 catalog_manager.cc:5671] Ignoring report from unknown tablet 71ff569d68154b61bcc14937b2e37fe5
I20260925 13:57:11.153059 21579 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35994:
table { table_name: "default.Table_51d65314582841419e34561a4a8d6bbb" } new_table_name: "default.Table_d6100b0f2b294ab099c179a7681cb4ed" modify_external_catalogs: true
I20260925 13:57:11.156493 22212 ts_tablet_manager.cc:1434] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.029s user 0.005s sys 0.000s
I20260925 13:57:11.155619 22211 consensus_queue.cc:260] T 5eb86772ca2e4306a5c164342514b7b0 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:11.158284 22211 ts_tablet_manager.cc:1434] T 5eb86772ca2e4306a5c164342514b7b0 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.066s user 0.004s sys 0.001s
I20260925 13:57:11.161568 22212 tablet_bootstrap.cc:492] T 5eb86772ca2e4306a5c164342514b7b0 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:11.165345 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet 71ff569d68154b61bcc14937b2e37fe5 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]) successfully deleted
I20260925 13:57:11.167857 22212 tablet_bootstrap.cc:654] T 5eb86772ca2e4306a5c164342514b7b0 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:11.170835 22214 ts_tablet_manager.cc:1929] T cd30e39a98fd4d7fbe793bdbc02eea13 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:11.171276 22214 log.cc:1199] T cd30e39a98fd4d7fbe793bdbc02eea13 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/cd30e39a98fd4d7fbe793bdbc02eea13
I20260925 13:57:11.172151 22214 ts_tablet_manager.cc:1950] T cd30e39a98fd4d7fbe793bdbc02eea13 P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:11.175596 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet cd30e39a98fd4d7fbe793bdbc02eea13 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]) successfully deleted
I20260925 13:57:11.180485 22212 tablet_bootstrap.cc:492] T 5eb86772ca2e4306a5c164342514b7b0 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
W20260925 13:57:11.182174 21580 catalog_manager.cc:5671] Ignoring report from unknown tablet 71ff569d68154b61bcc14937b2e37fe5
I20260925 13:57:11.183923 22212 ts_tablet_manager.cc:1403] T 5eb86772ca2e4306a5c164342514b7b0 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.023s user 0.009s sys 0.000s
I20260925 13:57:11.186610 22293 tablet_bootstrap.cc:654] T 5eb86772ca2e4306a5c164342514b7b0 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:11.186882 22226 raft_consensus.cc:493] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:11.187300 22226 raft_consensus.cc:515] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:11.189293 22226 leader_election.cc:290] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:11.191214 21580 catalog_manager.cc:5938] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291) does not have the latest schema for tablet 5eb86772ca2e4306a5c164342514b7b0 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 1 got 0
W20260925 13:57:11.192190 22043 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
W20260925 13:57:11.192741 22040 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
I20260925 13:57:11.193152 22040 leader_election.cc:304] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:11.194248 22226 raft_consensus.cc:2749] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:57:11.194348 21580 catalog_manager.cc:6038] Failed to send AlterTable RPC for tablet 5eb86772ca2e4306a5c164342514b7b0 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 5eb86772ca2e4306a5c164342514b7b0 consensus state has no leader
I20260925 13:57:11.199019 22293 tablet_bootstrap.cc:492] T 5eb86772ca2e4306a5c164342514b7b0 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:11.199505 22293 ts_tablet_manager.cc:1403] T 5eb86772ca2e4306a5c164342514b7b0 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.061s user 0.012s sys 0.007s
I20260925 13:57:11.200453 22212 raft_consensus.cc:359] T 5eb86772ca2e4306a5c164342514b7b0 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:11.202579 22212 raft_consensus.cc:385] T 5eb86772ca2e4306a5c164342514b7b0 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:11.203018 22293 raft_consensus.cc:359] T 5eb86772ca2e4306a5c164342514b7b0 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:11.203197 22212 raft_consensus.cc:740] T 5eb86772ca2e4306a5c164342514b7b0 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:11.203790 22293 raft_consensus.cc:385] T 5eb86772ca2e4306a5c164342514b7b0 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:11.204116 22293 raft_consensus.cc:740] T 5eb86772ca2e4306a5c164342514b7b0 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:11.204612 21580 master_service.cc:960] ReplaceTablet: received request to replace tablet 5eb86772ca2e4306a5c164342514b7b0 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:11.204852 22212 consensus_queue.cc:260] T 5eb86772ca2e4306a5c164342514b7b0 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:11.205042 22293 consensus_queue.cc:260] T 5eb86772ca2e4306a5c164342514b7b0 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:11.208500 22212 ts_tablet_manager.cc:1434] T 5eb86772ca2e4306a5c164342514b7b0 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.024s user 0.005s sys 0.000s
I20260925 13:57:11.216461 22293 ts_tablet_manager.cc:1434] T 5eb86772ca2e4306a5c164342514b7b0 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.017s user 0.002s sys 0.004s
W20260925 13:57:11.229272 22292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:57:11.242846 21580 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet 5eb86772ca2e4306a5c164342514b7b0
I20260925 13:57:11.244606 21580 catalog_manager.cc:6934] ReplaceTablet: tablet 5eb86772ca2e4306a5c164342514b7b0 deleted and replaced by tablet 3d59d9b6dfa5474f82e73e07b25b056e
I20260925 13:57:11.245347 21937 tablet_service.cc:1558] Processing DeleteTablet for tablet 5eb86772ca2e4306a5c164342514b7b0 with delete_type TABLET_DATA_DELETED (replaced by tablet 3d59d9b6dfa5474f82e73e07b25b056e) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:11.245803 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet 5eb86772ca2e4306a5c164342514b7b0 with delete_type TABLET_DATA_DELETED (replaced by tablet 3d59d9b6dfa5474f82e73e07b25b056e) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:11.246160 22215 tablet_replica.cc:333] T 5eb86772ca2e4306a5c164342514b7b0 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:11.246608 22214 tablet_replica.cc:333] T 5eb86772ca2e4306a5c164342514b7b0 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:11.246995 22215 raft_consensus.cc:2243] T 5eb86772ca2e4306a5c164342514b7b0 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:11.247500 22214 raft_consensus.cc:2243] T 5eb86772ca2e4306a5c164342514b7b0 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:11.247537 22215 raft_consensus.cc:2272] T 5eb86772ca2e4306a5c164342514b7b0 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:11.248011 22214 raft_consensus.cc:2272] T 5eb86772ca2e4306a5c164342514b7b0 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:11.249971 22215 ts_tablet_manager.cc:1916] T 5eb86772ca2e4306a5c164342514b7b0 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:11.250377 22214 ts_tablet_manager.cc:1916] T 5eb86772ca2e4306a5c164342514b7b0 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:11.253130 21776 tablet_service.cc:1558] Processing DeleteTablet for tablet 5eb86772ca2e4306a5c164342514b7b0 with delete_type TABLET_DATA_DELETED (replaced by tablet 3d59d9b6dfa5474f82e73e07b25b056e) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:11.253937 22213 tablet_replica.cc:333] T 5eb86772ca2e4306a5c164342514b7b0 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:11.254736 22213 raft_consensus.cc:2243] T 5eb86772ca2e4306a5c164342514b7b0 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:11.255216 22213 raft_consensus.cc:2272] T 5eb86772ca2e4306a5c164342514b7b0 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:11.257648 22213 ts_tablet_manager.cc:1916] T 5eb86772ca2e4306a5c164342514b7b0 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:11.290508 21776 tablet_service.cc:1511] Processing CreateTablet for tablet 3d59d9b6dfa5474f82e73e07b25b056e (DEFAULT_TABLE table=default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:11.291643 21776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d59d9b6dfa5474f82e73e07b25b056e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:11.294728 21937 tablet_service.cc:1511] Processing CreateTablet for tablet 3d59d9b6dfa5474f82e73e07b25b056e (DEFAULT_TABLE table=default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:11.295298 22083 tablet_service.cc:1511] Processing CreateTablet for tablet 3d59d9b6dfa5474f82e73e07b25b056e (DEFAULT_TABLE table=default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:11.295811 21937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d59d9b6dfa5474f82e73e07b25b056e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:11.296375 22083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d59d9b6dfa5474f82e73e07b25b056e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:11.330046 22213 ts_tablet_manager.cc:1929] T 5eb86772ca2e4306a5c164342514b7b0 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:11.330955 22215 ts_tablet_manager.cc:1929] T 5eb86772ca2e4306a5c164342514b7b0 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:11.334123 22212 tablet_bootstrap.cc:492] T 3d59d9b6dfa5474f82e73e07b25b056e P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:11.334540 22215 log.cc:1199] T 5eb86772ca2e4306a5c164342514b7b0 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/5eb86772ca2e4306a5c164342514b7b0
I20260925 13:57:11.337059 22215 ts_tablet_manager.cc:1950] T 5eb86772ca2e4306a5c164342514b7b0 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:11.340054 22212 tablet_bootstrap.cc:654] T 3d59d9b6dfa5474f82e73e07b25b056e P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:11.345556 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet 5eb86772ca2e4306a5c164342514b7b0 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]) successfully deleted
I20260925 13:57:11.351393 22213 log.cc:1199] T 5eb86772ca2e4306a5c164342514b7b0 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/5eb86772ca2e4306a5c164342514b7b0
I20260925 13:57:11.352257 22213 ts_tablet_manager.cc:1950] T 5eb86772ca2e4306a5c164342514b7b0 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:11.355685 22293 tablet_bootstrap.cc:492] T 3d59d9b6dfa5474f82e73e07b25b056e P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:11.355927 22211 tablet_bootstrap.cc:492] T 3d59d9b6dfa5474f82e73e07b25b056e P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:11.356411 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet 5eb86772ca2e4306a5c164342514b7b0 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]) successfully deleted
I20260925 13:57:11.359833 21580 master_service.cc:960] ReplaceTablet: received request to replace tablet aa509c1466e24caba7ca5caf1395bd86 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:11.361881 22211 tablet_bootstrap.cc:654] T 3d59d9b6dfa5474f82e73e07b25b056e P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:11.362300 22293 tablet_bootstrap.cc:654] T 3d59d9b6dfa5474f82e73e07b25b056e P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:11.369256 22214 ts_tablet_manager.cc:1929] T 5eb86772ca2e4306a5c164342514b7b0 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260925 13:57:11.384709 21575 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 5eb86772ca2e4306a5c164342514b7b0 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 5eb86772ca2e4306a5c164342514b7b0 consensus state has no leader
I20260925 13:57:11.386471 21580 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet aa509c1466e24caba7ca5caf1395bd86
I20260925 13:57:11.387259 22211 tablet_bootstrap.cc:492] T 3d59d9b6dfa5474f82e73e07b25b056e P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:11.387742 22211 ts_tablet_manager.cc:1403] T 3d59d9b6dfa5474f82e73e07b25b056e P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.032s user 0.011s sys 0.000s
I20260925 13:57:11.393618 22214 log.cc:1199] T 5eb86772ca2e4306a5c164342514b7b0 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/5eb86772ca2e4306a5c164342514b7b0
I20260925 13:57:11.369733 22212 tablet_bootstrap.cc:492] T 3d59d9b6dfa5474f82e73e07b25b056e P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:11.394271 22212 ts_tablet_manager.cc:1403] T 3d59d9b6dfa5474f82e73e07b25b056e P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.060s user 0.014s sys 0.001s
I20260925 13:57:11.394131 22211 raft_consensus.cc:359] T 3d59d9b6dfa5474f82e73e07b25b056e P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:11.394752 22211 raft_consensus.cc:385] T 3d59d9b6dfa5474f82e73e07b25b056e P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:11.395036 22211 raft_consensus.cc:740] T 3d59d9b6dfa5474f82e73e07b25b056e P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:11.396648 21580 catalog_manager.cc:6934] ReplaceTablet: tablet aa509c1466e24caba7ca5caf1395bd86 deleted and replaced by tablet a6c3053d4f6a4e8a80187a98c99e459d
I20260925 13:57:11.397043 22212 raft_consensus.cc:359] T 3d59d9b6dfa5474f82e73e07b25b056e P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:11.398088 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet aa509c1466e24caba7ca5caf1395bd86 with delete_type TABLET_DATA_DELETED (replaced by tablet a6c3053d4f6a4e8a80187a98c99e459d) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:11.398159 21937 tablet_service.cc:1558] Processing DeleteTablet for tablet aa509c1466e24caba7ca5caf1395bd86 with delete_type TABLET_DATA_DELETED (replaced by tablet a6c3053d4f6a4e8a80187a98c99e459d) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:11.398888 22215 tablet_replica.cc:333] T aa509c1466e24caba7ca5caf1395bd86 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:11.398880 21776 tablet_service.cc:1558] Processing DeleteTablet for tablet aa509c1466e24caba7ca5caf1395bd86 with delete_type TABLET_DATA_DELETED (replaced by tablet a6c3053d4f6a4e8a80187a98c99e459d) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:11.399766 22213 tablet_replica.cc:333] T aa509c1466e24caba7ca5caf1395bd86 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:11.400728 22213 raft_consensus.cc:2243] T aa509c1466e24caba7ca5caf1395bd86 P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:11.400975 22215 raft_consensus.cc:2243] T aa509c1466e24caba7ca5caf1395bd86 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:11.401480 22215 raft_consensus.cc:2272] T aa509c1466e24caba7ca5caf1395bd86 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:11.401544 22213 pending_rounds.cc:70] T aa509c1466e24caba7ca5caf1395bd86 P 5e2731975dbf433aade4a859df9699e2: Trying to abort 1 pending ops.
I20260925 13:57:11.401786 22213 pending_rounds.cc:77] T aa509c1466e24caba7ca5caf1395bd86 P 5e2731975dbf433aade4a859df9699e2: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251606707798016 op_type: NO_OP noop_request { }
I20260925 13:57:11.402217 22213 raft_consensus.cc:2889] T aa509c1466e24caba7ca5caf1395bd86 P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:57:11.403064 22214 ts_tablet_manager.cc:1950] T 5eb86772ca2e4306a5c164342514b7b0 P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:11.403263 22215 ts_tablet_manager.cc:1916] T aa509c1466e24caba7ca5caf1395bd86 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:11.408653 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet 5eb86772ca2e4306a5c164342514b7b0 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]) successfully deleted
I20260925 13:57:11.397750 22212 raft_consensus.cc:385] T 3d59d9b6dfa5474f82e73e07b25b056e P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:11.409293 22212 raft_consensus.cc:740] T 3d59d9b6dfa5474f82e73e07b25b056e P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:11.409993 22212 consensus_queue.cc:260] T 3d59d9b6dfa5474f82e73e07b25b056e P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:11.415660 22211 consensus_queue.cc:260] T 3d59d9b6dfa5474f82e73e07b25b056e P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:11.423230 22212 ts_tablet_manager.cc:1434] T 3d59d9b6dfa5474f82e73e07b25b056e P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.029s user 0.002s sys 0.003s
I20260925 13:57:11.427596 22213 raft_consensus.cc:2272] T aa509c1466e24caba7ca5caf1395bd86 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:11.405488 22293 tablet_bootstrap.cc:492] T 3d59d9b6dfa5474f82e73e07b25b056e P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:11.428373 22293 ts_tablet_manager.cc:1403] T 3d59d9b6dfa5474f82e73e07b25b056e P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.073s user 0.009s sys 0.005s
I20260925 13:57:11.429327 21580 catalog_manager.cc:5938] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873) does not have the latest schema for tablet 3d59d9b6dfa5474f82e73e07b25b056e (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 1 got 0
I20260925 13:57:11.434664 21585 catalog_manager.cc:5938] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291) does not have the latest schema for tablet 3d59d9b6dfa5474f82e73e07b25b056e (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 1 got 0
I20260925 13:57:11.431128 22293 raft_consensus.cc:359] T 3d59d9b6dfa5474f82e73e07b25b056e P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:11.435442 22293 raft_consensus.cc:385] T 3d59d9b6dfa5474f82e73e07b25b056e P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:11.436610 22211 ts_tablet_manager.cc:1434] T 3d59d9b6dfa5474f82e73e07b25b056e P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.049s user 0.006s sys 0.000s
I20260925 13:57:11.436272 22293 raft_consensus.cc:740] T 3d59d9b6dfa5474f82e73e07b25b056e P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:11.437920 22293 consensus_queue.cc:260] T 3d59d9b6dfa5474f82e73e07b25b056e P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:11.451378 21585 catalog_manager.cc:5938] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291) does not have the latest schema for tablet 3d59d9b6dfa5474f82e73e07b25b056e (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 1 got 0
I20260925 13:57:11.453092 22293 ts_tablet_manager.cc:1434] T 3d59d9b6dfa5474f82e73e07b25b056e P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.024s user 0.006s sys 0.000s
I20260925 13:57:11.463377 22214 tablet_replica.cc:333] T aa509c1466e24caba7ca5caf1395bd86 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:11.464565 22214 raft_consensus.cc:2243] T aa509c1466e24caba7ca5caf1395bd86 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:11.465184 22214 raft_consensus.cc:2272] T aa509c1466e24caba7ca5caf1395bd86 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:11.466662 21585 catalog_manager.cc:5938] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881) does not have the latest schema for tablet 3d59d9b6dfa5474f82e73e07b25b056e (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 1 got 0
I20260925 13:57:11.467001 22214 ts_tablet_manager.cc:1916] T aa509c1466e24caba7ca5caf1395bd86 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:11.468672 21585 master_service.cc:960] ReplaceTablet: received request to replace tablet aa509c1466e24caba7ca5caf1395bd86 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:11.474989 21776 tablet_service.cc:1511] Processing CreateTablet for tablet a6c3053d4f6a4e8a80187a98c99e459d (DEFAULT_TABLE table=default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:11.476141 21776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6c3053d4f6a4e8a80187a98c99e459d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:11.477936 22215 ts_tablet_manager.cc:1929] T aa509c1466e24caba7ca5caf1395bd86 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:11.478327 22215 log.cc:1199] T aa509c1466e24caba7ca5caf1395bd86 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/aa509c1466e24caba7ca5caf1395bd86
W20260925 13:57:11.490450 21580 catalog_manager.cc:6038] Failed to send AlterTable RPC for tablet 3d59d9b6dfa5474f82e73e07b25b056e (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 3d59d9b6dfa5474f82e73e07b25b056e consensus state has no leader
I20260925 13:57:11.493178 21937 tablet_service.cc:1511] Processing CreateTablet for tablet a6c3053d4f6a4e8a80187a98c99e459d (DEFAULT_TABLE table=default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:11.494221 21937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6c3053d4f6a4e8a80187a98c99e459d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:11.501075 22083 tablet_service.cc:1511] Processing CreateTablet for tablet a6c3053d4f6a4e8a80187a98c99e459d (DEFAULT_TABLE table=default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:11.502187 22083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6c3053d4f6a4e8a80187a98c99e459d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:11.512579 21580 catalog_manager.cc:5938] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881) does not have the latest schema for tablet 3d59d9b6dfa5474f82e73e07b25b056e (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 1 got 0
I20260925 13:57:11.519445 22215 ts_tablet_manager.cc:1950] T aa509c1466e24caba7ca5caf1395bd86 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:11.521008 22213 ts_tablet_manager.cc:1916] T aa509c1466e24caba7ca5caf1395bd86 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:11.522511 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet aa509c1466e24caba7ca5caf1395bd86 (table default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]) successfully deleted
I20260925 13:57:11.524847 22214 ts_tablet_manager.cc:1929] T aa509c1466e24caba7ca5caf1395bd86 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:11.525629 22214 log.cc:1199] T aa509c1466e24caba7ca5caf1395bd86 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/aa509c1466e24caba7ca5caf1395bd86
I20260925 13:57:11.526328 22214 ts_tablet_manager.cc:1950] T aa509c1466e24caba7ca5caf1395bd86 P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:11.528981 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet aa509c1466e24caba7ca5caf1395bd86 (table default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]) successfully deleted
I20260925 13:57:11.547457 22211 tablet_bootstrap.cc:492] T a6c3053d4f6a4e8a80187a98c99e459d P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:11.552347 22293 tablet_bootstrap.cc:492] T a6c3053d4f6a4e8a80187a98c99e459d P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:11.553022 22211 tablet_bootstrap.cc:654] T a6c3053d4f6a4e8a80187a98c99e459d P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:11.559561 22293 tablet_bootstrap.cc:654] T a6c3053d4f6a4e8a80187a98c99e459d P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:11.562172 21585 catalog_manager.cc:5938] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873) does not have the latest schema for tablet 3d59d9b6dfa5474f82e73e07b25b056e (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 1 got 0
I20260925 13:57:11.562463 22211 tablet_bootstrap.cc:492] T a6c3053d4f6a4e8a80187a98c99e459d P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:11.562937 22211 ts_tablet_manager.cc:1403] T a6c3053d4f6a4e8a80187a98c99e459d P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.002s
I20260925 13:57:11.565495 22211 raft_consensus.cc:359] T a6c3053d4f6a4e8a80187a98c99e459d P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:11.569579 22212 tablet_bootstrap.cc:492] T a6c3053d4f6a4e8a80187a98c99e459d P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:11.575165 22212 tablet_bootstrap.cc:654] T a6c3053d4f6a4e8a80187a98c99e459d P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:11.581115 22211 raft_consensus.cc:385] T a6c3053d4f6a4e8a80187a98c99e459d P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:11.581408 22211 raft_consensus.cc:740] T a6c3053d4f6a4e8a80187a98c99e459d P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:11.591679 22293 tablet_bootstrap.cc:492] T a6c3053d4f6a4e8a80187a98c99e459d P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:11.594518 22293 ts_tablet_manager.cc:1403] T a6c3053d4f6a4e8a80187a98c99e459d P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.042s user 0.007s sys 0.004s
I20260925 13:57:11.588491 22211 consensus_queue.cc:260] T a6c3053d4f6a4e8a80187a98c99e459d P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:11.597388 22293 raft_consensus.cc:359] T a6c3053d4f6a4e8a80187a98c99e459d P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:11.598217 22293 raft_consensus.cc:385] T a6c3053d4f6a4e8a80187a98c99e459d P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:11.598501 22293 raft_consensus.cc:740] T a6c3053d4f6a4e8a80187a98c99e459d P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
W20260925 13:57:11.605090 21585 catalog_manager.cc:5671] Ignoring report from unknown tablet d267def3f0864435a0d2c40fd7052eab
I20260925 13:57:11.609087 22211 ts_tablet_manager.cc:1434] T a6c3053d4f6a4e8a80187a98c99e459d P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.046s user 0.004s sys 0.001s
I20260925 13:57:11.615811 21585 master_service.cc:960] ReplaceTablet: received request to replace tablet d267def3f0864435a0d2c40fd7052eab from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:11.631735 22289 raft_consensus.cc:493] T 3d59d9b6dfa5474f82e73e07b25b056e P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:11.632197 22289 raft_consensus.cc:515] T 3d59d9b6dfa5474f82e73e07b25b056e P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:11.635174 22289 leader_election.cc:290] T 3d59d9b6dfa5474f82e73e07b25b056e P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:11.636657 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d59d9b6dfa5474f82e73e07b25b056e" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:11.637290 21960 raft_consensus.cc:2468] T 3d59d9b6dfa5474f82e73e07b25b056e P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:11.639441 22105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d59d9b6dfa5474f82e73e07b25b056e" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:11.640223 22105 raft_consensus.cc:2468] T 3d59d9b6dfa5474f82e73e07b25b056e P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:11.641556 22212 tablet_bootstrap.cc:492] T a6c3053d4f6a4e8a80187a98c99e459d P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:11.642012 22212 ts_tablet_manager.cc:1403] T a6c3053d4f6a4e8a80187a98c99e459d P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.073s user 0.014s sys 0.003s
I20260925 13:57:11.641911 21732 leader_election.cc:304] T 3d59d9b6dfa5474f82e73e07b25b056e P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:11.643165 22289 raft_consensus.cc:2804] T 3d59d9b6dfa5474f82e73e07b25b056e P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:11.643493 22289 raft_consensus.cc:493] T 3d59d9b6dfa5474f82e73e07b25b056e P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:11.643837 22289 raft_consensus.cc:3060] T 3d59d9b6dfa5474f82e73e07b25b056e P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:11.648101 22212 raft_consensus.cc:359] T a6c3053d4f6a4e8a80187a98c99e459d P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:11.650200 22289 raft_consensus.cc:515] T 3d59d9b6dfa5474f82e73e07b25b056e P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:11.650876 22212 raft_consensus.cc:385] T a6c3053d4f6a4e8a80187a98c99e459d P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:11.651180 22212 raft_consensus.cc:740] T a6c3053d4f6a4e8a80187a98c99e459d P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:11.652024 22212 consensus_queue.cc:260] T a6c3053d4f6a4e8a80187a98c99e459d P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:11.660960 22212 ts_tablet_manager.cc:1434] T a6c3053d4f6a4e8a80187a98c99e459d P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.019s user 0.005s sys 0.000s
I20260925 13:57:11.621323 22293 consensus_queue.cc:260] T a6c3053d4f6a4e8a80187a98c99e459d P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:11.672115 22293 ts_tablet_manager.cc:1434] T a6c3053d4f6a4e8a80187a98c99e459d P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.077s user 0.007s sys 0.000s
I20260925 13:57:11.688810 22105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d59d9b6dfa5474f82e73e07b25b056e" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:11.689395 22105 raft_consensus.cc:3060] T 3d59d9b6dfa5474f82e73e07b25b056e P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:11.690538 22289 leader_election.cc:290] T 3d59d9b6dfa5474f82e73e07b25b056e P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Requested vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:11.691205 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d59d9b6dfa5474f82e73e07b25b056e" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
I20260925 13:57:11.691803 21960 raft_consensus.cc:3060] T 3d59d9b6dfa5474f82e73e07b25b056e P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:11.696089 22105 raft_consensus.cc:2468] T 3d59d9b6dfa5474f82e73e07b25b056e P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:11.700453 21732 leader_election.cc:304] T 3d59d9b6dfa5474f82e73e07b25b056e P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:11.701489 22305 raft_consensus.cc:2804] T 3d59d9b6dfa5474f82e73e07b25b056e P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:11.701937 22305 raft_consensus.cc:697] T 3d59d9b6dfa5474f82e73e07b25b056e P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Becoming Leader. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Running, Role: LEADER
I20260925 13:57:11.702677 22305 consensus_queue.cc:237] T 3d59d9b6dfa5474f82e73e07b25b056e P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:11.703287 21960 raft_consensus.cc:2468] T 3d59d9b6dfa5474f82e73e07b25b056e P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:11.701203 22213 ts_tablet_manager.cc:1929] T aa509c1466e24caba7ca5caf1395bd86 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:11.715482 22213 log.cc:1199] T aa509c1466e24caba7ca5caf1395bd86 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/aa509c1466e24caba7ca5caf1395bd86
I20260925 13:57:11.716239 22213 ts_tablet_manager.cc:1950] T aa509c1466e24caba7ca5caf1395bd86 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:11.717108 22289 raft_consensus.cc:493] T a6c3053d4f6a4e8a80187a98c99e459d P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:11.717950 22289 raft_consensus.cc:515] T a6c3053d4f6a4e8a80187a98c99e459d P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:11.723165 21585 catalog_manager.cc:5838] T 3d59d9b6dfa5474f82e73e07b25b056e P 5e2731975dbf433aade4a859df9699e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e2731975dbf433aade4a859df9699e2 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "5e2731975dbf433aade4a859df9699e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: UNKNOWN } } }
I20260925 13:57:11.724081 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c3053d4f6a4e8a80187a98c99e459d" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:11.724717 21960 raft_consensus.cc:2468] T a6c3053d4f6a4e8a80187a98c99e459d P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:11.725932 21732 leader_election.cc:304] T a6c3053d4f6a4e8a80187a98c99e459d P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:11.726671 22305 raft_consensus.cc:2804] T a6c3053d4f6a4e8a80187a98c99e459d P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:11.726969 22305 raft_consensus.cc:493] T a6c3053d4f6a4e8a80187a98c99e459d P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:11.727249 22305 raft_consensus.cc:3060] T a6c3053d4f6a4e8a80187a98c99e459d P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:11.725276 21585 catalog_manager.cc:5938] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881) does not have the latest schema for tablet 3d59d9b6dfa5474f82e73e07b25b056e (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 1 got 0
I20260925 13:57:11.729347 22105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c3053d4f6a4e8a80187a98c99e459d" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:11.729928 22105 raft_consensus.cc:2468] T a6c3053d4f6a4e8a80187a98c99e459d P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:11.732677 22289 leader_election.cc:290] T a6c3053d4f6a4e8a80187a98c99e459d P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:11.733469 22305 raft_consensus.cc:515] T a6c3053d4f6a4e8a80187a98c99e459d P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:11.735752 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet aa509c1466e24caba7ca5caf1395bd86 (table default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]) successfully deleted
I20260925 13:57:11.742307 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c3053d4f6a4e8a80187a98c99e459d" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
I20260925 13:57:11.742861 21960 raft_consensus.cc:3060] T a6c3053d4f6a4e8a80187a98c99e459d P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:11.744019 21580 master_service.cc:960] ReplaceTablet: received request to replace tablet d4774462e5a8486fba564154d1861e6c from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:11.749279 21960 raft_consensus.cc:2468] T a6c3053d4f6a4e8a80187a98c99e459d P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:11.750582 21732 leader_election.cc:304] T a6c3053d4f6a4e8a80187a98c99e459d P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:11.751400 22289 raft_consensus.cc:2804] T a6c3053d4f6a4e8a80187a98c99e459d P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:11.751816 22289 raft_consensus.cc:697] T a6c3053d4f6a4e8a80187a98c99e459d P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Becoming Leader. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Running, Role: LEADER
I20260925 13:57:11.752562 22289 consensus_queue.cc:237] T a6c3053d4f6a4e8a80187a98c99e459d P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:11.753676 22105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c3053d4f6a4e8a80187a98c99e459d" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:11.754195 22105 raft_consensus.cc:3060] T a6c3053d4f6a4e8a80187a98c99e459d P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:11.760794 22105 raft_consensus.cc:2468] T a6c3053d4f6a4e8a80187a98c99e459d P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:11.763521 22305 leader_election.cc:290] T a6c3053d4f6a4e8a80187a98c99e459d P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Requested vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:11.784148 21580 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet d4774462e5a8486fba564154d1861e6c
I20260925 13:57:11.786168 21937 tablet_service.cc:1558] Processing DeleteTablet for tablet d4774462e5a8486fba564154d1861e6c with delete_type TABLET_DATA_DELETED (replaced by tablet 524f19a326ad4dd78353d5c4dab8bcc1) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:11.786856 22215 tablet_replica.cc:333] T d4774462e5a8486fba564154d1861e6c P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:11.787688 22215 raft_consensus.cc:2243] T d4774462e5a8486fba564154d1861e6c P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:11.788174 22215 raft_consensus.cc:2272] T d4774462e5a8486fba564154d1861e6c P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:11.789924 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet d4774462e5a8486fba564154d1861e6c with delete_type TABLET_DATA_DELETED (replaced by tablet 524f19a326ad4dd78353d5c4dab8bcc1) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:11.790800 22215 ts_tablet_manager.cc:1916] T d4774462e5a8486fba564154d1861e6c P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:11.793309 22214 tablet_replica.cc:333] T d4774462e5a8486fba564154d1861e6c P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:11.794044 22214 raft_consensus.cc:2243] T d4774462e5a8486fba564154d1861e6c P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:11.794490 22214 raft_consensus.cc:2272] T d4774462e5a8486fba564154d1861e6c P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:11.795990 21776 tablet_service.cc:1558] Processing DeleteTablet for tablet d4774462e5a8486fba564154d1861e6c with delete_type TABLET_DATA_DELETED (replaced by tablet 524f19a326ad4dd78353d5c4dab8bcc1) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:11.796633 22213 tablet_replica.cc:333] T d4774462e5a8486fba564154d1861e6c P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:11.796540 22214 ts_tablet_manager.cc:1916] T d4774462e5a8486fba564154d1861e6c P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:11.796933 21580 catalog_manager.cc:6934] ReplaceTablet: tablet d4774462e5a8486fba564154d1861e6c deleted and replaced by tablet 524f19a326ad4dd78353d5c4dab8bcc1
I20260925 13:57:11.797300 22213 raft_consensus.cc:2243] T d4774462e5a8486fba564154d1861e6c P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:11.797925 22213 pending_rounds.cc:70] T d4774462e5a8486fba564154d1861e6c P 5e2731975dbf433aade4a859df9699e2: Trying to abort 1 pending ops.
I20260925 13:57:11.798147 22213 pending_rounds.cc:77] T d4774462e5a8486fba564154d1861e6c P 5e2731975dbf433aade4a859df9699e2: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251608104054784 op_type: NO_OP noop_request { }
I20260925 13:57:11.798458 22213 raft_consensus.cc:2889] T d4774462e5a8486fba564154d1861e6c P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:57:11.798768 22213 raft_consensus.cc:2272] T d4774462e5a8486fba564154d1861e6c P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:11.800809 22213 ts_tablet_manager.cc:1916] T d4774462e5a8486fba564154d1861e6c P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:11.815375 22214 ts_tablet_manager.cc:1929] T d4774462e5a8486fba564154d1861e6c P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:11.815783 22214 log.cc:1199] T d4774462e5a8486fba564154d1861e6c P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/d4774462e5a8486fba564154d1861e6c
I20260925 13:57:11.816341 22214 ts_tablet_manager.cc:1950] T d4774462e5a8486fba564154d1861e6c P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:11.818914 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet d4774462e5a8486fba564154d1861e6c (table default.table_492c567ddda14e77851270f34471730e [id=754c40502e614c5bb18230c852aa0016]) successfully deleted
I20260925 13:57:11.824182 21580 master_service.cc:960] ReplaceTablet: received request to replace tablet a6c3053d4f6a4e8a80187a98c99e459d from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:11.827302 22213 ts_tablet_manager.cc:1929] T d4774462e5a8486fba564154d1861e6c P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:11.833065 22213 log.cc:1199] T d4774462e5a8486fba564154d1861e6c P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/d4774462e5a8486fba564154d1861e6c
I20260925 13:57:11.834020 22213 ts_tablet_manager.cc:1950] T d4774462e5a8486fba564154d1861e6c P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:11.838763 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet d4774462e5a8486fba564154d1861e6c (table default.table_492c567ddda14e77851270f34471730e [id=754c40502e614c5bb18230c852aa0016]) successfully deleted
I20260925 13:57:11.853713 21776 tablet_service.cc:1511] Processing CreateTablet for tablet 524f19a326ad4dd78353d5c4dab8bcc1 (DEFAULT_TABLE table=default.table_492c567ddda14e77851270f34471730e [id=754c40502e614c5bb18230c852aa0016]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:11.854727 21776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 524f19a326ad4dd78353d5c4dab8bcc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:11.878048 21937 tablet_service.cc:1511] Processing CreateTablet for tablet 524f19a326ad4dd78353d5c4dab8bcc1 (DEFAULT_TABLE table=default.table_492c567ddda14e77851270f34471730e [id=754c40502e614c5bb18230c852aa0016]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:11.879475 21937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 524f19a326ad4dd78353d5c4dab8bcc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:11.897958 21580 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet a6c3053d4f6a4e8a80187a98c99e459d
I20260925 13:57:11.900171 21938 tablet_service.cc:1558] Processing DeleteTablet for tablet a6c3053d4f6a4e8a80187a98c99e459d with delete_type TABLET_DATA_DELETED (replaced by tablet 771c16f54fcd4cea9d38bc3834b043f1) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:11.901648 21777 tablet_service.cc:1558] Processing DeleteTablet for tablet a6c3053d4f6a4e8a80187a98c99e459d with delete_type TABLET_DATA_DELETED (replaced by tablet 771c16f54fcd4cea9d38bc3834b043f1) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:11.902365 22213 tablet_replica.cc:333] T a6c3053d4f6a4e8a80187a98c99e459d P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:11.903000 22213 raft_consensus.cc:2243] T a6c3053d4f6a4e8a80187a98c99e459d P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:11.903055 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet a6c3053d4f6a4e8a80187a98c99e459d with delete_type TABLET_DATA_DELETED (replaced by tablet 771c16f54fcd4cea9d38bc3834b043f1) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:11.903651 22213 pending_rounds.cc:70] T a6c3053d4f6a4e8a80187a98c99e459d P 5e2731975dbf433aade4a859df9699e2: Trying to abort 1 pending ops.
I20260925 13:57:11.903697 22214 tablet_replica.cc:333] T a6c3053d4f6a4e8a80187a98c99e459d P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:11.903858 22213 pending_rounds.cc:77] T a6c3053d4f6a4e8a80187a98c99e459d P 5e2731975dbf433aade4a859df9699e2: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251611670302720 op_type: NO_OP noop_request { }
I20260925 13:57:11.904189 22213 raft_consensus.cc:2889] T a6c3053d4f6a4e8a80187a98c99e459d P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:57:11.904357 22214 raft_consensus.cc:2243] T a6c3053d4f6a4e8a80187a98c99e459d P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:11.904490 22213 raft_consensus.cc:2272] T a6c3053d4f6a4e8a80187a98c99e459d P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:11.904755 22214 raft_consensus.cc:2272] T a6c3053d4f6a4e8a80187a98c99e459d P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:11.906839 22214 ts_tablet_manager.cc:1916] T a6c3053d4f6a4e8a80187a98c99e459d P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:11.907027 22213 ts_tablet_manager.cc:1916] T a6c3053d4f6a4e8a80187a98c99e459d P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:11.914713 21580 catalog_manager.cc:6934] ReplaceTablet: tablet a6c3053d4f6a4e8a80187a98c99e459d deleted and replaced by tablet 771c16f54fcd4cea9d38bc3834b043f1
I20260925 13:57:11.921591 22083 tablet_service.cc:1511] Processing CreateTablet for tablet 524f19a326ad4dd78353d5c4dab8bcc1 (DEFAULT_TABLE table=default.table_492c567ddda14e77851270f34471730e [id=754c40502e614c5bb18230c852aa0016]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:11.922547 22083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 524f19a326ad4dd78353d5c4dab8bcc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:11.955009 22213 ts_tablet_manager.cc:1929] T a6c3053d4f6a4e8a80187a98c99e459d P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:11.955404 22213 log.cc:1199] T a6c3053d4f6a4e8a80187a98c99e459d P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/a6c3053d4f6a4e8a80187a98c99e459d
I20260925 13:57:11.956127 22213 ts_tablet_manager.cc:1950] T a6c3053d4f6a4e8a80187a98c99e459d P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:11.958621 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet a6c3053d4f6a4e8a80187a98c99e459d (table default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]) successfully deleted
I20260925 13:57:11.968394 22293 tablet_bootstrap.cc:492] T 524f19a326ad4dd78353d5c4dab8bcc1 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:11.972903 22212 tablet_bootstrap.cc:492] T 524f19a326ad4dd78353d5c4dab8bcc1 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:11.974645 22293 tablet_bootstrap.cc:654] T 524f19a326ad4dd78353d5c4dab8bcc1 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:11.978382 22212 tablet_bootstrap.cc:654] T 524f19a326ad4dd78353d5c4dab8bcc1 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:11.988293 22214 ts_tablet_manager.cc:1929] T a6c3053d4f6a4e8a80187a98c99e459d P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:11.988761 22214 log.cc:1199] T a6c3053d4f6a4e8a80187a98c99e459d P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/a6c3053d4f6a4e8a80187a98c99e459d
I20260925 13:57:11.971463 22215 ts_tablet_manager.cc:1929] T d4774462e5a8486fba564154d1861e6c P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:11.989333 22215 log.cc:1199] T d4774462e5a8486fba564154d1861e6c P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/d4774462e5a8486fba564154d1861e6c
I20260925 13:57:11.990052 22215 ts_tablet_manager.cc:1950] T d4774462e5a8486fba564154d1861e6c P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:11.990968 22214 ts_tablet_manager.cc:1950] T a6c3053d4f6a4e8a80187a98c99e459d P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:11.992093 22293 tablet_bootstrap.cc:492] T 524f19a326ad4dd78353d5c4dab8bcc1 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:11.992517 22293 ts_tablet_manager.cc:1403] T 524f19a326ad4dd78353d5c4dab8bcc1 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.024s user 0.008s sys 0.003s
I20260925 13:57:11.996897 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet a6c3053d4f6a4e8a80187a98c99e459d (table default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]) successfully deleted
I20260925 13:57:11.999397 22293 raft_consensus.cc:359] T 524f19a326ad4dd78353d5c4dab8bcc1 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:12.000053 22293 raft_consensus.cc:385] T 524f19a326ad4dd78353d5c4dab8bcc1 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:12.000310 22293 raft_consensus.cc:740] T 524f19a326ad4dd78353d5c4dab8bcc1 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:12.001065 22293 consensus_queue.cc:260] T 524f19a326ad4dd78353d5c4dab8bcc1 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:12.002982 22212 tablet_bootstrap.cc:492] T 524f19a326ad4dd78353d5c4dab8bcc1 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:12.003453 22212 ts_tablet_manager.cc:1403] T 524f19a326ad4dd78353d5c4dab8bcc1 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.031s user 0.004s sys 0.006s
I20260925 13:57:12.004736 21585 master_service.cc:960] ReplaceTablet: received request to replace tablet 5c593c8aa09c4423b6c7d7bbcf94f4ab from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:12.007757 22293 ts_tablet_manager.cc:1434] T 524f19a326ad4dd78353d5c4dab8bcc1 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.015s user 0.007s sys 0.000s
I20260925 13:57:12.025002 22212 raft_consensus.cc:359] T 524f19a326ad4dd78353d5c4dab8bcc1 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:12.025770 22212 raft_consensus.cc:385] T 524f19a326ad4dd78353d5c4dab8bcc1 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:12.026104 22212 raft_consensus.cc:740] T 524f19a326ad4dd78353d5c4dab8bcc1 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:12.026896 22212 consensus_queue.cc:260] T 524f19a326ad4dd78353d5c4dab8bcc1 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:12.035497 22215 tablet_replica.cc:333] T a6c3053d4f6a4e8a80187a98c99e459d P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:12.036185 22215 raft_consensus.cc:2243] T a6c3053d4f6a4e8a80187a98c99e459d P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:12.036612 22215 raft_consensus.cc:2272] T a6c3053d4f6a4e8a80187a98c99e459d P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:12.039202 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet d4774462e5a8486fba564154d1861e6c (table default.table_492c567ddda14e77851270f34471730e [id=754c40502e614c5bb18230c852aa0016]) successfully deleted
I20260925 13:57:12.047585 22211 tablet_bootstrap.cc:492] T 524f19a326ad4dd78353d5c4dab8bcc1 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:12.048640 22215 ts_tablet_manager.cc:1916] T a6c3053d4f6a4e8a80187a98c99e459d P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:12.056229 22211 tablet_bootstrap.cc:654] T 524f19a326ad4dd78353d5c4dab8bcc1 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:12.079680 21585 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet 5c593c8aa09c4423b6c7d7bbcf94f4ab
I20260925 13:57:12.080765 22211 tablet_bootstrap.cc:492] T 524f19a326ad4dd78353d5c4dab8bcc1 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:12.081154 22211 ts_tablet_manager.cc:1403] T 524f19a326ad4dd78353d5c4dab8bcc1 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.034s user 0.011s sys 0.003s
I20260925 13:57:12.083369 22211 raft_consensus.cc:359] T 524f19a326ad4dd78353d5c4dab8bcc1 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:12.084012 22211 raft_consensus.cc:385] T 524f19a326ad4dd78353d5c4dab8bcc1 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:12.084267 22211 raft_consensus.cc:740] T 524f19a326ad4dd78353d5c4dab8bcc1 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:12.084993 22211 consensus_queue.cc:260] T 524f19a326ad4dd78353d5c4dab8bcc1 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:12.089910 22211 ts_tablet_manager.cc:1434] T 524f19a326ad4dd78353d5c4dab8bcc1 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260925 13:57:12.079229 22215 ts_tablet_manager.cc:1929] T a6c3053d4f6a4e8a80187a98c99e459d P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:12.090576 22215 log.cc:1199] T a6c3053d4f6a4e8a80187a98c99e459d P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/a6c3053d4f6a4e8a80187a98c99e459d
I20260925 13:57:12.091202 22215 ts_tablet_manager.cc:1950] T a6c3053d4f6a4e8a80187a98c99e459d P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:12.108893 21777 tablet_service.cc:1511] Processing CreateTablet for tablet 771c16f54fcd4cea9d38bc3834b043f1 (DEFAULT_TABLE table=default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:12.109952 21777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 771c16f54fcd4cea9d38bc3834b043f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:12.095577 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet a6c3053d4f6a4e8a80187a98c99e459d (table default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]) successfully deleted
I20260925 13:57:12.119257 22083 tablet_service.cc:1511] Processing CreateTablet for tablet 771c16f54fcd4cea9d38bc3834b043f1 (DEFAULT_TABLE table=default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:12.119722 21937 tablet_service.cc:1511] Processing CreateTablet for tablet 771c16f54fcd4cea9d38bc3834b043f1 (DEFAULT_TABLE table=default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:12.120435 22083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 771c16f54fcd4cea9d38bc3834b043f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:12.120739 21937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 771c16f54fcd4cea9d38bc3834b043f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:12.125396 21938 tablet_service.cc:1558] Processing DeleteTablet for tablet 5c593c8aa09c4423b6c7d7bbcf94f4ab with delete_type TABLET_DATA_DELETED (replaced by tablet 5888d8f93cec4450847ccfe4854e4012) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:12.126128 22215 tablet_replica.cc:333] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:12.126742 22215 raft_consensus.cc:2243] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:12.127285 22215 raft_consensus.cc:2272] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:12.129262 22215 ts_tablet_manager.cc:1916] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:12.135502 21775 tablet_service.cc:1558] Processing DeleteTablet for tablet 5c593c8aa09c4423b6c7d7bbcf94f4ab with delete_type TABLET_DATA_DELETED (replaced by tablet 5888d8f93cec4450847ccfe4854e4012) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:12.136304 22213 tablet_replica.cc:333] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:12.136934 22213 raft_consensus.cc:2243] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:12.137781 22213 raft_consensus.cc:2272] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:12.140050 22213 ts_tablet_manager.cc:1916] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:12.140270 22087 tablet_service.cc:1558] Processing DeleteTablet for tablet 5c593c8aa09c4423b6c7d7bbcf94f4ab with delete_type TABLET_DATA_DELETED (replaced by tablet 5888d8f93cec4450847ccfe4854e4012) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:12.140995 22214 tablet_replica.cc:333] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:12.141718 22214 raft_consensus.cc:2243] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:12.142261 22214 raft_consensus.cc:2272] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:12.143481 21585 catalog_manager.cc:6934] ReplaceTablet: tablet 5c593c8aa09c4423b6c7d7bbcf94f4ab deleted and replaced by tablet 5888d8f93cec4450847ccfe4854e4012
I20260925 13:57:12.144053 22214 ts_tablet_manager.cc:1916] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:12.146938 21585 master_service.cc:960] ReplaceTablet: received request to replace tablet 524f19a326ad4dd78353d5c4dab8bcc1 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:12.171106 22213 ts_tablet_manager.cc:1929] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:57:12.185590 22211 tablet_bootstrap.cc:492] T 771c16f54fcd4cea9d38bc3834b043f1 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:12.194518 22211 tablet_bootstrap.cc:654] T 771c16f54fcd4cea9d38bc3834b043f1 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:12.206475 22213 log.cc:1199] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/5c593c8aa09c4423b6c7d7bbcf94f4ab
I20260925 13:57:12.208050 22213 ts_tablet_manager.cc:1950] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:12.213999 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet 5c593c8aa09c4423b6c7d7bbcf94f4ab (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]) successfully deleted
I20260925 13:57:12.218730 22211 tablet_bootstrap.cc:492] T 771c16f54fcd4cea9d38bc3834b043f1 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:12.219127 22211 ts_tablet_manager.cc:1403] T 771c16f54fcd4cea9d38bc3834b043f1 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.034s user 0.010s sys 0.004s
I20260925 13:57:12.221280 22211 raft_consensus.cc:359] T 771c16f54fcd4cea9d38bc3834b043f1 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:12.222710 22211 raft_consensus.cc:385] T 771c16f54fcd4cea9d38bc3834b043f1 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:12.222968 22211 raft_consensus.cc:740] T 771c16f54fcd4cea9d38bc3834b043f1 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:12.223615 22211 consensus_queue.cc:260] T 771c16f54fcd4cea9d38bc3834b043f1 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:12.225365 22211 ts_tablet_manager.cc:1434] T 771c16f54fcd4cea9d38bc3834b043f1 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260925 13:57:12.242069 22105 raft_consensus.cc:1275] T 3d59d9b6dfa5474f82e73e07b25b056e P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Refusing update from remote peer 5e2731975dbf433aade4a859df9699e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:57:12.243778 22215 ts_tablet_manager.cc:1929] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:57:12.244148 22215 log.cc:1199] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/5c593c8aa09c4423b6c7d7bbcf94f4ab
I20260925 13:57:12.244635 22305 consensus_queue.cc:1048] T 3d59d9b6dfa5474f82e73e07b25b056e P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 13:57:12.245009 22215 ts_tablet_manager.cc:1950] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:12.246902 21960 raft_consensus.cc:1275] T 3d59d9b6dfa5474f82e73e07b25b056e P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Refusing update from remote peer 5e2731975dbf433aade4a859df9699e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:57:12.254597 22293 tablet_bootstrap.cc:492] T 771c16f54fcd4cea9d38bc3834b043f1 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:12.255025 22305 consensus_queue.cc:1048] T 3d59d9b6dfa5474f82e73e07b25b056e P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260925 13:57:12.261013 22293 tablet_bootstrap.cc:654] T 771c16f54fcd4cea9d38bc3834b043f1 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:12.280437 22293 tablet_bootstrap.cc:492] T 771c16f54fcd4cea9d38bc3834b043f1 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:12.280382 22214 ts_tablet_manager.cc:1929] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:57:12.280768 22214 log.cc:1199] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/5c593c8aa09c4423b6c7d7bbcf94f4ab
I20260925 13:57:12.280859 22293 ts_tablet_manager.cc:1403] T 771c16f54fcd4cea9d38bc3834b043f1 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.027s user 0.010s sys 0.002s
I20260925 13:57:12.281502 22214 ts_tablet_manager.cc:1950] T 5c593c8aa09c4423b6c7d7bbcf94f4ab P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:12.283344 22293 raft_consensus.cc:359] T 771c16f54fcd4cea9d38bc3834b043f1 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:12.284168 22293 raft_consensus.cc:385] T 771c16f54fcd4cea9d38bc3834b043f1 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:12.284449 22293 raft_consensus.cc:740] T 771c16f54fcd4cea9d38bc3834b043f1 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:12.284581 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet 5c593c8aa09c4423b6c7d7bbcf94f4ab (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]) successfully deleted
I20260925 13:57:12.285075 22293 consensus_queue.cc:260] T 771c16f54fcd4cea9d38bc3834b043f1 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:12.290558 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet 5c593c8aa09c4423b6c7d7bbcf94f4ab (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]) successfully deleted
I20260925 13:57:12.309401 22212 ts_tablet_manager.cc:1434] T 524f19a326ad4dd78353d5c4dab8bcc1 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.306s user 0.008s sys 0.000s
I20260925 13:57:12.310925 22212 tablet_bootstrap.cc:492] T 771c16f54fcd4cea9d38bc3834b043f1 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:12.312321 21585 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet 524f19a326ad4dd78353d5c4dab8bcc1
I20260925 13:57:12.314393 21776 tablet_service.cc:1558] Processing DeleteTablet for tablet 524f19a326ad4dd78353d5c4dab8bcc1 with delete_type TABLET_DATA_DELETED (replaced by tablet 0d216948345a4db488fd0a9ff367bfd3) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:12.315073 22213 tablet_replica.cc:333] T 524f19a326ad4dd78353d5c4dab8bcc1 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:12.315709 22213 raft_consensus.cc:2243] T 524f19a326ad4dd78353d5c4dab8bcc1 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:12.316128 22213 raft_consensus.cc:2272] T 524f19a326ad4dd78353d5c4dab8bcc1 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:12.319685 21937 tablet_service.cc:1558] Processing DeleteTablet for tablet 524f19a326ad4dd78353d5c4dab8bcc1 with delete_type TABLET_DATA_DELETED (replaced by tablet 0d216948345a4db488fd0a9ff367bfd3) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:12.320353 22215 tablet_replica.cc:333] T 524f19a326ad4dd78353d5c4dab8bcc1 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:12.321022 22215 raft_consensus.cc:2243] T 524f19a326ad4dd78353d5c4dab8bcc1 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:12.321467 22215 raft_consensus.cc:2272] T 524f19a326ad4dd78353d5c4dab8bcc1 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:12.319447 22213 ts_tablet_manager.cc:1916] T 524f19a326ad4dd78353d5c4dab8bcc1 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:12.326860 22325 tablet.cc:1751] T 3d59d9b6dfa5474f82e73e07b25b056e P 145e520e0a5a445d943a53cf8c27efee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:57:12.344848 22324 tablet.cc:1751] T 3d59d9b6dfa5474f82e73e07b25b056e P 5e2731975dbf433aade4a859df9699e2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:57:12.346005 22226 raft_consensus.cc:493] T 524f19a326ad4dd78353d5c4dab8bcc1 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:12.346552 22226 raft_consensus.cc:515] T 524f19a326ad4dd78353d5c4dab8bcc1 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:12.348677 22226 leader_election.cc:290] T 524f19a326ad4dd78353d5c4dab8bcc1 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:12.351437 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "524f19a326ad4dd78353d5c4dab8bcc1" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2" is_pre_election: true
W20260925 13:57:12.352955 22040 leader_election.cc:343] T 524f19a326ad4dd78353d5c4dab8bcc1 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Illegal state: Tablet not RUNNING: STOPPED
I20260925 13:57:12.363612 22326 tablet.cc:1751] T 3d59d9b6dfa5474f82e73e07b25b056e P ee02f77594864349982492862d668a36: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:57:12.376636 21585 catalog_manager.cc:6934] ReplaceTablet: tablet 524f19a326ad4dd78353d5c4dab8bcc1 deleted and replaced by tablet 0d216948345a4db488fd0a9ff367bfd3
I20260925 13:57:12.377126 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet 524f19a326ad4dd78353d5c4dab8bcc1 with delete_type TABLET_DATA_DELETED (replaced by tablet 0d216948345a4db488fd0a9ff367bfd3) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:12.378294 22214 tablet_replica.cc:333] T 524f19a326ad4dd78353d5c4dab8bcc1 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:12.378999 22214 raft_consensus.cc:2243] T 524f19a326ad4dd78353d5c4dab8bcc1 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:12.379490 22214 raft_consensus.cc:2272] T 524f19a326ad4dd78353d5c4dab8bcc1 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:12.383416 22214 ts_tablet_manager.cc:1916] T 524f19a326ad4dd78353d5c4dab8bcc1 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:12.384511 22213 ts_tablet_manager.cc:1929] T 524f19a326ad4dd78353d5c4dab8bcc1 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:12.384853 22213 log.cc:1199] T 524f19a326ad4dd78353d5c4dab8bcc1 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/524f19a326ad4dd78353d5c4dab8bcc1
I20260925 13:57:12.328747 22215 ts_tablet_manager.cc:1916] T 524f19a326ad4dd78353d5c4dab8bcc1 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:12.392686 22293 ts_tablet_manager.cc:1434] T 771c16f54fcd4cea9d38bc3834b043f1 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.112s user 0.018s sys 0.016s
I20260925 13:57:12.419085 22213 ts_tablet_manager.cc:1950] T 524f19a326ad4dd78353d5c4dab8bcc1 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:12.422026 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet 524f19a326ad4dd78353d5c4dab8bcc1 (table default.table_492c567ddda14e77851270f34471730e [id=754c40502e614c5bb18230c852aa0016]) successfully deleted
I20260925 13:57:12.379635 21585 catalog_manager.cc:5938] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873) does not have the latest schema for tablet 3d59d9b6dfa5474f82e73e07b25b056e (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 1 got 0
I20260925 13:57:12.432394 21580 catalog_manager.cc:5938] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291) does not have the latest schema for tablet 3d59d9b6dfa5474f82e73e07b25b056e (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 1 got 0
I20260925 13:57:12.462222 22212 tablet_bootstrap.cc:654] T 771c16f54fcd4cea9d38bc3834b043f1 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:12.363075 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "524f19a326ad4dd78353d5c4dab8bcc1" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:12.375319 21937 tablet_service.cc:1558] Processing DeleteTablet for tablet 524f19a326ad4dd78353d5c4dab8bcc1 with delete_type TABLET_DATA_DELETED (replaced by tablet 0d216948345a4db488fd0a9ff367bfd3) from {username='slave'} at 127.0.0.1:55588
W20260925 13:57:12.471490 22043 leader_election.cc:343] T 524f19a326ad4dd78353d5c4dab8bcc1 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:57:12.472007 22043 leader_election.cc:304] T 524f19a326ad4dd78353d5c4dab8bcc1 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
W20260925 13:57:12.472549 22043 raft_consensus.cc:2686] T 524f19a326ad4dd78353d5c4dab8bcc1 P ee02f77594864349982492862d668a36: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260925 13:57:12.497803 22215 ts_tablet_manager.cc:1929] T 524f19a326ad4dd78353d5c4dab8bcc1 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:12.498719 22215 log.cc:1199] T 524f19a326ad4dd78353d5c4dab8bcc1 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/524f19a326ad4dd78353d5c4dab8bcc1
I20260925 13:57:12.524027 22215 ts_tablet_manager.cc:1950] T 524f19a326ad4dd78353d5c4dab8bcc1 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:12.527086 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet 524f19a326ad4dd78353d5c4dab8bcc1 (table default.table_492c567ddda14e77851270f34471730e [id=754c40502e614c5bb18230c852aa0016]) successfully deleted
I20260925 13:57:12.533768 21585 catalog_manager.cc:6391] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb] alter complete (version 1)
I20260925 13:57:12.544518 22212 tablet_bootstrap.cc:492] T 771c16f54fcd4cea9d38bc3834b043f1 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:12.544582 21776 tablet_service.cc:1511] Processing CreateTablet for tablet 0d216948345a4db488fd0a9ff367bfd3 (DEFAULT_TABLE table=default.table_492c567ddda14e77851270f34471730e [id=754c40502e614c5bb18230c852aa0016]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:12.544953 22212 ts_tablet_manager.cc:1403] T 771c16f54fcd4cea9d38bc3834b043f1 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.234s user 0.009s sys 0.000s
I20260925 13:57:12.545571 21776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d216948345a4db488fd0a9ff367bfd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:12.551302 22322 raft_consensus.cc:493] T 771c16f54fcd4cea9d38bc3834b043f1 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:12.551777 22322 raft_consensus.cc:515] T 771c16f54fcd4cea9d38bc3834b043f1 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:12.552301 22212 raft_consensus.cc:359] T 771c16f54fcd4cea9d38bc3834b043f1 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:12.552945 22212 raft_consensus.cc:385] T 771c16f54fcd4cea9d38bc3834b043f1 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:12.553260 22212 raft_consensus.cc:740] T 771c16f54fcd4cea9d38bc3834b043f1 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:12.553752 22322 leader_election.cc:290] T 771c16f54fcd4cea9d38bc3834b043f1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:12.555621 22105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "771c16f54fcd4cea9d38bc3834b043f1" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
W20260925 13:57:12.555891 21575 catalog_manager.cc:5030] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): delete failed for tablet 524f19a326ad4dd78353d5c4dab8bcc1 because the tablet was not found. No further retry: Not found: Tablet not found: 524f19a326ad4dd78353d5c4dab8bcc1
I20260925 13:57:12.556926 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "771c16f54fcd4cea9d38bc3834b043f1" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:12.557544 21960 raft_consensus.cc:2468] T 771c16f54fcd4cea9d38bc3834b043f1 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:12.558923 21732 leader_election.cc:304] T 771c16f54fcd4cea9d38bc3834b043f1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:12.559808 22322 raft_consensus.cc:2804] T 771c16f54fcd4cea9d38bc3834b043f1 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:12.560123 22322 raft_consensus.cc:493] T 771c16f54fcd4cea9d38bc3834b043f1 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:12.560434 22322 raft_consensus.cc:3060] T 771c16f54fcd4cea9d38bc3834b043f1 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:12.560590 21585 master_service.cc:960] ReplaceTablet: received request to replace tablet 771c16f54fcd4cea9d38bc3834b043f1 from {username='slave'} at 127.0.0.1:35994
W20260925 13:57:12.563944 21572 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet cd30e39a98fd4d7fbe793bdbc02eea13 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet cd30e39a98fd4d7fbe793bdbc02eea13 consensus state has no leader
I20260925 13:57:12.570417 22214 ts_tablet_manager.cc:1929] T 524f19a326ad4dd78353d5c4dab8bcc1 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:12.570840 22214 log.cc:1199] T 524f19a326ad4dd78353d5c4dab8bcc1 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/524f19a326ad4dd78353d5c4dab8bcc1
I20260925 13:57:12.571528 22214 ts_tablet_manager.cc:1950] T 524f19a326ad4dd78353d5c4dab8bcc1 P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:12.574241 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet 524f19a326ad4dd78353d5c4dab8bcc1 (table default.table_492c567ddda14e77851270f34471730e [id=754c40502e614c5bb18230c852aa0016]) successfully deleted
I20260925 13:57:12.576285 22212 consensus_queue.cc:260] T 771c16f54fcd4cea9d38bc3834b043f1 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:12.577200 22105 raft_consensus.cc:2468] T 771c16f54fcd4cea9d38bc3834b043f1 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:12.578200 21937 tablet_service.cc:1511] Processing CreateTablet for tablet 0d216948345a4db488fd0a9ff367bfd3 (DEFAULT_TABLE table=default.table_492c567ddda14e77851270f34471730e [id=754c40502e614c5bb18230c852aa0016]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:12.579783 21937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d216948345a4db488fd0a9ff367bfd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:12.603611 22212 ts_tablet_manager.cc:1434] T 771c16f54fcd4cea9d38bc3834b043f1 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.058s user 0.008s sys 0.000s
I20260925 13:57:12.609475 21585 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet 771c16f54fcd4cea9d38bc3834b043f1
I20260925 13:57:12.611922 21777 tablet_service.cc:1558] Processing DeleteTablet for tablet 771c16f54fcd4cea9d38bc3834b043f1 with delete_type TABLET_DATA_DELETED (replaced by tablet 25912fa60e154d7ebd183adfda942dcb) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:12.612665 21938 tablet_service.cc:1558] Processing DeleteTablet for tablet 771c16f54fcd4cea9d38bc3834b043f1 with delete_type TABLET_DATA_DELETED (replaced by tablet 25912fa60e154d7ebd183adfda942dcb) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:12.613440 22213 tablet_replica.cc:333] T 771c16f54fcd4cea9d38bc3834b043f1 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:12.614631 22215 tablet_replica.cc:333] T 771c16f54fcd4cea9d38bc3834b043f1 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:12.615823 22215 raft_consensus.cc:2243] T 771c16f54fcd4cea9d38bc3834b043f1 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:12.616286 22215 raft_consensus.cc:2272] T 771c16f54fcd4cea9d38bc3834b043f1 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:12.619663 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet 771c16f54fcd4cea9d38bc3834b043f1 with delete_type TABLET_DATA_DELETED (replaced by tablet 25912fa60e154d7ebd183adfda942dcb) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:12.623071 22215 ts_tablet_manager.cc:1916] T 771c16f54fcd4cea9d38bc3834b043f1 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:12.625429 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet 771c16f54fcd4cea9d38bc3834b043f1 with delete_type TABLET_DATA_DELETED (replaced by tablet 25912fa60e154d7ebd183adfda942dcb) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:12.627750 21777 tablet_service.cc:1511] Processing CreateTablet for tablet 5888d8f93cec4450847ccfe4854e4012 (DEFAULT_TABLE table=default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:12.629253 21777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5888d8f93cec4450847ccfe4854e4012. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:12.571029 22322 raft_consensus.cc:515] T 771c16f54fcd4cea9d38bc3834b043f1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:12.632453 22083 tablet_service.cc:1511] Processing CreateTablet for tablet 0d216948345a4db488fd0a9ff367bfd3 (DEFAULT_TABLE table=default.table_492c567ddda14e77851270f34471730e [id=754c40502e614c5bb18230c852aa0016]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:12.633455 22083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d216948345a4db488fd0a9ff367bfd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:12.634047 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "771c16f54fcd4cea9d38bc3834b043f1" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
W20260925 13:57:12.635632 21732 leader_election.cc:343] T 771c16f54fcd4cea9d38bc3834b043f1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Illegal state: Tablet not RUNNING: STOPPED
I20260925 13:57:12.640582 22213 raft_consensus.cc:2243] T 771c16f54fcd4cea9d38bc3834b043f1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:12.641402 22105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "771c16f54fcd4cea9d38bc3834b043f1" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:12.642162 22105 raft_consensus.cc:3060] T 771c16f54fcd4cea9d38bc3834b043f1 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:12.642875 22213 raft_consensus.cc:2272] T 771c16f54fcd4cea9d38bc3834b043f1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:12.649539 22105 raft_consensus.cc:2468] T 771c16f54fcd4cea9d38bc3834b043f1 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:12.650976 21732 leader_election.cc:304] T 771c16f54fcd4cea9d38bc3834b043f1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2, ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee
W20260925 13:57:12.651695 21732 raft_consensus.cc:2686] T 771c16f54fcd4cea9d38bc3834b043f1 P 5e2731975dbf433aade4a859df9699e2: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260925 13:57:12.652310 21585 catalog_manager.cc:6934] ReplaceTablet: tablet 771c16f54fcd4cea9d38bc3834b043f1 deleted and replaced by tablet 25912fa60e154d7ebd183adfda942dcb
I20260925 13:57:12.655869 21938 tablet_service.cc:1511] Processing CreateTablet for tablet 5888d8f93cec4450847ccfe4854e4012 (DEFAULT_TABLE table=default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:12.658093 22087 tablet_service.cc:1511] Processing CreateTablet for tablet 5888d8f93cec4450847ccfe4854e4012 (DEFAULT_TABLE table=default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:12.658190 21585 master_service.cc:960] ReplaceTablet: received request to replace tablet 771c16f54fcd4cea9d38bc3834b043f1 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:12.659482 22087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5888d8f93cec4450847ccfe4854e4012. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:12.670789 22214 tablet_replica.cc:333] T 771c16f54fcd4cea9d38bc3834b043f1 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:12.672523 22214 raft_consensus.cc:2243] T 771c16f54fcd4cea9d38bc3834b043f1 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:12.673552 22214 raft_consensus.cc:2272] T 771c16f54fcd4cea9d38bc3834b043f1 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:12.688630 21938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5888d8f93cec4450847ccfe4854e4012. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:12.689476 22322 leader_election.cc:290] T 771c16f54fcd4cea9d38bc3834b043f1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Requested vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:12.693397 22213 ts_tablet_manager.cc:1916] T 771c16f54fcd4cea9d38bc3834b043f1 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:12.712049 22293 tablet_bootstrap.cc:492] T 0d216948345a4db488fd0a9ff367bfd3 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:12.721868 22211 tablet_bootstrap.cc:492] T 0d216948345a4db488fd0a9ff367bfd3 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:12.726114 22212 tablet_bootstrap.cc:492] T 5888d8f93cec4450847ccfe4854e4012 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:12.727924 22211 tablet_bootstrap.cc:654] T 0d216948345a4db488fd0a9ff367bfd3 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:12.746520 22211 tablet_bootstrap.cc:492] T 0d216948345a4db488fd0a9ff367bfd3 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:12.735846 22293 tablet_bootstrap.cc:654] T 0d216948345a4db488fd0a9ff367bfd3 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:12.747571 22211 ts_tablet_manager.cc:1403] T 0d216948345a4db488fd0a9ff367bfd3 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.026s user 0.004s sys 0.011s
I20260925 13:57:12.745528 22213 ts_tablet_manager.cc:1929] T 771c16f54fcd4cea9d38bc3834b043f1 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:12.750531 22213 log.cc:1199] T 771c16f54fcd4cea9d38bc3834b043f1 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/771c16f54fcd4cea9d38bc3834b043f1
I20260925 13:57:12.750449 22211 raft_consensus.cc:359] T 0d216948345a4db488fd0a9ff367bfd3 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:12.751183 22213 ts_tablet_manager.cc:1950] T 771c16f54fcd4cea9d38bc3834b043f1 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:12.751125 22211 raft_consensus.cc:385] T 0d216948345a4db488fd0a9ff367bfd3 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:12.735741 22214 ts_tablet_manager.cc:1916] T 771c16f54fcd4cea9d38bc3834b043f1 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:12.758726 22211 raft_consensus.cc:740] T 0d216948345a4db488fd0a9ff367bfd3 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:12.759920 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet 771c16f54fcd4cea9d38bc3834b043f1 (table default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]) successfully deleted
I20260925 13:57:12.761993 22087 tablet_service.cc:1511] Processing CreateTablet for tablet 25912fa60e154d7ebd183adfda942dcb (DEFAULT_TABLE table=default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:12.763252 22087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25912fa60e154d7ebd183adfda942dcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:12.762954 22211 consensus_queue.cc:260] T 0d216948345a4db488fd0a9ff367bfd3 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:12.764155 21937 tablet_service.cc:1511] Processing CreateTablet for tablet 25912fa60e154d7ebd183adfda942dcb (DEFAULT_TABLE table=default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:12.765270 21937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25912fa60e154d7ebd183adfda942dcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:12.769973 22224 raft_consensus.cc:493] T 0d216948345a4db488fd0a9ff367bfd3 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:12.770433 22224 raft_consensus.cc:515] T 0d216948345a4db488fd0a9ff367bfd3 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:12.772471 22224 leader_election.cc:290] T 0d216948345a4db488fd0a9ff367bfd3 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:12.775071 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d216948345a4db488fd0a9ff367bfd3" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2" is_pre_election: true
W20260925 13:57:12.776777 21893 leader_election.cc:343] T 0d216948345a4db488fd0a9ff367bfd3 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:57:12.778853 22211 ts_tablet_manager.cc:1434] T 0d216948345a4db488fd0a9ff367bfd3 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.031s user 0.004s sys 0.003s
W20260925 13:57:12.779681 21896 leader_election.cc:343] T 0d216948345a4db488fd0a9ff367bfd3 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Not found: Tablet not found: 0d216948345a4db488fd0a9ff367bfd3
I20260925 13:57:12.780109 21896 leader_election.cc:304] T 0d216948345a4db488fd0a9ff367bfd3 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee; no voters: 5e2731975dbf433aade4a859df9699e2, ee02f77594864349982492862d668a36
I20260925 13:57:12.781255 22224 raft_consensus.cc:2749] T 0d216948345a4db488fd0a9ff367bfd3 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:57:12.788152 22212 tablet_bootstrap.cc:654] T 5888d8f93cec4450847ccfe4854e4012 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:12.798588 21776 tablet_service.cc:1511] Processing CreateTablet for tablet 25912fa60e154d7ebd183adfda942dcb (DEFAULT_TABLE table=default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 13:57:12.801836 21641 hms_client.cc:355] Time spent drop HMS table: real 2.176s user 0.000s sys 0.000s
I20260925 13:57:12.802294 21587 catalog_manager.cc:2770] deleted HMS catalog entry for table default.table_492c567ddda14e77851270f34471730e [id=754c40502e614c5bb18230c852aa0016]
I20260925 13:57:12.808521 21776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25912fa60e154d7ebd183adfda942dcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:12.838420 22293 tablet_bootstrap.cc:492] T 0d216948345a4db488fd0a9ff367bfd3 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:12.839021 22293 ts_tablet_manager.cc:1403] T 0d216948345a4db488fd0a9ff367bfd3 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.127s user 0.030s sys 0.004s
I20260925 13:57:12.848378 22293 raft_consensus.cc:359] T 0d216948345a4db488fd0a9ff367bfd3 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:12.849467 22293 raft_consensus.cc:385] T 0d216948345a4db488fd0a9ff367bfd3 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:12.849762 22293 raft_consensus.cc:740] T 0d216948345a4db488fd0a9ff367bfd3 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:12.860275 21580 master_service.cc:960] ReplaceTablet: received request to replace tablet d267def3f0864435a0d2c40fd7052eab from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:12.865381 22293 consensus_queue.cc:260] T 0d216948345a4db488fd0a9ff367bfd3 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:12.872752 22293 ts_tablet_manager.cc:1434] T 0d216948345a4db488fd0a9ff367bfd3 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.033s user 0.010s sys 0.000s
I20260925 13:57:12.874009 22293 tablet_bootstrap.cc:492] T 5888d8f93cec4450847ccfe4854e4012 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
W20260925 13:57:12.874276 21580 catalog_manager.cc:5671] Ignoring report from unknown tablet d267def3f0864435a0d2c40fd7052eab
I20260925 13:57:12.886515 22211 tablet_bootstrap.cc:492] T 5888d8f93cec4450847ccfe4854e4012 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:12.890102 22212 tablet_bootstrap.cc:492] T 5888d8f93cec4450847ccfe4854e4012 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:12.890477 22212 ts_tablet_manager.cc:1403] T 5888d8f93cec4450847ccfe4854e4012 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.165s user 0.010s sys 0.000s
I20260925 13:57:12.892438 22211 tablet_bootstrap.cc:654] T 5888d8f93cec4450847ccfe4854e4012 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:12.902282 22212 raft_consensus.cc:359] T 5888d8f93cec4450847ccfe4854e4012 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:12.902669 22293 tablet_bootstrap.cc:654] T 5888d8f93cec4450847ccfe4854e4012 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:12.902854 22212 raft_consensus.cc:385] T 5888d8f93cec4450847ccfe4854e4012 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:12.903072 22212 raft_consensus.cc:740] T 5888d8f93cec4450847ccfe4854e4012 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:12.905191 22211 tablet_bootstrap.cc:492] T 5888d8f93cec4450847ccfe4854e4012 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:12.905597 22211 ts_tablet_manager.cc:1403] T 5888d8f93cec4450847ccfe4854e4012 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.019s user 0.012s sys 0.000s
I20260925 13:57:12.907423 22215 ts_tablet_manager.cc:1929] T 771c16f54fcd4cea9d38bc3834b043f1 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:12.907804 22215 log.cc:1199] T 771c16f54fcd4cea9d38bc3834b043f1 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/771c16f54fcd4cea9d38bc3834b043f1
I20260925 13:57:12.908429 22215 ts_tablet_manager.cc:1950] T 771c16f54fcd4cea9d38bc3834b043f1 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:12.910883 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet 771c16f54fcd4cea9d38bc3834b043f1 (table default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]) successfully deleted
I20260925 13:57:12.911056 22211 raft_consensus.cc:359] T 5888d8f93cec4450847ccfe4854e4012 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:12.911957 22211 raft_consensus.cc:385] T 5888d8f93cec4450847ccfe4854e4012 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:12.915601 22211 raft_consensus.cc:740] T 5888d8f93cec4450847ccfe4854e4012 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:12.916738 22211 consensus_queue.cc:260] T 5888d8f93cec4450847ccfe4854e4012 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:12.920384 22211 ts_tablet_manager.cc:1434] T 5888d8f93cec4450847ccfe4854e4012 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.015s user 0.005s sys 0.000s
I20260925 13:57:12.926456 21585 master_service.cc:960] ReplaceTablet: received request to replace tablet d267def3f0864435a0d2c40fd7052eab from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:12.926779 22293 tablet_bootstrap.cc:492] T 5888d8f93cec4450847ccfe4854e4012 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:12.927306 22293 ts_tablet_manager.cc:1403] T 5888d8f93cec4450847ccfe4854e4012 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.054s user 0.008s sys 0.002s
I20260925 13:57:12.927208 21580 catalog_manager.cc:5938] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291) does not have the latest schema for tablet 5888d8f93cec4450847ccfe4854e4012 (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]). Expected version 1 got 0
W20260925 13:57:12.929749 21580 catalog_manager.cc:6038] Failed to send AlterTable RPC for tablet 5888d8f93cec4450847ccfe4854e4012 (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 5888d8f93cec4450847ccfe4854e4012 consensus state has no leader
I20260925 13:57:12.934777 21580 catalog_manager.cc:5938] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291) does not have the latest schema for tablet 5888d8f93cec4450847ccfe4854e4012 (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]). Expected version 1 got 0
I20260925 13:57:12.935154 22214 ts_tablet_manager.cc:1929] T 771c16f54fcd4cea9d38bc3834b043f1 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:12.935568 22214 log.cc:1199] T 771c16f54fcd4cea9d38bc3834b043f1 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/771c16f54fcd4cea9d38bc3834b043f1
I20260925 13:57:12.930701 22293 raft_consensus.cc:359] T 5888d8f93cec4450847ccfe4854e4012 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:12.936038 22293 raft_consensus.cc:385] T 5888d8f93cec4450847ccfe4854e4012 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:12.936569 22293 raft_consensus.cc:740] T 5888d8f93cec4450847ccfe4854e4012 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:12.914829 22212 consensus_queue.cc:260] T 5888d8f93cec4450847ccfe4854e4012 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:12.938689 22212 ts_tablet_manager.cc:1434] T 5888d8f93cec4450847ccfe4854e4012 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.048s user 0.004s sys 0.000s
I20260925 13:57:12.939803 22293 consensus_queue.cc:260] T 5888d8f93cec4450847ccfe4854e4012 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:12.947154 22212 tablet_bootstrap.cc:492] T 0d216948345a4db488fd0a9ff367bfd3 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:12.948055 21585 catalog_manager.cc:5938] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873) does not have the latest schema for tablet 5888d8f93cec4450847ccfe4854e4012 (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]). Expected version 1 got 0
I20260925 13:57:12.943034 22293 ts_tablet_manager.cc:1434] T 5888d8f93cec4450847ccfe4854e4012 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.015s user 0.007s sys 0.000s
I20260925 13:57:12.956609 22293 tablet_bootstrap.cc:492] T 25912fa60e154d7ebd183adfda942dcb P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:12.957391 21580 catalog_manager.cc:5938] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881) does not have the latest schema for tablet 5888d8f93cec4450847ccfe4854e4012 (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]). Expected version 1 got 0
I20260925 13:57:12.965747 21585 catalog_manager.cc:5938] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881) does not have the latest schema for tablet 5888d8f93cec4450847ccfe4854e4012 (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]). Expected version 1 got 0
I20260925 13:57:12.979812 22211 tablet_bootstrap.cc:492] T 25912fa60e154d7ebd183adfda942dcb P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:12.980093 22212 tablet_bootstrap.cc:654] T 0d216948345a4db488fd0a9ff367bfd3 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:12.982585 22214 ts_tablet_manager.cc:1950] T 771c16f54fcd4cea9d38bc3834b043f1 P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:12.985086 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet 771c16f54fcd4cea9d38bc3834b043f1 (table default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]) successfully deleted
W20260925 13:57:12.986625 21575 catalog_manager.cc:5030] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): delete failed for tablet 771c16f54fcd4cea9d38bc3834b043f1 because the tablet was not found. No further retry: Not found: Tablet not found: 771c16f54fcd4cea9d38bc3834b043f1
I20260925 13:57:12.987174 22211 tablet_bootstrap.cc:654] T 25912fa60e154d7ebd183adfda942dcb P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:12.992355 22293 tablet_bootstrap.cc:654] T 25912fa60e154d7ebd183adfda942dcb P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:13.010959 22211 tablet_bootstrap.cc:492] T 25912fa60e154d7ebd183adfda942dcb P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:13.011471 22211 ts_tablet_manager.cc:1403] T 25912fa60e154d7ebd183adfda942dcb P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.032s user 0.019s sys 0.008s
I20260925 13:57:13.012770 22212 tablet_bootstrap.cc:492] T 0d216948345a4db488fd0a9ff367bfd3 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:13.013196 22212 ts_tablet_manager.cc:1403] T 0d216948345a4db488fd0a9ff367bfd3 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.066s user 0.011s sys 0.000s
I20260925 13:57:13.014240 22211 raft_consensus.cc:359] T 25912fa60e154d7ebd183adfda942dcb P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.016171 22212 raft_consensus.cc:359] T 0d216948345a4db488fd0a9ff367bfd3 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.018021 22212 raft_consensus.cc:385] T 0d216948345a4db488fd0a9ff367bfd3 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:13.018265 22212 raft_consensus.cc:740] T 0d216948345a4db488fd0a9ff367bfd3 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:13.024633 22212 consensus_queue.cc:260] T 0d216948345a4db488fd0a9ff367bfd3 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.042558 22322 raft_consensus.cc:493] T 0d216948345a4db488fd0a9ff367bfd3 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:13.043004 22322 raft_consensus.cc:515] T 0d216948345a4db488fd0a9ff367bfd3 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.044747 22322 leader_election.cc:290] T 0d216948345a4db488fd0a9ff367bfd3 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:13.045680 22211 raft_consensus.cc:385] T 25912fa60e154d7ebd183adfda942dcb P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:13.045984 22211 raft_consensus.cc:740] T 25912fa60e154d7ebd183adfda942dcb P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:13.046627 22211 consensus_queue.cc:260] T 25912fa60e154d7ebd183adfda942dcb P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.047606 22212 ts_tablet_manager.cc:1434] T 0d216948345a4db488fd0a9ff367bfd3 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.034s user 0.008s sys 0.000s
I20260925 13:57:13.048174 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d216948345a4db488fd0a9ff367bfd3" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:13.048703 21960 raft_consensus.cc:2468] T 0d216948345a4db488fd0a9ff367bfd3 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:13.049846 22105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d216948345a4db488fd0a9ff367bfd3" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:13.050401 22105 raft_consensus.cc:2468] T 0d216948345a4db488fd0a9ff367bfd3 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:13.050668 21732 leader_election.cc:304] T 0d216948345a4db488fd0a9ff367bfd3 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:13.051488 22322 raft_consensus.cc:2804] T 0d216948345a4db488fd0a9ff367bfd3 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:13.051770 22322 raft_consensus.cc:493] T 0d216948345a4db488fd0a9ff367bfd3 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:13.052040 22322 raft_consensus.cc:3060] T 0d216948345a4db488fd0a9ff367bfd3 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:13.058121 22212 tablet_bootstrap.cc:492] T 25912fa60e154d7ebd183adfda942dcb P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:13.058779 22211 ts_tablet_manager.cc:1434] T 25912fa60e154d7ebd183adfda942dcb P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.047s user 0.005s sys 0.000s
I20260925 13:57:13.062592 22322 raft_consensus.cc:515] T 0d216948345a4db488fd0a9ff367bfd3 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.064277 22322 leader_election.cc:290] T 0d216948345a4db488fd0a9ff367bfd3 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Requested vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:13.066316 22293 tablet_bootstrap.cc:492] T 25912fa60e154d7ebd183adfda942dcb P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:13.066681 22293 ts_tablet_manager.cc:1403] T 25912fa60e154d7ebd183adfda942dcb P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.110s user 0.018s sys 0.009s
I20260925 13:57:13.074841 22105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d216948345a4db488fd0a9ff367bfd3" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:13.075614 22105 raft_consensus.cc:3060] T 0d216948345a4db488fd0a9ff367bfd3 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:13.068820 22293 raft_consensus.cc:359] T 25912fa60e154d7ebd183adfda942dcb P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.076938 22293 raft_consensus.cc:385] T 25912fa60e154d7ebd183adfda942dcb P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:13.077180 22293 raft_consensus.cc:740] T 25912fa60e154d7ebd183adfda942dcb P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:13.077731 22293 consensus_queue.cc:260] T 25912fa60e154d7ebd183adfda942dcb P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.083860 22293 ts_tablet_manager.cc:1434] T 25912fa60e154d7ebd183adfda942dcb P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.017s user 0.006s sys 0.000s
I20260925 13:57:13.089866 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d216948345a4db488fd0a9ff367bfd3" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
I20260925 13:57:13.090433 21960 raft_consensus.cc:3060] T 0d216948345a4db488fd0a9ff367bfd3 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:13.110392 22212 tablet_bootstrap.cc:654] T 25912fa60e154d7ebd183adfda942dcb P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:13.110896 21580 master_service.cc:960] ReplaceTablet: received request to replace tablet d267def3f0864435a0d2c40fd7052eab from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:13.117532 22105 raft_consensus.cc:2468] T 0d216948345a4db488fd0a9ff367bfd3 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:13.129941 21960 raft_consensus.cc:2468] T 0d216948345a4db488fd0a9ff367bfd3 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:13.136309 21580 master_service.cc:960] ReplaceTablet: received request to replace tablet 25912fa60e154d7ebd183adfda942dcb from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:13.142431 21732 leader_election.cc:304] T 0d216948345a4db488fd0a9ff367bfd3 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:13.141675 22212 tablet_bootstrap.cc:492] T 25912fa60e154d7ebd183adfda942dcb P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:13.144760 22322 raft_consensus.cc:2804] T 0d216948345a4db488fd0a9ff367bfd3 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:13.145006 22212 ts_tablet_manager.cc:1403] T 25912fa60e154d7ebd183adfda942dcb P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.087s user 0.010s sys 0.000s
I20260925 13:57:13.152428 22212 raft_consensus.cc:359] T 25912fa60e154d7ebd183adfda942dcb P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.153074 22212 raft_consensus.cc:385] T 25912fa60e154d7ebd183adfda942dcb P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:13.153245 21580 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet 25912fa60e154d7ebd183adfda942dcb
I20260925 13:57:13.153321 22212 raft_consensus.cc:740] T 25912fa60e154d7ebd183adfda942dcb P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:13.154423 21580 catalog_manager.cc:6934] ReplaceTablet: tablet 25912fa60e154d7ebd183adfda942dcb deleted and replaced by tablet 7b7143a9509c4df7b955285e3508c4fe
I20260925 13:57:13.159776 22087 tablet_service.cc:1558] Processing DeleteTablet for tablet 25912fa60e154d7ebd183adfda942dcb with delete_type TABLET_DATA_DELETED (replaced by tablet 7b7143a9509c4df7b955285e3508c4fe) from {username='slave'} at 127.0.0.1:50052
W20260925 13:57:13.161337 21575 catalog_manager.cc:5043] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): delete failed for tablet 25912fa60e154d7ebd183adfda942dcb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 25912fa60e154d7ebd183adfda942dcb already in progress: creating tablet
I20260925 13:57:13.161693 21776 tablet_service.cc:1558] Processing DeleteTablet for tablet 25912fa60e154d7ebd183adfda942dcb with delete_type TABLET_DATA_DELETED (replaced by tablet 7b7143a9509c4df7b955285e3508c4fe) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:13.162302 22213 tablet_replica.cc:333] T 25912fa60e154d7ebd183adfda942dcb P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:13.162930 22213 raft_consensus.cc:2243] T 25912fa60e154d7ebd183adfda942dcb P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:13.163345 22213 raft_consensus.cc:2272] T 25912fa60e154d7ebd183adfda942dcb P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:13.165445 22213 ts_tablet_manager.cc:1916] T 25912fa60e154d7ebd183adfda942dcb P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:13.179011 22322 raft_consensus.cc:697] T 0d216948345a4db488fd0a9ff367bfd3 P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Becoming Leader. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Running, Role: LEADER
I20260925 13:57:13.179929 22322 consensus_queue.cc:237] T 0d216948345a4db488fd0a9ff367bfd3 P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.185178 22213 ts_tablet_manager.cc:1929] T 25912fa60e154d7ebd183adfda942dcb P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:13.163153 22212 consensus_queue.cc:260] T 25912fa60e154d7ebd183adfda942dcb P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.185245 22087 tablet_service.cc:1511] Processing CreateTablet for tablet 7b7143a9509c4df7b955285e3508c4fe (DEFAULT_TABLE table=default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:13.192553 22212 ts_tablet_manager.cc:1434] T 25912fa60e154d7ebd183adfda942dcb P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.047s user 0.005s sys 0.000s
I20260925 13:57:13.196677 22087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b7143a9509c4df7b955285e3508c4fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:13.186870 21776 tablet_service.cc:1511] Processing CreateTablet for tablet 7b7143a9509c4df7b955285e3508c4fe (DEFAULT_TABLE table=default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:13.197640 21776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b7143a9509c4df7b955285e3508c4fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:13.207545 22213 log.cc:1199] T 25912fa60e154d7ebd183adfda942dcb P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/25912fa60e154d7ebd183adfda942dcb
I20260925 13:57:13.207902 21583 catalog_manager.cc:2213] created HMS catalog entry for table [id=7fb6c448c78344719796980b4be1cda9]
I20260925 13:57:13.211124 22213 ts_tablet_manager.cc:1950] T 25912fa60e154d7ebd183adfda942dcb P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:13.215623 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet 25912fa60e154d7ebd183adfda942dcb with delete_type TABLET_DATA_DELETED (replaced by tablet 7b7143a9509c4df7b955285e3508c4fe) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:13.216305 22214 tablet_replica.cc:333] T 25912fa60e154d7ebd183adfda942dcb P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:13.216967 22214 raft_consensus.cc:2243] T 25912fa60e154d7ebd183adfda942dcb P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:13.217382 22214 raft_consensus.cc:2272] T 25912fa60e154d7ebd183adfda942dcb P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:13.218251 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet 25912fa60e154d7ebd183adfda942dcb (table default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]) successfully deleted
I20260925 13:57:13.222987 22214 ts_tablet_manager.cc:1916] T 25912fa60e154d7ebd183adfda942dcb P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:13.229581 22293 tablet_bootstrap.cc:492] T 7b7143a9509c4df7b955285e3508c4fe P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:13.227213 21583 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35994 (request call id 10) took 4351 ms. Trace:
I20260925 13:57:13.236958 22293 tablet_bootstrap.cc:654] T 7b7143a9509c4df7b955285e3508c4fe P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:13.230029 21583 rpcz_store.cc:276] 0925 13:57:08.875408 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:08.875645 (+ 237us) service_pool.cc:224] Handling call
0925 13:57:10.685728 (+1810083us) catalog_manager.cc:2140] Acquired catalog manager lock
0925 13:57:10.687426 (+ 1698us) catalog_manager.cc:2186] Created new table and tablet info
0925 13:57:13.207848 (+2520422us) catalog_manager.cc:2212] Created new table in HMS catalog
0925 13:57:13.226393 (+ 18545us) catalog_manager.cc:2246] Wrote table and tablets to system table
0925 13:57:13.226814 (+ 421us) catalog_manager.cc:2280] Inserted table and tablets into CatalogManager maps
0925 13:57:13.227148 (+ 334us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:57:13.209625 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:57:13.210032 (+ 407us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:57:13.210056 (+ 24us) write_op.cc:435] Acquired schema lock
0925 13:57:13.210069 (+ 13us) tablet.cc:662] Decoding operations
0925 13:57:13.210444 (+ 375us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:57:13.210481 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0925 13:57:13.210502 (+ 21us) tablet.cc:685] Acquiring locks for 2 operations
0925 13:57:13.210760 (+ 258us) tablet.cc:701] Row locks acquired
0925 13:57:13.210776 (+ 16us) write_op.cc:260] PREPARE: finished
0925 13:57:13.210887 (+ 111us) write_op.cc:270] Start()
0925 13:57:13.210964 (+ 77us) write_op.cc:276] Timestamp: P: 1790344633210843 usec, L: 0
0925 13:57:13.210978 (+ 14us) op_driver.cc:348] REPLICATION: starting
0925 13:57:13.211481 (+ 503us) log.cc:844] Serialized 432 byte log entry
0925 13:57:13.223118 (+ 11637us) op_driver.cc:464] REPLICATION: finished
0925 13:57:13.223566 (+ 448us) write_op.cc:301] APPLY: starting
0925 13:57:13.223636 (+ 70us) tablet.cc:1394] starting BulkCheckPresence
0925 13:57:13.223764 (+ 128us) tablet.cc:1397] finished BulkCheckPresence
0925 13:57:13.223846 (+ 82us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:57:13.224180 (+ 334us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:57:13.224303 (+ 123us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 13:57:13.224341 (+ 38us) write_op.cc:312] APPLY: finished
0925 13:57:13.224517 (+ 176us) log.cc:844] Serialized 28 byte log entry
0925 13:57:13.224671 (+ 154us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:13.224736 (+ 65us) write_op.cc:454] Released schema lock
0925 13:57:13.224772 (+ 36us) write_op.cc:341] FINISH: Updating metrics
0925 13:57:13.225007 (+ 235us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:13.225044 (+ 37us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":2114434,"Hive Metastore.run_cpu_time_us":1317,"Hive Metastore.run_wall_time_us":405076,"child_traces":[["op",{"apply.queue_time_us":270,"apply.run_cpu_time_us":1390,"apply.run_wall_time_us":1390,"num_ops":2,"prepare.queue_time_us":109,"prepare.run_cpu_time_us":2209,"prepare.run_wall_time_us":2209,"raft.queue_time_us":505,"raft.run_cpu_time_us":669,"raft.run_wall_time_us":668,"replication_time_us":12050,"spinlock_wait_cycles":3456}]]}
I20260925 13:57:13.257998 21580 catalog_manager.cc:5838] T 0d216948345a4db488fd0a9ff367bfd3 P 5e2731975dbf433aade4a859df9699e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e2731975dbf433aade4a859df9699e2 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "5e2731975dbf433aade4a859df9699e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: UNKNOWN } } }
I20260925 13:57:13.259682 22226 raft_consensus.cc:493] T 5888d8f93cec4450847ccfe4854e4012 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:13.260212 21937 tablet_service.cc:1558] Processing DeleteTablet for tablet 25912fa60e154d7ebd183adfda942dcb with delete_type TABLET_DATA_DELETED (replaced by tablet 7b7143a9509c4df7b955285e3508c4fe) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:13.260275 22226 raft_consensus.cc:515] T 5888d8f93cec4450847ccfe4854e4012 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.261704 21938 tablet_service.cc:1511] Processing CreateTablet for tablet 7b7143a9509c4df7b955285e3508c4fe (DEFAULT_TABLE table=default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:13.262182 22226 leader_election.cc:290] T 5888d8f93cec4450847ccfe4854e4012 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:13.264058 22215 tablet_replica.cc:333] T 25912fa60e154d7ebd183adfda942dcb P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:13.264271 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5888d8f93cec4450847ccfe4854e4012" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2" is_pre_election: true
I20260925 13:57:13.264920 21797 raft_consensus.cc:2468] T 5888d8f93cec4450847ccfe4854e4012 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee02f77594864349982492862d668a36 in term 0.
I20260925 13:57:13.265806 22215 raft_consensus.cc:2243] T 25912fa60e154d7ebd183adfda942dcb P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:13.266741 22215 raft_consensus.cc:2272] T 25912fa60e154d7ebd183adfda942dcb P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:13.266705 22040 leader_election.cc:304] T 5888d8f93cec4450847ccfe4854e4012 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:13.268747 21938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b7143a9509c4df7b955285e3508c4fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:13.269157 22226 raft_consensus.cc:2804] T 5888d8f93cec4450847ccfe4854e4012 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:13.269481 22226 raft_consensus.cc:493] T 5888d8f93cec4450847ccfe4854e4012 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:13.269797 22226 raft_consensus.cc:3060] T 5888d8f93cec4450847ccfe4854e4012 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:13.268777 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5888d8f93cec4450847ccfe4854e4012" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:13.272199 22215 ts_tablet_manager.cc:1916] T 25912fa60e154d7ebd183adfda942dcb P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:13.272348 21960 raft_consensus.cc:2468] T 5888d8f93cec4450847ccfe4854e4012 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee02f77594864349982492862d668a36 in term 0.
I20260925 13:57:13.285372 22226 raft_consensus.cc:515] T 5888d8f93cec4450847ccfe4854e4012 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.289297 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5888d8f93cec4450847ccfe4854e4012" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2"
I20260925 13:57:13.290088 21797 raft_consensus.cc:3060] T 5888d8f93cec4450847ccfe4854e4012 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:13.293298 22293 tablet_bootstrap.cc:492] T 7b7143a9509c4df7b955285e3508c4fe P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:13.298050 22293 ts_tablet_manager.cc:1403] T 7b7143a9509c4df7b955285e3508c4fe P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.069s user 0.003s sys 0.010s
I20260925 13:57:13.304077 21937 tablet_service.cc:1511] Processing CreateTablet for tablet d595ab2f7dc74fdfbf3546e477685a01 (DEFAULT_TABLE table=default.table_8b5db7d66475449d9772db568c9ded56 [id=7fb6c448c78344719796980b4be1cda9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:13.305076 21937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d595ab2f7dc74fdfbf3546e477685a01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:13.304833 21776 tablet_service.cc:1511] Processing CreateTablet for tablet d595ab2f7dc74fdfbf3546e477685a01 (DEFAULT_TABLE table=default.table_8b5db7d66475449d9772db568c9ded56 [id=7fb6c448c78344719796980b4be1cda9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:13.305747 21776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d595ab2f7dc74fdfbf3546e477685a01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:13.309182 22083 tablet_service.cc:1511] Processing CreateTablet for tablet d595ab2f7dc74fdfbf3546e477685a01 (DEFAULT_TABLE table=default.table_8b5db7d66475449d9772db568c9ded56 [id=7fb6c448c78344719796980b4be1cda9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:13.310119 22083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d595ab2f7dc74fdfbf3546e477685a01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:13.318064 21797 raft_consensus.cc:2468] T 5888d8f93cec4450847ccfe4854e4012 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee02f77594864349982492862d668a36 in term 1.
I20260925 13:57:13.321847 22040 leader_election.cc:304] T 5888d8f93cec4450847ccfe4854e4012 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:13.334664 22215 ts_tablet_manager.cc:1929] T 25912fa60e154d7ebd183adfda942dcb P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:13.341373 22293 raft_consensus.cc:359] T 7b7143a9509c4df7b955285e3508c4fe P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.342317 22293 raft_consensus.cc:385] T 7b7143a9509c4df7b955285e3508c4fe P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:13.342602 22293 raft_consensus.cc:740] T 7b7143a9509c4df7b955285e3508c4fe P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:13.342139 21583 master_service.cc:960] ReplaceTablet: received request to replace tablet 3d59d9b6dfa5474f82e73e07b25b056e from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:13.345124 22214 ts_tablet_manager.cc:1929] T 25912fa60e154d7ebd183adfda942dcb P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:13.345490 22214 log.cc:1199] T 25912fa60e154d7ebd183adfda942dcb P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/25912fa60e154d7ebd183adfda942dcb
I20260925 13:57:13.347944 22226 leader_election.cc:290] T 5888d8f93cec4450847ccfe4854e4012 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 election: Requested vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:13.349924 22226 raft_consensus.cc:2804] T 5888d8f93cec4450847ccfe4854e4012 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:13.347837 22293 consensus_queue.cc:260] T 7b7143a9509c4df7b955285e3508c4fe P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.340618 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5888d8f93cec4450847ccfe4854e4012" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
I20260925 13:57:13.350854 22212 tablet_bootstrap.cc:492] T 7b7143a9509c4df7b955285e3508c4fe P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:13.350805 21960 raft_consensus.cc:3060] T 5888d8f93cec4450847ccfe4854e4012 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:13.351166 22226 raft_consensus.cc:697] T 5888d8f93cec4450847ccfe4854e4012 P ee02f77594864349982492862d668a36 [term 1 LEADER]: Becoming Leader. State: Replica: ee02f77594864349982492862d668a36, State: Running, Role: LEADER
I20260925 13:57:13.352008 22226 consensus_queue.cc:237] T 5888d8f93cec4450847ccfe4854e4012 P ee02f77594864349982492862d668a36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.348634 22211 tablet_bootstrap.cc:492] T 7b7143a9509c4df7b955285e3508c4fe P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:13.359239 22212 tablet_bootstrap.cc:654] T 7b7143a9509c4df7b955285e3508c4fe P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:13.346120 22214 ts_tablet_manager.cc:1950] T 25912fa60e154d7ebd183adfda942dcb P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:13.365461 21583 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet 3d59d9b6dfa5474f82e73e07b25b056e
I20260925 13:57:13.367439 21583 catalog_manager.cc:6934] ReplaceTablet: tablet 3d59d9b6dfa5474f82e73e07b25b056e deleted and replaced by tablet 7075ed41f73349aaa7ac85d29950b494
I20260925 13:57:13.368175 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet 25912fa60e154d7ebd183adfda942dcb (table default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]) successfully deleted
I20260925 13:57:13.370023 22087 tablet_service.cc:1558] Processing DeleteTablet for tablet 3d59d9b6dfa5474f82e73e07b25b056e with delete_type TABLET_DATA_DELETED (replaced by tablet 7075ed41f73349aaa7ac85d29950b494) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:13.371922 22211 tablet_bootstrap.cc:654] T 7b7143a9509c4df7b955285e3508c4fe P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:13.373553 22214 tablet_replica.cc:333] T 3d59d9b6dfa5474f82e73e07b25b056e P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:13.374253 22214 raft_consensus.cc:2243] T 3d59d9b6dfa5474f82e73e07b25b056e P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:13.380211 22214 raft_consensus.cc:2272] T 3d59d9b6dfa5474f82e73e07b25b056e P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:13.384137 21583 master_service.cc:960] ReplaceTablet: received request to replace tablet 7b7143a9509c4df7b955285e3508c4fe from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:13.392422 21960 raft_consensus.cc:2468] T 5888d8f93cec4450847ccfe4854e4012 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee02f77594864349982492862d668a36 in term 1.
W20260925 13:57:13.384864 22040 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.193:39881, user_credentials={real_user=slave}} blocked reactor thread for 63098us
I20260925 13:57:13.394702 22215 log.cc:1199] T 25912fa60e154d7ebd183adfda942dcb P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/25912fa60e154d7ebd183adfda942dcb
I20260925 13:57:13.395509 22215 ts_tablet_manager.cc:1950] T 25912fa60e154d7ebd183adfda942dcb P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:13.392364 22212 tablet_bootstrap.cc:492] T 7b7143a9509c4df7b955285e3508c4fe P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:13.397123 22212 ts_tablet_manager.cc:1403] T 7b7143a9509c4df7b955285e3508c4fe P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.046s user 0.011s sys 0.002s
I20260925 13:57:13.397970 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet 25912fa60e154d7ebd183adfda942dcb (table default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]) successfully deleted
I20260925 13:57:13.399689 22212 raft_consensus.cc:359] T 7b7143a9509c4df7b955285e3508c4fe P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.400295 22212 raft_consensus.cc:385] T 7b7143a9509c4df7b955285e3508c4fe P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:13.400152 22322 raft_consensus.cc:493] T 7b7143a9509c4df7b955285e3508c4fe P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:13.400519 22212 raft_consensus.cc:740] T 7b7143a9509c4df7b955285e3508c4fe P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:13.400759 22322 raft_consensus.cc:515] T 7b7143a9509c4df7b955285e3508c4fe P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.402657 22322 leader_election.cc:290] T 7b7143a9509c4df7b955285e3508c4fe P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:13.405447 22293 ts_tablet_manager.cc:1434] T 7b7143a9509c4df7b955285e3508c4fe P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.107s user 0.004s sys 0.001s
I20260925 13:57:13.406270 22293 tablet_bootstrap.cc:492] T d595ab2f7dc74fdfbf3546e477685a01 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:13.406347 21776 tablet_service.cc:1558] Processing DeleteTablet for tablet 3d59d9b6dfa5474f82e73e07b25b056e with delete_type TABLET_DATA_DELETED (replaced by tablet 7075ed41f73349aaa7ac85d29950b494) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:13.407279 22213 tablet_replica.cc:333] T 3d59d9b6dfa5474f82e73e07b25b056e P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:13.407145 22105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b7143a9509c4df7b955285e3508c4fe" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:13.407919 22213 raft_consensus.cc:2243] T 3d59d9b6dfa5474f82e73e07b25b056e P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:13.408684 22213 raft_consensus.cc:2272] T 3d59d9b6dfa5474f82e73e07b25b056e P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:13.409541 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b7143a9509c4df7b955285e3508c4fe" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
W20260925 13:57:13.410772 21732 leader_election.cc:343] T 7b7143a9509c4df7b955285e3508c4fe P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:57:13.412415 22213 ts_tablet_manager.cc:1916] T 3d59d9b6dfa5474f82e73e07b25b056e P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:13.412688 22212 consensus_queue.cc:260] T 7b7143a9509c4df7b955285e3508c4fe P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.413560 22105 raft_consensus.cc:2468] T 7b7143a9509c4df7b955285e3508c4fe P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:13.415241 21732 leader_election.cc:304] T 7b7143a9509c4df7b955285e3508c4fe P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2, ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee
I20260925 13:57:13.416487 22343 raft_consensus.cc:2804] T 7b7143a9509c4df7b955285e3508c4fe P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:13.416790 22343 raft_consensus.cc:493] T 7b7143a9509c4df7b955285e3508c4fe P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:13.417150 22343 raft_consensus.cc:3060] T 7b7143a9509c4df7b955285e3508c4fe P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:13.425238 22343 raft_consensus.cc:515] T 7b7143a9509c4df7b955285e3508c4fe P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.426091 21938 tablet_service.cc:1558] Processing DeleteTablet for tablet 3d59d9b6dfa5474f82e73e07b25b056e with delete_type TABLET_DATA_DELETED (replaced by tablet 7075ed41f73349aaa7ac85d29950b494) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:13.428277 22211 tablet_bootstrap.cc:492] T 7b7143a9509c4df7b955285e3508c4fe P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:13.429587 22211 ts_tablet_manager.cc:1403] T 7b7143a9509c4df7b955285e3508c4fe P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.081s user 0.011s sys 0.000s
I20260925 13:57:13.429996 22214 ts_tablet_manager.cc:1916] T 3d59d9b6dfa5474f82e73e07b25b056e P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:13.432288 22212 ts_tablet_manager.cc:1434] T 7b7143a9509c4df7b955285e3508c4fe P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.035s user 0.005s sys 0.001s
I20260925 13:57:13.432358 22211 raft_consensus.cc:359] T 7b7143a9509c4df7b955285e3508c4fe P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.434439 22211 raft_consensus.cc:385] T 7b7143a9509c4df7b955285e3508c4fe P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:13.434726 22211 raft_consensus.cc:740] T 7b7143a9509c4df7b955285e3508c4fe P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:13.435541 22215 tablet_replica.cc:333] T 3d59d9b6dfa5474f82e73e07b25b056e P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:13.436868 22215 raft_consensus.cc:2243] T 3d59d9b6dfa5474f82e73e07b25b056e P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:13.437414 22215 raft_consensus.cc:2272] T 3d59d9b6dfa5474f82e73e07b25b056e P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:13.439487 22215 ts_tablet_manager.cc:1916] T 3d59d9b6dfa5474f82e73e07b25b056e P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:13.444062 22212 tablet_bootstrap.cc:492] T d595ab2f7dc74fdfbf3546e477685a01 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:13.449806 21580 catalog_manager.cc:5838] T 5888d8f93cec4450847ccfe4854e4012 P ee02f77594864349982492862d668a36 reported cstate change: term changed from 0 to 1, leader changed from <none> to ee02f77594864349982492862d668a36 (127.18.182.195). New cstate: current_term: 1 leader_uuid: "ee02f77594864349982492862d668a36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: HEALTHY } } }
I20260925 13:57:13.451012 21580 catalog_manager.cc:5938] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873) does not have the latest schema for tablet 5888d8f93cec4450847ccfe4854e4012 (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]). Expected version 1 got 0
I20260925 13:57:13.460269 22343 leader_election.cc:290] T 7b7143a9509c4df7b955285e3508c4fe P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Requested vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:13.453980 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b7143a9509c4df7b955285e3508c4fe" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
I20260925 13:57:13.460963 22211 consensus_queue.cc:260] T 7b7143a9509c4df7b955285e3508c4fe P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.462311 21960 raft_consensus.cc:3060] T 7b7143a9509c4df7b955285e3508c4fe P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:13.462641 22105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b7143a9509c4df7b955285e3508c4fe" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:13.463438 22105 raft_consensus.cc:3060] T 7b7143a9509c4df7b955285e3508c4fe P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:13.451756 22293 tablet_bootstrap.cc:654] T d595ab2f7dc74fdfbf3546e477685a01 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:13.468537 21960 raft_consensus.cc:2468] T 7b7143a9509c4df7b955285e3508c4fe P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:13.469731 21732 leader_election.cc:304] T 7b7143a9509c4df7b955285e3508c4fe P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:13.471621 22212 tablet_bootstrap.cc:654] T d595ab2f7dc74fdfbf3546e477685a01 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:13.477043 22343 raft_consensus.cc:2804] T 7b7143a9509c4df7b955285e3508c4fe P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:13.478160 22343 raft_consensus.cc:697] T 7b7143a9509c4df7b955285e3508c4fe P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Becoming Leader. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Running, Role: LEADER
I20260925 13:57:13.479039 22211 ts_tablet_manager.cc:1434] T 7b7143a9509c4df7b955285e3508c4fe P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.049s user 0.005s sys 0.000s
I20260925 13:57:13.479836 22211 tablet_bootstrap.cc:492] T d595ab2f7dc74fdfbf3546e477685a01 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:13.478973 22343 consensus_queue.cc:237] T 7b7143a9509c4df7b955285e3508c4fe P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.485146 22211 tablet_bootstrap.cc:654] T d595ab2f7dc74fdfbf3546e477685a01 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:13.486061 21776 tablet_service.cc:1511] Processing CreateTablet for tablet 7075ed41f73349aaa7ac85d29950b494 (DEFAULT_TABLE table=default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:13.486967 21776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7075ed41f73349aaa7ac85d29950b494. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:13.492098 22105 raft_consensus.cc:2468] T 7b7143a9509c4df7b955285e3508c4fe P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:13.513338 22293 tablet_bootstrap.cc:492] T d595ab2f7dc74fdfbf3546e477685a01 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:13.513727 22293 ts_tablet_manager.cc:1403] T d595ab2f7dc74fdfbf3546e477685a01 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.108s user 0.008s sys 0.002s
I20260925 13:57:13.514259 21580 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35994:
name: "default.Table_61d48f9061c741fc8aa556e3edd984fb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:57:13.514775 21937 tablet_service.cc:1558] Processing DeleteTablet for tablet 7b7143a9509c4df7b955285e3508c4fe with delete_type TABLET_DATA_DELETED (replaced by tablet 412fd2781eec42eebd232d0c81c644c9) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:13.516297 21583 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet 7b7143a9509c4df7b955285e3508c4fe
I20260925 13:57:13.517928 21583 catalog_manager.cc:6934] ReplaceTablet: tablet 7b7143a9509c4df7b955285e3508c4fe deleted and replaced by tablet 412fd2781eec42eebd232d0c81c644c9
I20260925 13:57:13.518385 21937 tablet_service.cc:1558] Processing DeleteTablet for tablet 7b7143a9509c4df7b955285e3508c4fe with delete_type TABLET_DATA_DELETED (replaced by tablet 412fd2781eec42eebd232d0c81c644c9) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:13.519874 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet 7b7143a9509c4df7b955285e3508c4fe with delete_type TABLET_DATA_DELETED (replaced by tablet 412fd2781eec42eebd232d0c81c644c9) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:13.523869 21777 tablet_service.cc:1558] Processing DeleteTablet for tablet 7b7143a9509c4df7b955285e3508c4fe with delete_type TABLET_DATA_DELETED (replaced by tablet 412fd2781eec42eebd232d0c81c644c9) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:13.525326 21777 tablet_service.cc:1558] Processing DeleteTablet for tablet 7b7143a9509c4df7b955285e3508c4fe with delete_type TABLET_DATA_DELETED (replaced by tablet 412fd2781eec42eebd232d0c81c644c9) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:13.529757 22293 raft_consensus.cc:359] T d595ab2f7dc74fdfbf3546e477685a01 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.530390 22293 raft_consensus.cc:385] T d595ab2f7dc74fdfbf3546e477685a01 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:13.530608 22293 raft_consensus.cc:740] T d595ab2f7dc74fdfbf3546e477685a01 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:13.542048 21937 tablet_service.cc:1511] Processing CreateTablet for tablet 7075ed41f73349aaa7ac85d29950b494 (DEFAULT_TABLE table=default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:13.542955 21937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7075ed41f73349aaa7ac85d29950b494. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:13.544914 22083 tablet_service.cc:1511] Processing CreateTablet for tablet 7075ed41f73349aaa7ac85d29950b494 (DEFAULT_TABLE table=default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:13.545902 22083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7075ed41f73349aaa7ac85d29950b494. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:13.556027 22213 ts_tablet_manager.cc:1929] T 3d59d9b6dfa5474f82e73e07b25b056e P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:57:13.574818 22212 tablet_bootstrap.cc:492] T d595ab2f7dc74fdfbf3546e477685a01 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:13.575235 22212 ts_tablet_manager.cc:1403] T d595ab2f7dc74fdfbf3546e477685a01 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.131s user 0.014s sys 0.000s
I20260925 13:57:13.577597 22212 raft_consensus.cc:359] T d595ab2f7dc74fdfbf3546e477685a01 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.578197 22212 raft_consensus.cc:385] T d595ab2f7dc74fdfbf3546e477685a01 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:13.578503 22212 raft_consensus.cc:740] T d595ab2f7dc74fdfbf3546e477685a01 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:13.581804 21585 master_service.cc:960] ReplaceTablet: received request to replace tablet 3d59d9b6dfa5474f82e73e07b25b056e from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:13.584080 22212 consensus_queue.cc:260] T d595ab2f7dc74fdfbf3546e477685a01 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.584439 22215 ts_tablet_manager.cc:1929] T 3d59d9b6dfa5474f82e73e07b25b056e P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:57:13.584801 22215 log.cc:1199] T 3d59d9b6dfa5474f82e73e07b25b056e P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/3d59d9b6dfa5474f82e73e07b25b056e
I20260925 13:57:13.585480 22215 ts_tablet_manager.cc:1950] T 3d59d9b6dfa5474f82e73e07b25b056e P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:13.587844 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet 3d59d9b6dfa5474f82e73e07b25b056e (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]) successfully deleted
I20260925 13:57:13.600591 22215 tablet_replica.cc:333] T 7b7143a9509c4df7b955285e3508c4fe P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:13.601440 22215 raft_consensus.cc:2243] T 7b7143a9509c4df7b955285e3508c4fe P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:13.601857 22215 raft_consensus.cc:2272] T 7b7143a9509c4df7b955285e3508c4fe P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:13.603765 22215 ts_tablet_manager.cc:1916] T 7b7143a9509c4df7b955285e3508c4fe P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:13.563223 22293 consensus_queue.cc:260] T d595ab2f7dc74fdfbf3546e477685a01 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.604400 22213 log.cc:1199] T 3d59d9b6dfa5474f82e73e07b25b056e P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/3d59d9b6dfa5474f82e73e07b25b056e
I20260925 13:57:13.605202 22213 ts_tablet_manager.cc:1950] T 3d59d9b6dfa5474f82e73e07b25b056e P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:13.605835 22293 ts_tablet_manager.cc:1434] T d595ab2f7dc74fdfbf3546e477685a01 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.092s user 0.007s sys 0.000s
I20260925 13:57:13.605983 22212 ts_tablet_manager.cc:1434] T d595ab2f7dc74fdfbf3546e477685a01 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.030s user 0.004s sys 0.000s
I20260925 13:57:13.606860 22293 tablet_bootstrap.cc:492] T 7075ed41f73349aaa7ac85d29950b494 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:13.613142 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet 3d59d9b6dfa5474f82e73e07b25b056e (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]) successfully deleted
I20260925 13:57:13.616510 21960 raft_consensus.cc:1275] T 5888d8f93cec4450847ccfe4854e4012 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Refusing update from remote peer ee02f77594864349982492862d668a36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:57:13.618214 22226 consensus_queue.cc:1048] T 5888d8f93cec4450847ccfe4854e4012 P ee02f77594864349982492862d668a36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:13.620221 21797 raft_consensus.cc:1275] T 5888d8f93cec4450847ccfe4854e4012 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Refusing update from remote peer ee02f77594864349982492862d668a36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:57:13.623498 22293 tablet_bootstrap.cc:654] T 7075ed41f73349aaa7ac85d29950b494 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:13.624177 22212 tablet_bootstrap.cc:492] T 7075ed41f73349aaa7ac85d29950b494 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:13.625665 22226 consensus_queue.cc:1048] T 5888d8f93cec4450847ccfe4854e4012 P ee02f77594864349982492862d668a36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:13.628757 22211 tablet_bootstrap.cc:492] T d595ab2f7dc74fdfbf3546e477685a01 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:13.629173 22211 ts_tablet_manager.cc:1403] T d595ab2f7dc74fdfbf3546e477685a01 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.150s user 0.015s sys 0.026s
I20260925 13:57:13.645582 22212 tablet_bootstrap.cc:654] T 7075ed41f73349aaa7ac85d29950b494 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:13.647504 22211 raft_consensus.cc:359] T d595ab2f7dc74fdfbf3546e477685a01 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.648165 22211 raft_consensus.cc:385] T d595ab2f7dc74fdfbf3546e477685a01 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:13.648437 22211 raft_consensus.cc:740] T d595ab2f7dc74fdfbf3546e477685a01 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:13.643898 22213 tablet_replica.cc:333] T 7b7143a9509c4df7b955285e3508c4fe P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:13.651105 22293 tablet_bootstrap.cc:492] T 7075ed41f73349aaa7ac85d29950b494 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:13.651515 22293 ts_tablet_manager.cc:1403] T 7075ed41f73349aaa7ac85d29950b494 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.045s user 0.009s sys 0.002s
I20260925 13:57:13.651871 22213 raft_consensus.cc:2243] T 7b7143a9509c4df7b955285e3508c4fe P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:13.652611 22213 pending_rounds.cc:70] T 7b7143a9509c4df7b955285e3508c4fe P 5e2731975dbf433aade4a859df9699e2: Trying to abort 1 pending ops.
I20260925 13:57:13.656582 21585 catalog_manager.cc:5938] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291) does not have the latest schema for tablet 5888d8f93cec4450847ccfe4854e4012 (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]). Expected version 1 got 0
I20260925 13:57:13.676206 22215 ts_tablet_manager.cc:1929] T 7b7143a9509c4df7b955285e3508c4fe P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:13.678892 22211 consensus_queue.cc:260] T d595ab2f7dc74fdfbf3546e477685a01 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.681905 22212 tablet_bootstrap.cc:492] T 7075ed41f73349aaa7ac85d29950b494 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:13.682312 22212 ts_tablet_manager.cc:1403] T 7075ed41f73349aaa7ac85d29950b494 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.058s user 0.008s sys 0.000s
I20260925 13:57:13.676579 22215 log.cc:1199] T 7b7143a9509c4df7b955285e3508c4fe P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/7b7143a9509c4df7b955285e3508c4fe
I20260925 13:57:13.683291 22215 ts_tablet_manager.cc:1950] T 7b7143a9509c4df7b955285e3508c4fe P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:13.683334 21583 catalog_manager.cc:5938] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291) does not have the latest schema for tablet 5888d8f93cec4450847ccfe4854e4012 (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]). Expected version 1 got 0
I20260925 13:57:13.686190 22362 tablet.cc:1751] T 5888d8f93cec4450847ccfe4854e4012 P ee02f77594864349982492862d668a36: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:57:13.687530 22211 ts_tablet_manager.cc:1434] T d595ab2f7dc74fdfbf3546e477685a01 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.058s user 0.005s sys 0.003s
I20260925 13:57:13.653613 22293 raft_consensus.cc:359] T 7075ed41f73349aaa7ac85d29950b494 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.690233 22293 raft_consensus.cc:385] T 7075ed41f73349aaa7ac85d29950b494 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:13.690492 22293 raft_consensus.cc:740] T 7075ed41f73349aaa7ac85d29950b494 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:13.691098 22293 consensus_queue.cc:260] T 7075ed41f73349aaa7ac85d29950b494 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.691699 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet 7b7143a9509c4df7b955285e3508c4fe (table default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]) successfully deleted
I20260925 13:57:13.692555 22293 ts_tablet_manager.cc:1434] T 7075ed41f73349aaa7ac85d29950b494 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.041s user 0.005s sys 0.000s
W20260925 13:57:13.693013 21575 catalog_manager.cc:5030] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): delete failed for tablet 7b7143a9509c4df7b955285e3508c4fe because the tablet was not found. No further retry: Not found: Tablet not found: 7b7143a9509c4df7b955285e3508c4fe
I20260925 13:57:13.675765 22363 tablet.cc:1751] T 5888d8f93cec4450847ccfe4854e4012 P 145e520e0a5a445d943a53cf8c27efee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:57:13.694669 22211 tablet_bootstrap.cc:492] T 7075ed41f73349aaa7ac85d29950b494 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:13.679890 21776 tablet_service.cc:1511] Processing CreateTablet for tablet 412fd2781eec42eebd232d0c81c644c9 (DEFAULT_TABLE table=default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:13.696961 21776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 412fd2781eec42eebd232d0c81c644c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:13.703497 22213 pending_rounds.cc:77] T 7b7143a9509c4df7b955285e3508c4fe P 5e2731975dbf433aade4a859df9699e2: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251618746748928 op_type: NO_OP noop_request { }
I20260925 13:57:13.704778 22214 ts_tablet_manager.cc:1929] T 3d59d9b6dfa5474f82e73e07b25b056e P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:57:13.716821 22211 tablet_bootstrap.cc:654] T 7075ed41f73349aaa7ac85d29950b494 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:13.717245 22213 raft_consensus.cc:2889] T 7b7143a9509c4df7b955285e3508c4fe P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:57:13.721704 22214 log.cc:1199] T 3d59d9b6dfa5474f82e73e07b25b056e P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/3d59d9b6dfa5474f82e73e07b25b056e
I20260925 13:57:13.723341 21937 tablet_service.cc:1511] Processing CreateTablet for tablet 412fd2781eec42eebd232d0c81c644c9 (DEFAULT_TABLE table=default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:13.724349 21937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 412fd2781eec42eebd232d0c81c644c9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:57:13.727898 21573 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.182.253:46177: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20260925 13:57:13.732192 21573 consensus_peers.cc:597] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 -> Peer 3ea1e11bc6d84114b7616815b6af4635 (127.18.182.253:46177): Couldn't send request to peer 3ea1e11bc6d84114b7616815b6af4635. Status: Network error: Client connection negotiation failed: client connection to 127.18.182.253:46177: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260925 13:57:13.734824 21585 catalog_manager.cc:5938] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881) does not have the latest schema for tablet 5888d8f93cec4450847ccfe4854e4012 (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]). Expected version 1 got 0
I20260925 13:57:13.738214 22213 raft_consensus.cc:2272] T 7b7143a9509c4df7b955285e3508c4fe P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:13.742067 22213 ts_tablet_manager.cc:1916] T 7b7143a9509c4df7b955285e3508c4fe P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:13.723345 22083 tablet_service.cc:1511] Processing CreateTablet for tablet 412fd2781eec42eebd232d0c81c644c9 (DEFAULT_TABLE table=default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:13.744068 22083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 412fd2781eec42eebd232d0c81c644c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:13.738982 22212 raft_consensus.cc:359] T 7075ed41f73349aaa7ac85d29950b494 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.744695 22212 raft_consensus.cc:385] T 7075ed41f73349aaa7ac85d29950b494 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:13.745060 22212 raft_consensus.cc:740] T 7075ed41f73349aaa7ac85d29950b494 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:13.745733 22212 consensus_queue.cc:260] T 7075ed41f73349aaa7ac85d29950b494 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.748377 22211 tablet_bootstrap.cc:492] T 7075ed41f73349aaa7ac85d29950b494 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:13.748790 22211 ts_tablet_manager.cc:1403] T 7075ed41f73349aaa7ac85d29950b494 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.061s user 0.008s sys 0.000s
I20260925 13:57:13.751128 22211 raft_consensus.cc:359] T 7075ed41f73349aaa7ac85d29950b494 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.765264 22211 raft_consensus.cc:385] T 7075ed41f73349aaa7ac85d29950b494 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:13.765288 22212 ts_tablet_manager.cc:1434] T 7075ed41f73349aaa7ac85d29950b494 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.083s user 0.007s sys 0.000s
I20260925 13:57:13.765520 22211 raft_consensus.cc:740] T 7075ed41f73349aaa7ac85d29950b494 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:13.767679 22211 consensus_queue.cc:260] T 7075ed41f73349aaa7ac85d29950b494 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.770232 22214 ts_tablet_manager.cc:1950] T 3d59d9b6dfa5474f82e73e07b25b056e P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:13.776554 22211 ts_tablet_manager.cc:1434] T 7075ed41f73349aaa7ac85d29950b494 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.028s user 0.002s sys 0.005s
I20260925 13:57:13.782979 22224 raft_consensus.cc:493] T d595ab2f7dc74fdfbf3546e477685a01 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:13.783407 22224 raft_consensus.cc:515] T d595ab2f7dc74fdfbf3546e477685a01 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.785802 22224 leader_election.cc:290] T d595ab2f7dc74fdfbf3546e477685a01 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:13.786008 21585 catalog_manager.cc:5938] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881) does not have the latest schema for tablet 7075ed41f73349aaa7ac85d29950b494 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 1 got 0
I20260925 13:57:13.787257 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d595ab2f7dc74fdfbf3546e477685a01" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2" is_pre_election: true
I20260925 13:57:13.787896 21795 raft_consensus.cc:2468] T d595ab2f7dc74fdfbf3546e477685a01 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 0.
I20260925 13:57:13.788482 22211 tablet_bootstrap.cc:492] T 412fd2781eec42eebd232d0c81c644c9 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:13.789748 22105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d595ab2f7dc74fdfbf3546e477685a01" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:13.790318 22105 raft_consensus.cc:2468] T d595ab2f7dc74fdfbf3546e477685a01 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 0.
I20260925 13:57:13.793450 21896 leader_election.cc:304] T d595ab2f7dc74fdfbf3546e477685a01 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:13.794893 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet 3d59d9b6dfa5474f82e73e07b25b056e (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]) successfully deleted
I20260925 13:57:13.796818 21583 catalog_manager.cc:5938] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873) does not have the latest schema for tablet 7075ed41f73349aaa7ac85d29950b494 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 1 got 0
I20260925 13:57:13.797955 22211 tablet_bootstrap.cc:654] T 412fd2781eec42eebd232d0c81c644c9 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
W20260925 13:57:13.798494 21583 catalog_manager.cc:6038] Failed to send AlterTable RPC for tablet 7075ed41f73349aaa7ac85d29950b494 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 7075ed41f73349aaa7ac85d29950b494 consensus state has no leader
I20260925 13:57:13.799959 22224 raft_consensus.cc:2804] T d595ab2f7dc74fdfbf3546e477685a01 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:13.800300 22224 raft_consensus.cc:493] T d595ab2f7dc74fdfbf3546e477685a01 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:13.800638 22224 raft_consensus.cc:3060] T d595ab2f7dc74fdfbf3546e477685a01 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:13.805207 21583 catalog_manager.cc:5938] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291) does not have the latest schema for tablet 7075ed41f73349aaa7ac85d29950b494 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 1 got 0
I20260925 13:57:13.809487 22224 raft_consensus.cc:515] T d595ab2f7dc74fdfbf3546e477685a01 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.813454 21585 catalog_manager.cc:5938] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873) does not have the latest schema for tablet 7075ed41f73349aaa7ac85d29950b494 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 1 got 0
I20260925 13:57:13.821615 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d595ab2f7dc74fdfbf3546e477685a01" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2"
I20260925 13:57:13.822355 21797 raft_consensus.cc:3060] T d595ab2f7dc74fdfbf3546e477685a01 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:13.828974 22211 tablet_bootstrap.cc:492] T 412fd2781eec42eebd232d0c81c644c9 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:13.829366 22211 ts_tablet_manager.cc:1403] T 412fd2781eec42eebd232d0c81c644c9 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.052s user 0.006s sys 0.003s
I20260925 13:57:13.834992 22212 tablet_bootstrap.cc:492] T 412fd2781eec42eebd232d0c81c644c9 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:13.838266 22214 tablet_replica.cc:333] T 7b7143a9509c4df7b955285e3508c4fe P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:13.838318 22105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d595ab2f7dc74fdfbf3546e477685a01" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:13.839635 22105 raft_consensus.cc:3060] T d595ab2f7dc74fdfbf3546e477685a01 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:13.839798 22214 raft_consensus.cc:2243] T 7b7143a9509c4df7b955285e3508c4fe P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:13.840232 22214 raft_consensus.cc:2272] T 7b7143a9509c4df7b955285e3508c4fe P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:13.843830 22211 raft_consensus.cc:359] T 412fd2781eec42eebd232d0c81c644c9 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.844974 22211 raft_consensus.cc:385] T 412fd2781eec42eebd232d0c81c644c9 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:13.845296 22211 raft_consensus.cc:740] T 412fd2781eec42eebd232d0c81c644c9 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:13.848505 22105 raft_consensus.cc:2468] T d595ab2f7dc74fdfbf3546e477685a01 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 1.
I20260925 13:57:13.849865 21896 leader_election.cc:304] T d595ab2f7dc74fdfbf3546e477685a01 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:13.851257 22212 tablet_bootstrap.cc:654] T 412fd2781eec42eebd232d0c81c644c9 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:13.852491 22211 consensus_queue.cc:260] T 412fd2781eec42eebd232d0c81c644c9 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.854481 21583 master_service.cc:960] ReplaceTablet: received request to replace tablet 7075ed41f73349aaa7ac85d29950b494 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:13.857549 22368 raft_consensus.cc:2804] T d595ab2f7dc74fdfbf3546e477685a01 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election won for term 1
W20260925 13:57:13.858696 21572 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 7075ed41f73349aaa7ac85d29950b494 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 7075ed41f73349aaa7ac85d29950b494 consensus state has no leader
I20260925 13:57:13.860284 22211 ts_tablet_manager.cc:1434] T 412fd2781eec42eebd232d0c81c644c9 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.031s user 0.007s sys 0.001s
I20260925 13:57:13.870358 22224 leader_election.cc:290] T d595ab2f7dc74fdfbf3546e477685a01 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Requested vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:13.871603 21797 raft_consensus.cc:2468] T d595ab2f7dc74fdfbf3546e477685a01 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 1.
I20260925 13:57:13.872975 22214 ts_tablet_manager.cc:1916] T 7b7143a9509c4df7b955285e3508c4fe P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:13.876681 22368 raft_consensus.cc:697] T d595ab2f7dc74fdfbf3546e477685a01 P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: Becoming Leader. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Running, Role: LEADER
I20260925 13:57:13.882542 22368 consensus_queue.cc:237] T d595ab2f7dc74fdfbf3546e477685a01 P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.909668 21583 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet 7075ed41f73349aaa7ac85d29950b494
I20260925 13:57:13.911887 21777 tablet_service.cc:1558] Processing DeleteTablet for tablet 7075ed41f73349aaa7ac85d29950b494 with delete_type TABLET_DATA_DELETED (replaced by tablet 52d8da0caee84c8ea79f382564575b94) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:13.880475 22365 tablet.cc:1751] T 5888d8f93cec4450847ccfe4854e4012 P 5e2731975dbf433aade4a859df9699e2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:57:13.930150 22293 tablet_bootstrap.cc:492] T 412fd2781eec42eebd232d0c81c644c9 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:13.936022 22293 tablet_bootstrap.cc:654] T 412fd2781eec42eebd232d0c81c644c9 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:13.936020 21937 tablet_service.cc:1558] Processing DeleteTablet for tablet 7075ed41f73349aaa7ac85d29950b494 with delete_type TABLET_DATA_DELETED (replaced by tablet 52d8da0caee84c8ea79f382564575b94) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:13.937285 21583 catalog_manager.cc:6934] ReplaceTablet: tablet 7075ed41f73349aaa7ac85d29950b494 deleted and replaced by tablet 52d8da0caee84c8ea79f382564575b94
I20260925 13:57:13.937672 22215 tablet_replica.cc:333] T 7075ed41f73349aaa7ac85d29950b494 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:13.938593 22215 raft_consensus.cc:2243] T 7075ed41f73349aaa7ac85d29950b494 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:13.939070 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet 7075ed41f73349aaa7ac85d29950b494 with delete_type TABLET_DATA_DELETED (replaced by tablet 52d8da0caee84c8ea79f382564575b94) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:13.939146 22215 raft_consensus.cc:2272] T 7075ed41f73349aaa7ac85d29950b494 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:13.945565 22215 ts_tablet_manager.cc:1916] T 7075ed41f73349aaa7ac85d29950b494 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:13.958214 22215 ts_tablet_manager.cc:1929] T 7075ed41f73349aaa7ac85d29950b494 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:13.958526 22215 log.cc:1199] T 7075ed41f73349aaa7ac85d29950b494 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/7075ed41f73349aaa7ac85d29950b494
I20260925 13:57:13.959116 22215 ts_tablet_manager.cc:1950] T 7075ed41f73349aaa7ac85d29950b494 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:13.962083 21583 catalog_manager.cc:5938] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881) does not have the latest schema for tablet 5888d8f93cec4450847ccfe4854e4012 (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]). Expected version 1 got 0
I20260925 13:57:13.967332 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet 7075ed41f73349aaa7ac85d29950b494 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]) successfully deleted
I20260925 13:57:13.978618 22213 ts_tablet_manager.cc:1929] T 7b7143a9509c4df7b955285e3508c4fe P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:13.983461 22212 tablet_bootstrap.cc:492] T 412fd2781eec42eebd232d0c81c644c9 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:13.983880 22212 ts_tablet_manager.cc:1403] T 412fd2781eec42eebd232d0c81c644c9 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.149s user 0.010s sys 0.000s
I20260925 13:57:13.986254 22212 raft_consensus.cc:359] T 412fd2781eec42eebd232d0c81c644c9 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.999603 22212 raft_consensus.cc:385] T 412fd2781eec42eebd232d0c81c644c9 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:13.999876 22212 raft_consensus.cc:740] T 412fd2781eec42eebd232d0c81c644c9 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:14.000509 22212 consensus_queue.cc:260] T 412fd2781eec42eebd232d0c81c644c9 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:13.994931 22293 tablet_bootstrap.cc:492] T 412fd2781eec42eebd232d0c81c644c9 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:14.013230 22213 log.cc:1199] T 7b7143a9509c4df7b955285e3508c4fe P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/7b7143a9509c4df7b955285e3508c4fe
I20260925 13:57:14.013403 22293 ts_tablet_manager.cc:1403] T 412fd2781eec42eebd232d0c81c644c9 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.083s user 0.006s sys 0.009s
I20260925 13:57:14.014052 22213 ts_tablet_manager.cc:1950] T 7b7143a9509c4df7b955285e3508c4fe P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:14.014846 21583 master_service.cc:960] ReplaceTablet: received request to replace tablet 7075ed41f73349aaa7ac85d29950b494 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:14.019186 22293 raft_consensus.cc:359] T 412fd2781eec42eebd232d0c81c644c9 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:14.020073 22293 raft_consensus.cc:385] T 412fd2781eec42eebd232d0c81c644c9 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:14.020393 22293 raft_consensus.cc:740] T 412fd2781eec42eebd232d0c81c644c9 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:14.021426 22293 consensus_queue.cc:260] T 412fd2781eec42eebd232d0c81c644c9 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:14.023862 22293 ts_tablet_manager.cc:1434] T 412fd2781eec42eebd232d0c81c644c9 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20260925 13:57:14.030918 21586 catalog_manager.cc:2770] deleted HMS catalog entry for table default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]
I20260925 13:57:14.038291 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet 7b7143a9509c4df7b955285e3508c4fe (table default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]) successfully deleted
W20260925 13:57:14.039985 21572 catalog_manager.cc:5030] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): delete failed for tablet 7b7143a9509c4df7b955285e3508c4fe because the tablet was not found. No further retry: Not found: Tablet not found: 7b7143a9509c4df7b955285e3508c4fe
I20260925 13:57:14.040881 22213 tablet_replica.cc:333] T 7075ed41f73349aaa7ac85d29950b494 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:14.041576 22213 raft_consensus.cc:2243] T 7075ed41f73349aaa7ac85d29950b494 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:14.041980 22213 raft_consensus.cc:2272] T 7075ed41f73349aaa7ac85d29950b494 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:14.055200 22212 ts_tablet_manager.cc:1434] T 412fd2781eec42eebd232d0c81c644c9 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.071s user 0.006s sys 0.000s
I20260925 13:57:14.057353 22213 ts_tablet_manager.cc:1916] T 7075ed41f73349aaa7ac85d29950b494 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:14.072057 21776 tablet_service.cc:1511] Processing CreateTablet for tablet 52d8da0caee84c8ea79f382564575b94 (DEFAULT_TABLE table=default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:14.073066 21776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52d8da0caee84c8ea79f382564575b94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:14.073012 21937 tablet_service.cc:1511] Processing CreateTablet for tablet 52d8da0caee84c8ea79f382564575b94 (DEFAULT_TABLE table=default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:14.073984 21937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52d8da0caee84c8ea79f382564575b94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:14.086971 22343 consensus_queue.cc:1048] T 0d216948345a4db488fd0a9ff367bfd3 P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:14.089231 21585 catalog_manager.cc:5838] T d595ab2f7dc74fdfbf3546e477685a01 P 145e520e0a5a445d943a53cf8c27efee reported cstate change: term changed from 0 to 1, leader changed from <none> to 145e520e0a5a445d943a53cf8c27efee (127.18.182.194). New cstate: current_term: 1 leader_uuid: "145e520e0a5a445d943a53cf8c27efee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: UNKNOWN } } }
I20260925 13:57:14.109210 22213 ts_tablet_manager.cc:1929] T 7075ed41f73349aaa7ac85d29950b494 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:14.109844 22213 log.cc:1199] T 7075ed41f73349aaa7ac85d29950b494 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/7075ed41f73349aaa7ac85d29950b494
I20260925 13:57:14.110755 22213 ts_tablet_manager.cc:1950] T 7075ed41f73349aaa7ac85d29950b494 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:14.113803 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet 7075ed41f73349aaa7ac85d29950b494 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]) successfully deleted
I20260925 13:57:14.123404 22214 ts_tablet_manager.cc:1929] T 7b7143a9509c4df7b955285e3508c4fe P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:14.123854 22214 log.cc:1199] T 7b7143a9509c4df7b955285e3508c4fe P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/7b7143a9509c4df7b955285e3508c4fe
I20260925 13:57:14.124498 22214 ts_tablet_manager.cc:1950] T 7b7143a9509c4df7b955285e3508c4fe P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:14.126818 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet 7b7143a9509c4df7b955285e3508c4fe (table default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]) successfully deleted
I20260925 13:57:14.127717 22214 tablet_replica.cc:333] T 7075ed41f73349aaa7ac85d29950b494 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:14.129906 22083 tablet_service.cc:1511] Processing CreateTablet for tablet 52d8da0caee84c8ea79f382564575b94 (DEFAULT_TABLE table=default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:14.130820 22083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52d8da0caee84c8ea79f382564575b94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:14.145169 22212 tablet_bootstrap.cc:492] T 52d8da0caee84c8ea79f382564575b94 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:14.150260 22212 tablet_bootstrap.cc:654] T 52d8da0caee84c8ea79f382564575b94 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:14.165248 22212 tablet_bootstrap.cc:492] T 52d8da0caee84c8ea79f382564575b94 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:14.165647 22212 ts_tablet_manager.cc:1403] T 52d8da0caee84c8ea79f382564575b94 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.021s user 0.012s sys 0.000s
I20260925 13:57:14.168277 22212 raft_consensus.cc:359] T 52d8da0caee84c8ea79f382564575b94 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:14.168860 22212 raft_consensus.cc:385] T 52d8da0caee84c8ea79f382564575b94 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:14.169111 22212 raft_consensus.cc:740] T 52d8da0caee84c8ea79f382564575b94 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:14.169688 22212 consensus_queue.cc:260] T 52d8da0caee84c8ea79f382564575b94 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:14.173599 21583 master_service.cc:960] ReplaceTablet: received request to replace tablet 5888d8f93cec4450847ccfe4854e4012 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:14.176152 22212 ts_tablet_manager.cc:1434] T 52d8da0caee84c8ea79f382564575b94 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.010s user 0.006s sys 0.001s
I20260925 13:57:14.176177 22343 raft_consensus.cc:493] T 412fd2781eec42eebd232d0c81c644c9 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:14.176654 22343 raft_consensus.cc:515] T 412fd2781eec42eebd232d0c81c644c9 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:14.179700 21959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "412fd2781eec42eebd232d0c81c644c9" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:14.180306 21959 raft_consensus.cc:2468] T 412fd2781eec42eebd232d0c81c644c9 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:14.182255 21732 leader_election.cc:304] T 412fd2781eec42eebd232d0c81c644c9 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:14.188815 22375 raft_consensus.cc:2804] T 412fd2781eec42eebd232d0c81c644c9 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:14.189145 22375 raft_consensus.cc:493] T 412fd2781eec42eebd232d0c81c644c9 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:14.189561 22375 raft_consensus.cc:3060] T 412fd2781eec42eebd232d0c81c644c9 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:14.193248 22343 leader_election.cc:290] T 412fd2781eec42eebd232d0c81c644c9 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:14.202826 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "412fd2781eec42eebd232d0c81c644c9" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:14.203416 22107 raft_consensus.cc:2468] T 412fd2781eec42eebd232d0c81c644c9 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:14.205615 22343 consensus_queue.cc:1048] T 0d216948345a4db488fd0a9ff367bfd3 P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:14.212848 22293 tablet_bootstrap.cc:492] T 52d8da0caee84c8ea79f382564575b94 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:14.222054 22293 tablet_bootstrap.cc:654] T 52d8da0caee84c8ea79f382564575b94 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:14.224020 21583 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet 5888d8f93cec4450847ccfe4854e4012
I20260925 13:57:14.228235 21938 tablet_service.cc:1558] Processing DeleteTablet for tablet 5888d8f93cec4450847ccfe4854e4012 with delete_type TABLET_DATA_DELETED (replaced by tablet d71049b217a74662b7a9974e776d1448) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:14.227666 21776 tablet_service.cc:1558] Processing DeleteTablet for tablet 5888d8f93cec4450847ccfe4854e4012 with delete_type TABLET_DATA_DELETED (replaced by tablet d71049b217a74662b7a9974e776d1448) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:14.228995 22215 tablet_replica.cc:333] T 5888d8f93cec4450847ccfe4854e4012 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:14.229194 22213 tablet_replica.cc:333] T 5888d8f93cec4450847ccfe4854e4012 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:14.229627 21583 catalog_manager.cc:6934] ReplaceTablet: tablet 5888d8f93cec4450847ccfe4854e4012 deleted and replaced by tablet d71049b217a74662b7a9974e776d1448
I20260925 13:57:14.229629 22215 raft_consensus.cc:2243] T 5888d8f93cec4450847ccfe4854e4012 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:14.229830 22213 raft_consensus.cc:2243] T 5888d8f93cec4450847ccfe4854e4012 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:14.230113 22215 raft_consensus.cc:2272] T 5888d8f93cec4450847ccfe4854e4012 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:14.230726 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet 5888d8f93cec4450847ccfe4854e4012 with delete_type TABLET_DATA_DELETED (replaced by tablet d71049b217a74662b7a9974e776d1448) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:14.232887 22215 ts_tablet_manager.cc:1916] T 5888d8f93cec4450847ccfe4854e4012 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:14.236320 22214 raft_consensus.cc:2243] T 7075ed41f73349aaa7ac85d29950b494 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:14.246109 22211 tablet_bootstrap.cc:492] T 52d8da0caee84c8ea79f382564575b94 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:14.251515 22211 tablet_bootstrap.cc:654] T 52d8da0caee84c8ea79f382564575b94 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:14.251637 22214 raft_consensus.cc:2272] T 7075ed41f73349aaa7ac85d29950b494 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:14.252612 22213 raft_consensus.cc:2272] T 5888d8f93cec4450847ccfe4854e4012 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:14.253311 22214 ts_tablet_manager.cc:1916] T 7075ed41f73349aaa7ac85d29950b494 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:14.254942 22213 ts_tablet_manager.cc:1916] T 5888d8f93cec4450847ccfe4854e4012 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:14.263252 22211 tablet_bootstrap.cc:492] T 52d8da0caee84c8ea79f382564575b94 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:14.263655 22211 ts_tablet_manager.cc:1403] T 52d8da0caee84c8ea79f382564575b94 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.003s
I20260925 13:57:14.266328 22211 raft_consensus.cc:359] T 52d8da0caee84c8ea79f382564575b94 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:14.266911 22211 raft_consensus.cc:385] T 52d8da0caee84c8ea79f382564575b94 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:14.269604 22293 tablet_bootstrap.cc:492] T 52d8da0caee84c8ea79f382564575b94 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:14.270025 22293 ts_tablet_manager.cc:1403] T 52d8da0caee84c8ea79f382564575b94 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.057s user 0.012s sys 0.007s
I20260925 13:57:14.269971 22215 ts_tablet_manager.cc:1929] T 5888d8f93cec4450847ccfe4854e4012 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:57:14.270304 22215 log.cc:1199] T 5888d8f93cec4450847ccfe4854e4012 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/5888d8f93cec4450847ccfe4854e4012
I20260925 13:57:14.271015 22215 ts_tablet_manager.cc:1950] T 5888d8f93cec4450847ccfe4854e4012 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:14.272436 22293 raft_consensus.cc:359] T 52d8da0caee84c8ea79f382564575b94 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:14.273025 22293 raft_consensus.cc:385] T 52d8da0caee84c8ea79f382564575b94 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:14.273280 22293 raft_consensus.cc:740] T 52d8da0caee84c8ea79f382564575b94 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:14.273509 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet 5888d8f93cec4450847ccfe4854e4012 (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]) successfully deleted
I20260925 13:57:14.273869 22293 consensus_queue.cc:260] T 52d8da0caee84c8ea79f382564575b94 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:14.267154 22211 raft_consensus.cc:740] T 52d8da0caee84c8ea79f382564575b94 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:14.276468 22211 consensus_queue.cc:260] T 52d8da0caee84c8ea79f382564575b94 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:14.281926 22211 ts_tablet_manager.cc:1434] T 52d8da0caee84c8ea79f382564575b94 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.018s user 0.006s sys 0.000s
I20260925 13:57:14.285157 22293 ts_tablet_manager.cc:1434] T 52d8da0caee84c8ea79f382564575b94 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.015s user 0.002s sys 0.004s
I20260925 13:57:14.312743 21583 catalog_manager.cc:5938] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881) does not have the latest schema for tablet 52d8da0caee84c8ea79f382564575b94 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 1 got 0
W20260925 13:57:14.314772 21583 catalog_manager.cc:6038] Failed to send AlterTable RPC for tablet 52d8da0caee84c8ea79f382564575b94 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 52d8da0caee84c8ea79f382564575b94 consensus state has no leader
I20260925 13:57:14.320938 21776 tablet_service.cc:1511] Processing CreateTablet for tablet d71049b217a74662b7a9974e776d1448 (DEFAULT_TABLE table=default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:14.322260 21776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d71049b217a74662b7a9974e776d1448. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:14.325380 21583 catalog_manager.cc:5938] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873) does not have the latest schema for tablet 52d8da0caee84c8ea79f382564575b94 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 1 got 0
I20260925 13:57:14.325660 21937 tablet_service.cc:1511] Processing CreateTablet for tablet d71049b217a74662b7a9974e776d1448 (DEFAULT_TABLE table=default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:14.326625 21937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d71049b217a74662b7a9974e776d1448. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:14.333209 21585 catalog_manager.cc:5938] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881) does not have the latest schema for tablet 52d8da0caee84c8ea79f382564575b94 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 1 got 0
I20260925 13:57:14.341300 22083 tablet_service.cc:1511] Processing CreateTablet for tablet d71049b217a74662b7a9974e776d1448 (DEFAULT_TABLE table=default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:14.342356 22083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d71049b217a74662b7a9974e776d1448. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:14.344717 22213 ts_tablet_manager.cc:1929] T 5888d8f93cec4450847ccfe4854e4012 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:57:14.345054 22213 log.cc:1199] T 5888d8f93cec4450847ccfe4854e4012 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/5888d8f93cec4450847ccfe4854e4012
I20260925 13:57:14.345715 22213 ts_tablet_manager.cc:1950] T 5888d8f93cec4450847ccfe4854e4012 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:14.346909 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet 5888d8f93cec4450847ccfe4854e4012 (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]) successfully deleted
I20260925 13:57:14.356999 22375 raft_consensus.cc:515] T 412fd2781eec42eebd232d0c81c644c9 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:14.359705 21959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "412fd2781eec42eebd232d0c81c644c9" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
I20260925 13:57:14.360250 21959 raft_consensus.cc:3060] T 412fd2781eec42eebd232d0c81c644c9 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:14.372471 22293 tablet_bootstrap.cc:492] T d71049b217a74662b7a9974e776d1448 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:14.376328 22226 raft_consensus.cc:493] T 52d8da0caee84c8ea79f382564575b94 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:14.376791 22226 raft_consensus.cc:515] T 52d8da0caee84c8ea79f382564575b94 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:14.378647 22293 tablet_bootstrap.cc:654] T d71049b217a74662b7a9974e776d1448 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:14.378669 22226 leader_election.cc:290] T 52d8da0caee84c8ea79f382564575b94 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:14.380400 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52d8da0caee84c8ea79f382564575b94" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:14.380566 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52d8da0caee84c8ea79f382564575b94" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2" is_pre_election: true
I20260925 13:57:14.380991 21960 raft_consensus.cc:2468] T 52d8da0caee84c8ea79f382564575b94 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee02f77594864349982492862d668a36 in term 0.
I20260925 13:57:14.381124 21797 raft_consensus.cc:2468] T 52d8da0caee84c8ea79f382564575b94 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee02f77594864349982492862d668a36 in term 0.
I20260925 13:57:14.382488 22043 leader_election.cc:304] T 52d8da0caee84c8ea79f382564575b94 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:14.383884 22226 raft_consensus.cc:2804] T 52d8da0caee84c8ea79f382564575b94 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:14.384191 22226 raft_consensus.cc:493] T 52d8da0caee84c8ea79f382564575b94 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:14.384388 21585 master_service.cc:960] ReplaceTablet: received request to replace tablet 0d216948345a4db488fd0a9ff367bfd3 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:14.384481 22226 raft_consensus.cc:3060] T 52d8da0caee84c8ea79f382564575b94 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:14.396157 22226 raft_consensus.cc:515] T 52d8da0caee84c8ea79f382564575b94 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:14.405087 22212 tablet_bootstrap.cc:492] T d71049b217a74662b7a9974e776d1448 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:14.411909 22212 tablet_bootstrap.cc:654] T d71049b217a74662b7a9974e776d1448 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:14.415125 22375 leader_election.cc:290] T 412fd2781eec42eebd232d0c81c644c9 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Requested vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:14.416249 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52d8da0caee84c8ea79f382564575b94" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2"
I20260925 13:57:14.416280 22105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "412fd2781eec42eebd232d0c81c644c9" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:14.416780 21797 raft_consensus.cc:3060] T 52d8da0caee84c8ea79f382564575b94 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:14.416935 22105 raft_consensus.cc:3060] T 412fd2781eec42eebd232d0c81c644c9 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:14.418676 22293 tablet_bootstrap.cc:492] T d71049b217a74662b7a9974e776d1448 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:14.419057 22293 ts_tablet_manager.cc:1403] T d71049b217a74662b7a9974e776d1448 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.047s user 0.012s sys 0.008s
I20260925 13:57:14.423991 21797 raft_consensus.cc:2468] T 52d8da0caee84c8ea79f382564575b94 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee02f77594864349982492862d668a36 in term 1.
I20260925 13:57:14.426738 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52d8da0caee84c8ea79f382564575b94" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
I20260925 13:57:14.427449 21960 raft_consensus.cc:3060] T 52d8da0caee84c8ea79f382564575b94 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:14.429668 22105 raft_consensus.cc:2468] T 412fd2781eec42eebd232d0c81c644c9 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:14.438848 21585 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet 0d216948345a4db488fd0a9ff367bfd3
I20260925 13:57:14.441980 22385 raft_consensus.cc:493] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:14.442168 22226 leader_election.cc:290] T 52d8da0caee84c8ea79f382564575b94 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 election: Requested vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:14.443317 21776 tablet_service.cc:1558] Processing DeleteTablet for tablet 0d216948345a4db488fd0a9ff367bfd3 with delete_type TABLET_DATA_DELETED (replaced by tablet 1dd5ef2086884c81b48000d7e9de14e2) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:14.443955 22213 tablet_replica.cc:333] T 0d216948345a4db488fd0a9ff367bfd3 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:14.444419 21938 tablet_service.cc:1558] Processing DeleteTablet for tablet 0d216948345a4db488fd0a9ff367bfd3 with delete_type TABLET_DATA_DELETED (replaced by tablet 1dd5ef2086884c81b48000d7e9de14e2) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:14.445091 22215 tablet_replica.cc:333] T 0d216948345a4db488fd0a9ff367bfd3 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:14.445873 22215 raft_consensus.cc:2243] T 0d216948345a4db488fd0a9ff367bfd3 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:14.445926 21583 catalog_manager.cc:5938] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291) does not have the latest schema for tablet 52d8da0caee84c8ea79f382564575b94 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 1 got 0
I20260925 13:57:14.446358 22215 raft_consensus.cc:2272] T 0d216948345a4db488fd0a9ff367bfd3 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:14.448799 22215 ts_tablet_manager.cc:1916] T 0d216948345a4db488fd0a9ff367bfd3 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:14.448858 22087 tablet_service.cc:1558] Processing DeleteTablet for tablet 0d216948345a4db488fd0a9ff367bfd3 with delete_type TABLET_DATA_DELETED (replaced by tablet 1dd5ef2086884c81b48000d7e9de14e2) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:14.444650 22213 raft_consensus.cc:2243] T 0d216948345a4db488fd0a9ff367bfd3 P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:14.455830 22385 raft_consensus.cc:515] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:14.457873 22385 leader_election.cc:290] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:14.458415 21585 catalog_manager.cc:6934] ReplaceTablet: tablet 0d216948345a4db488fd0a9ff367bfd3 deleted and replaced by tablet 1dd5ef2086884c81b48000d7e9de14e2
I20260925 13:57:14.459345 21585 master_service.cc:960] ReplaceTablet: received request to replace tablet 0d216948345a4db488fd0a9ff367bfd3 from {username='slave'} at 127.0.0.1:35994
W20260925 13:57:14.460843 22043 consensus_peers.cc:597] T 5888d8f93cec4450847ccfe4854e4012 P ee02f77594864349982492862d668a36 -> Peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Couldn't send request to peer 145e520e0a5a445d943a53cf8c27efee. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5888d8f93cec4450847ccfe4854e4012. This is attempt 1: this message will repeat every 5th retry.
W20260925 13:57:14.462383 22043 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
I20260925 13:57:14.463528 22213 raft_consensus.cc:2272] T 0d216948345a4db488fd0a9ff367bfd3 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:14.465612 21960 raft_consensus.cc:2468] T 52d8da0caee84c8ea79f382564575b94 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee02f77594864349982492862d668a36 in term 1.
I20260925 13:57:14.465994 22213 ts_tablet_manager.cc:1916] T 0d216948345a4db488fd0a9ff367bfd3 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:14.466899 22043 leader_election.cc:304] T 52d8da0caee84c8ea79f382564575b94 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:14.467711 22226 raft_consensus.cc:2804] T 52d8da0caee84c8ea79f382564575b94 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:14.467675 21959 raft_consensus.cc:2468] T 412fd2781eec42eebd232d0c81c644c9 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:14.468137 22226 raft_consensus.cc:697] T 52d8da0caee84c8ea79f382564575b94 P ee02f77594864349982492862d668a36 [term 1 LEADER]: Becoming Leader. State: Replica: ee02f77594864349982492862d668a36, State: Running, Role: LEADER
I20260925 13:57:14.468871 22226 consensus_queue.cc:237] T 52d8da0caee84c8ea79f382564575b94 P ee02f77594864349982492862d668a36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:14.469370 21732 leader_election.cc:304] T 412fd2781eec42eebd232d0c81c644c9 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:14.469467 22293 raft_consensus.cc:359] T d71049b217a74662b7a9974e776d1448 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:14.470299 22293 raft_consensus.cc:385] T d71049b217a74662b7a9974e776d1448 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:14.470427 22375 raft_consensus.cc:2804] T 412fd2781eec42eebd232d0c81c644c9 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:14.470610 22293 raft_consensus.cc:740] T d71049b217a74662b7a9974e776d1448 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:14.471453 22293 consensus_queue.cc:260] T d71049b217a74662b7a9974e776d1448 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:14.477420 22214 ts_tablet_manager.cc:1929] T 7075ed41f73349aaa7ac85d29950b494 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:14.477811 22214 log.cc:1199] T 7075ed41f73349aaa7ac85d29950b494 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/7075ed41f73349aaa7ac85d29950b494
I20260925 13:57:14.478528 22214 ts_tablet_manager.cc:1950] T 7075ed41f73349aaa7ac85d29950b494 P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:14.479570 22375 raft_consensus.cc:697] T 412fd2781eec42eebd232d0c81c644c9 P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Becoming Leader. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Running, Role: LEADER
I20260925 13:57:14.480973 22214 tablet_replica.cc:333] T 5888d8f93cec4450847ccfe4854e4012 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:14.481221 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet 7075ed41f73349aaa7ac85d29950b494 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]) successfully deleted
I20260925 13:57:14.480602 22375 consensus_queue.cc:237] T 412fd2781eec42eebd232d0c81c644c9 P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:14.487478 22211 tablet_bootstrap.cc:492] T d71049b217a74662b7a9974e776d1448 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:14.493618 22211 tablet_bootstrap.cc:654] T d71049b217a74662b7a9974e776d1448 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:14.497728 22215 ts_tablet_manager.cc:1929] T 0d216948345a4db488fd0a9ff367bfd3 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:14.498085 22215 log.cc:1199] T 0d216948345a4db488fd0a9ff367bfd3 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/0d216948345a4db488fd0a9ff367bfd3
I20260925 13:57:14.500938 22215 ts_tablet_manager.cc:1950] T 0d216948345a4db488fd0a9ff367bfd3 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:14.504109 22293 ts_tablet_manager.cc:1434] T d71049b217a74662b7a9974e776d1448 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.085s user 0.006s sys 0.000s
I20260925 13:57:14.505910 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet 0d216948345a4db488fd0a9ff367bfd3 (table default.table_492c567ddda14e77851270f34471730e [id=754c40502e614c5bb18230c852aa0016]) successfully deleted
I20260925 13:57:14.512920 22211 tablet_bootstrap.cc:492] T d71049b217a74662b7a9974e776d1448 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:14.513387 22211 ts_tablet_manager.cc:1403] T d71049b217a74662b7a9974e776d1448 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.026s user 0.010s sys 0.000s
I20260925 13:57:14.515873 22211 raft_consensus.cc:359] T d71049b217a74662b7a9974e776d1448 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:14.516561 22211 raft_consensus.cc:385] T d71049b217a74662b7a9974e776d1448 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:14.516866 22211 raft_consensus.cc:740] T d71049b217a74662b7a9974e776d1448 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:14.517659 22211 consensus_queue.cc:260] T d71049b217a74662b7a9974e776d1448 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:14.481307 22212 tablet_bootstrap.cc:492] T d71049b217a74662b7a9974e776d1448 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:14.521118 22212 ts_tablet_manager.cc:1403] T d71049b217a74662b7a9974e776d1448 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.116s user 0.010s sys 0.006s
I20260925 13:57:14.523598 22212 raft_consensus.cc:359] T d71049b217a74662b7a9974e776d1448 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:14.524505 22212 raft_consensus.cc:385] T d71049b217a74662b7a9974e776d1448 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:14.524819 22212 raft_consensus.cc:740] T d71049b217a74662b7a9974e776d1448 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:14.525617 22212 consensus_queue.cc:260] T d71049b217a74662b7a9974e776d1448 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:14.488269 22040 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
I20260925 13:57:14.528079 22212 ts_tablet_manager.cc:1434] T d71049b217a74662b7a9974e776d1448 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.007s user 0.004s sys 0.002s
I20260925 13:57:14.527298 22040 leader_election.cc:304] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:14.532862 22226 raft_consensus.cc:2749] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:57:14.546509 22214 raft_consensus.cc:2243] T 5888d8f93cec4450847ccfe4854e4012 P ee02f77594864349982492862d668a36 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:14.547591 22214 raft_consensus.cc:2272] T 5888d8f93cec4450847ccfe4854e4012 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:14.549784 22214 ts_tablet_manager.cc:1916] T 5888d8f93cec4450847ccfe4854e4012 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:14.560881 22213 ts_tablet_manager.cc:1929] T 0d216948345a4db488fd0a9ff367bfd3 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:14.561260 22213 log.cc:1199] T 0d216948345a4db488fd0a9ff367bfd3 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/0d216948345a4db488fd0a9ff367bfd3
I20260925 13:57:14.563727 22213 ts_tablet_manager.cc:1950] T 0d216948345a4db488fd0a9ff367bfd3 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:14.566097 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet 0d216948345a4db488fd0a9ff367bfd3 (table default.table_492c567ddda14e77851270f34471730e [id=754c40502e614c5bb18230c852aa0016]) successfully deleted
I20260925 13:57:14.568195 21583 catalog_manager.cc:5838] T 412fd2781eec42eebd232d0c81c644c9 P 5e2731975dbf433aade4a859df9699e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e2731975dbf433aade4a859df9699e2 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "5e2731975dbf433aade4a859df9699e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: UNKNOWN } } }
I20260925 13:57:14.570389 21937 tablet_service.cc:1511] Processing CreateTablet for tablet 1dd5ef2086884c81b48000d7e9de14e2 (DEFAULT_TABLE table=default.table_492c567ddda14e77851270f34471730e [id=754c40502e614c5bb18230c852aa0016]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:14.571364 21937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1dd5ef2086884c81b48000d7e9de14e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:14.571338 21585 catalog_manager.cc:5938] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291) does not have the latest schema for tablet d71049b217a74662b7a9974e776d1448 (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]). Expected version 1 got 0
W20260925 13:57:14.574608 21585 catalog_manager.cc:6038] Failed to send AlterTable RPC for tablet d71049b217a74662b7a9974e776d1448 (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet d71049b217a74662b7a9974e776d1448 consensus state has no leader
I20260925 13:57:14.576885 22083 tablet_service.cc:1511] Processing CreateTablet for tablet 1dd5ef2086884c81b48000d7e9de14e2 (DEFAULT_TABLE table=default.table_492c567ddda14e77851270f34471730e [id=754c40502e614c5bb18230c852aa0016]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:14.577162 21585 catalog_manager.cc:5838] T 52d8da0caee84c8ea79f382564575b94 P ee02f77594864349982492862d668a36 reported cstate change: term changed from 0 to 1, leader changed from <none> to ee02f77594864349982492862d668a36 (127.18.182.195). New cstate: current_term: 1 leader_uuid: "ee02f77594864349982492862d668a36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: HEALTHY } } }
I20260925 13:57:14.577898 22083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1dd5ef2086884c81b48000d7e9de14e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:14.578328 21585 catalog_manager.cc:5938] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873) does not have the latest schema for tablet 52d8da0caee84c8ea79f382564575b94 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 1 got 0
I20260925 13:57:14.581679 21776 tablet_service.cc:1511] Processing CreateTablet for tablet 1dd5ef2086884c81b48000d7e9de14e2 (DEFAULT_TABLE table=default.table_492c567ddda14e77851270f34471730e [id=754c40502e614c5bb18230c852aa0016]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:14.582615 21776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1dd5ef2086884c81b48000d7e9de14e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:14.600858 22293 tablet_bootstrap.cc:492] T 1dd5ef2086884c81b48000d7e9de14e2 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:14.602713 21583 catalog_manager.cc:5938] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881) does not have the latest schema for tablet d71049b217a74662b7a9974e776d1448 (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]). Expected version 1 got 0
I20260925 13:57:14.606026 22293 tablet_bootstrap.cc:654] T 1dd5ef2086884c81b48000d7e9de14e2 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:14.612572 22293 tablet_bootstrap.cc:492] T 1dd5ef2086884c81b48000d7e9de14e2 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:14.612953 22293 ts_tablet_manager.cc:1403] T 1dd5ef2086884c81b48000d7e9de14e2 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.004s
I20260925 13:57:14.618124 21585 catalog_manager.cc:5938] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873) does not have the latest schema for tablet d71049b217a74662b7a9974e776d1448 (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]). Expected version 1 got 0
I20260925 13:57:14.623558 22293 raft_consensus.cc:359] T 1dd5ef2086884c81b48000d7e9de14e2 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:14.520543 22211 ts_tablet_manager.cc:1434] T d71049b217a74662b7a9974e776d1448 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20260925 13:57:14.626224 22293 raft_consensus.cc:385] T 1dd5ef2086884c81b48000d7e9de14e2 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:14.627399 22293 raft_consensus.cc:740] T 1dd5ef2086884c81b48000d7e9de14e2 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:14.631811 22211 tablet_bootstrap.cc:492] T 1dd5ef2086884c81b48000d7e9de14e2 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:14.631150 22293 consensus_queue.cc:260] T 1dd5ef2086884c81b48000d7e9de14e2 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:14.645195 22293 ts_tablet_manager.cc:1434] T 1dd5ef2086884c81b48000d7e9de14e2 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.032s user 0.007s sys 0.003s
I20260925 13:57:14.659190 21585 catalog_manager.cc:5938] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291) does not have the latest schema for tablet d71049b217a74662b7a9974e776d1448 (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]). Expected version 1 got 0
I20260925 13:57:14.667665 21585 master_service.cc:960] ReplaceTablet: received request to replace tablet 412fd2781eec42eebd232d0c81c644c9 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:14.685062 21959 raft_consensus.cc:1275] T 52d8da0caee84c8ea79f382564575b94 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Refusing update from remote peer ee02f77594864349982492862d668a36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:57:14.685458 21797 raft_consensus.cc:1275] T 52d8da0caee84c8ea79f382564575b94 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Refusing update from remote peer ee02f77594864349982492862d668a36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:57:14.687301 22385 consensus_queue.cc:1048] T 52d8da0caee84c8ea79f382564575b94 P ee02f77594864349982492862d668a36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 13:57:14.688230 22226 consensus_queue.cc:1048] T 52d8da0caee84c8ea79f382564575b94 P ee02f77594864349982492862d668a36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:14.695400 21583 master_service.cc:960] ReplaceTablet: received request to replace tablet 412fd2781eec42eebd232d0c81c644c9 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:14.699599 22211 tablet_bootstrap.cc:654] T 1dd5ef2086884c81b48000d7e9de14e2 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:14.713227 22375 raft_consensus.cc:493] T d71049b217a74662b7a9974e776d1448 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:14.713738 22375 raft_consensus.cc:515] T d71049b217a74662b7a9974e776d1448 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:14.715997 22375 leader_election.cc:290] T d71049b217a74662b7a9974e776d1448 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:14.718042 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d71049b217a74662b7a9974e776d1448" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:14.718652 21960 raft_consensus.cc:2468] T d71049b217a74662b7a9974e776d1448 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:14.719980 21732 leader_election.cc:304] T d71049b217a74662b7a9974e776d1448 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:14.721463 22212 tablet_bootstrap.cc:492] T 1dd5ef2086884c81b48000d7e9de14e2 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:14.721370 22105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d71049b217a74662b7a9974e776d1448" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:14.721946 22105 raft_consensus.cc:2468] T d71049b217a74662b7a9974e776d1448 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:14.723460 22375 raft_consensus.cc:2804] T d71049b217a74662b7a9974e776d1448 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:14.723762 22375 raft_consensus.cc:493] T d71049b217a74662b7a9974e776d1448 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:14.724046 22375 raft_consensus.cc:3060] T d71049b217a74662b7a9974e776d1448 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:14.728066 22212 tablet_bootstrap.cc:654] T 1dd5ef2086884c81b48000d7e9de14e2 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:14.732396 21585 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet 412fd2781eec42eebd232d0c81c644c9
I20260925 13:57:14.733773 21585 catalog_manager.cc:6934] ReplaceTablet: tablet 412fd2781eec42eebd232d0c81c644c9 deleted and replaced by tablet ee060e8412974e3dbb41c1b4fd0180a9
I20260925 13:57:14.735507 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet 412fd2781eec42eebd232d0c81c644c9 with delete_type TABLET_DATA_DELETED (replaced by tablet ee060e8412974e3dbb41c1b4fd0180a9) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:14.736758 21937 tablet_service.cc:1558] Processing DeleteTablet for tablet 412fd2781eec42eebd232d0c81c644c9 with delete_type TABLET_DATA_DELETED (replaced by tablet ee060e8412974e3dbb41c1b4fd0180a9) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:14.737476 22215 tablet_replica.cc:333] T 412fd2781eec42eebd232d0c81c644c9 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:14.738170 22215 raft_consensus.cc:2243] T 412fd2781eec42eebd232d0c81c644c9 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:14.738615 22215 raft_consensus.cc:2272] T 412fd2781eec42eebd232d0c81c644c9 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:14.740263 22215 ts_tablet_manager.cc:1916] T 412fd2781eec42eebd232d0c81c644c9 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:14.743582 21776 tablet_service.cc:1558] Processing DeleteTablet for tablet 412fd2781eec42eebd232d0c81c644c9 with delete_type TABLET_DATA_DELETED (replaced by tablet ee060e8412974e3dbb41c1b4fd0180a9) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:14.744225 22213 tablet_replica.cc:333] T 412fd2781eec42eebd232d0c81c644c9 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:14.744889 22213 raft_consensus.cc:2243] T 412fd2781eec42eebd232d0c81c644c9 P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:14.745481 22213 pending_rounds.cc:70] T 412fd2781eec42eebd232d0c81c644c9 P 5e2731975dbf433aade4a859df9699e2: Trying to abort 1 pending ops.
I20260925 13:57:14.745687 22213 pending_rounds.cc:77] T 412fd2781eec42eebd232d0c81c644c9 P 5e2731975dbf433aade4a859df9699e2: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251622858149888 op_type: NO_OP noop_request { }
I20260925 13:57:14.745965 22213 raft_consensus.cc:2889] T 412fd2781eec42eebd232d0c81c644c9 P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:57:14.746238 22213 raft_consensus.cc:2272] T 412fd2781eec42eebd232d0c81c644c9 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:14.746487 22368 consensus_queue.cc:1048] T d595ab2f7dc74fdfbf3546e477685a01 P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:14.748121 22213 ts_tablet_manager.cc:1916] T 412fd2781eec42eebd232d0c81c644c9 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:14.766245 22375 raft_consensus.cc:515] T d71049b217a74662b7a9974e776d1448 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:14.773424 22215 ts_tablet_manager.cc:1929] T 412fd2781eec42eebd232d0c81c644c9 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:14.773793 22215 log.cc:1199] T 412fd2781eec42eebd232d0c81c644c9 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/412fd2781eec42eebd232d0c81c644c9
I20260925 13:57:14.774353 22215 ts_tablet_manager.cc:1950] T 412fd2781eec42eebd232d0c81c644c9 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:14.778277 22214 ts_tablet_manager.cc:1929] T 5888d8f93cec4450847ccfe4854e4012 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:57:14.778638 22214 log.cc:1199] T 5888d8f93cec4450847ccfe4854e4012 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/5888d8f93cec4450847ccfe4854e4012
I20260925 13:57:14.778723 21937 tablet_service.cc:1511] Processing CreateTablet for tablet ee060e8412974e3dbb41c1b4fd0180a9 (DEFAULT_TABLE table=default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:14.779701 21937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee060e8412974e3dbb41c1b4fd0180a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:14.781668 21776 tablet_service.cc:1511] Processing CreateTablet for tablet ee060e8412974e3dbb41c1b4fd0180a9 (DEFAULT_TABLE table=default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:14.782757 21776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee060e8412974e3dbb41c1b4fd0180a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:14.788622 22213 ts_tablet_manager.cc:1929] T 412fd2781eec42eebd232d0c81c644c9 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:14.788961 22213 log.cc:1199] T 412fd2781eec42eebd232d0c81c644c9 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/412fd2781eec42eebd232d0c81c644c9
I20260925 13:57:14.789625 22213 ts_tablet_manager.cc:1950] T 412fd2781eec42eebd232d0c81c644c9 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:14.779275 22214 ts_tablet_manager.cc:1950] T 5888d8f93cec4450847ccfe4854e4012 P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:14.796051 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet 5888d8f93cec4450847ccfe4854e4012 (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]) successfully deleted
I20260925 13:57:14.803530 22083 tablet_service.cc:1511] Processing CreateTablet for tablet ee060e8412974e3dbb41c1b4fd0180a9 (DEFAULT_TABLE table=default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:14.804752 22083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee060e8412974e3dbb41c1b4fd0180a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:14.805778 21584 catalog_manager.cc:2213] created HMS catalog entry for table [id=df6afcfb1e2c463493b14228e19ade26]
I20260925 13:57:14.806703 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet 412fd2781eec42eebd232d0c81c644c9 (table default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]) successfully deleted
I20260925 13:57:14.804836 22214 tablet_replica.cc:333] T 0d216948345a4db488fd0a9ff367bfd3 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:14.812633 22214 raft_consensus.cc:2243] T 0d216948345a4db488fd0a9ff367bfd3 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:14.813066 22214 raft_consensus.cc:2272] T 0d216948345a4db488fd0a9ff367bfd3 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:14.815260 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet 412fd2781eec42eebd232d0c81c644c9 (table default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]) successfully deleted
I20260925 13:57:14.820796 21583 catalog_manager.cc:5938] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291) does not have the latest schema for tablet 52d8da0caee84c8ea79f382564575b94 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 1 got 0
I20260925 13:57:14.834591 21583 catalog_manager.cc:5938] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881) does not have the latest schema for tablet 52d8da0caee84c8ea79f382564575b94 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 1 got 0
I20260925 13:57:14.843853 21584 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35994 (request call id 9) took 5970 ms. Trace:
I20260925 13:57:14.845180 21584 rpcz_store.cc:276] 0925 13:57:08.873452 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:08.873613 (+ 161us) service_pool.cc:224] Handling call
0925 13:57:08.884865 (+ 11252us) spinlock_profiling.cc:243] Waited 1.38 ms on lock 0x7b54000203d4. stack: 00007fd96d084323 00007fd96d084220 00007fd96cc28574 0000000000516964 00000000005168f9 00007fd9741825c0 00007fd9742216fa 00007fd975f92d8c 00007fd976092566 00007fd973c3d034 00007fd973c3cfc2 00007fd973c3cef1 00007fd973c3ce6c 00007fd973c3c122 00007fd971ee709c 00007fd971ee64d6
0925 13:57:10.830665 (+1945800us) catalog_manager.cc:2140] Acquired catalog manager lock
0925 13:57:10.833668 (+ 3003us) catalog_manager.cc:2186] Created new table and tablet info
0925 13:57:14.805738 (+3972070us) catalog_manager.cc:2212] Created new table in HMS catalog
0925 13:57:14.843107 (+ 37369us) catalog_manager.cc:2246] Wrote table and tablets to system table
0925 13:57:14.843472 (+ 365us) catalog_manager.cc:2280] Inserted table and tablets into CatalogManager maps
0925 13:57:14.843774 (+ 302us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:57:14.808075 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:57:14.808445 (+ 370us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:57:14.808467 (+ 22us) write_op.cc:435] Acquired schema lock
0925 13:57:14.808479 (+ 12us) tablet.cc:662] Decoding operations
0925 13:57:14.808746 (+ 267us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:57:14.808784 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0925 13:57:14.808805 (+ 21us) tablet.cc:685] Acquiring locks for 2 operations
0925 13:57:14.808987 (+ 182us) tablet.cc:701] Row locks acquired
0925 13:57:14.809000 (+ 13us) write_op.cc:260] PREPARE: finished
0925 13:57:14.809106 (+ 106us) write_op.cc:270] Start()
0925 13:57:14.809181 (+ 75us) write_op.cc:276] Timestamp: P: 1790344634809060 usec, L: 0
0925 13:57:14.809195 (+ 14us) op_driver.cc:348] REPLICATION: starting
0925 13:57:14.809638 (+ 443us) log.cc:844] Serialized 432 byte log entry
0925 13:57:14.840520 (+ 30882us) op_driver.cc:464] REPLICATION: finished
0925 13:57:14.841073 (+ 553us) write_op.cc:301] APPLY: starting
0925 13:57:14.841148 (+ 75us) tablet.cc:1394] starting BulkCheckPresence
0925 13:57:14.841283 (+ 135us) tablet.cc:1397] finished BulkCheckPresence
0925 13:57:14.841390 (+ 107us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:57:14.841833 (+ 443us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:57:14.841973 (+ 140us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 13:57:14.842018 (+ 45us) write_op.cc:312] APPLY: finished
0925 13:57:14.842193 (+ 175us) log.cc:844] Serialized 28 byte log entry
0925 13:57:14.842324 (+ 131us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:14.842392 (+ 68us) write_op.cc:454] Released schema lock
0925 13:57:14.842422 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0925 13:57:14.844286 (+ 1864us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:14.844310 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":3195850,"Hive Metastore.run_cpu_time_us":767,"Hive Metastore.run_wall_time_us":774808,"spinlock_wait_cycles":3173760,"child_traces":[["op",{"apply.queue_time_us":349,"apply.run_cpu_time_us":1566,"apply.run_wall_time_us":3196,"num_ops":2,"prepare.queue_time_us":714,"prepare.run_cpu_time_us":1874,"prepare.run_wall_time_us":2627,"raft.queue_time_us":271,"raft.run_cpu_time_us":573,"raft.run_wall_time_us":573,"replication_time_us":31197}]]}
I20260925 13:57:14.850360 22385 raft_consensus.cc:493] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:14.850805 22385 raft_consensus.cc:515] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:14.855033 22040 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
W20260925 13:57:14.858052 22043 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
I20260925 13:57:14.858464 22043 leader_election.cc:304] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:14.859313 22387 raft_consensus.cc:2749] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:57:14.860033 22385 leader_election.cc:290] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:14.862519 22212 tablet_bootstrap.cc:492] T 1dd5ef2086884c81b48000d7e9de14e2 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:14.863080 22212 ts_tablet_manager.cc:1403] T 1dd5ef2086884c81b48000d7e9de14e2 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.142s user 0.009s sys 0.005s
I20260925 13:57:14.865768 22212 raft_consensus.cc:359] T 1dd5ef2086884c81b48000d7e9de14e2 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:14.899616 21583 catalog_manager.cc:5938] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873) does not have the latest schema for tablet 52d8da0caee84c8ea79f382564575b94 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 1 got 0
I20260925 13:57:14.900789 22375 leader_election.cc:290] T d71049b217a74662b7a9974e776d1448 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Requested vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:14.910432 22293 tablet_bootstrap.cc:492] T ee060e8412974e3dbb41c1b4fd0180a9 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:14.915647 22293 tablet_bootstrap.cc:654] T ee060e8412974e3dbb41c1b4fd0180a9 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:14.924551 22402 tablet.cc:1751] T 52d8da0caee84c8ea79f382564575b94 P 5e2731975dbf433aade4a859df9699e2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:57:14.928012 22211 tablet_bootstrap.cc:492] T 1dd5ef2086884c81b48000d7e9de14e2 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:14.928457 22211 ts_tablet_manager.cc:1403] T 1dd5ef2086884c81b48000d7e9de14e2 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.297s user 0.021s sys 0.027s
I20260925 13:57:14.928992 22212 raft_consensus.cc:385] T 1dd5ef2086884c81b48000d7e9de14e2 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:14.930087 22212 raft_consensus.cc:740] T 1dd5ef2086884c81b48000d7e9de14e2 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:14.930260 22293 tablet_bootstrap.cc:492] T ee060e8412974e3dbb41c1b4fd0180a9 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:14.930640 22293 ts_tablet_manager.cc:1403] T ee060e8412974e3dbb41c1b4fd0180a9 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.020s user 0.009s sys 0.000s
I20260925 13:57:14.930943 22212 consensus_queue.cc:260] T 1dd5ef2086884c81b48000d7e9de14e2 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:14.902843 22105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d71049b217a74662b7a9974e776d1448" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:14.932090 22105 raft_consensus.cc:3060] T d71049b217a74662b7a9974e776d1448 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:14.863039 22214 ts_tablet_manager.cc:1916] T 0d216948345a4db488fd0a9ff367bfd3 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:14.933249 22293 raft_consensus.cc:359] T ee060e8412974e3dbb41c1b4fd0180a9 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:14.933826 22293 raft_consensus.cc:385] T ee060e8412974e3dbb41c1b4fd0180a9 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:14.934067 22293 raft_consensus.cc:740] T ee060e8412974e3dbb41c1b4fd0180a9 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:14.930943 22211 raft_consensus.cc:359] T 1dd5ef2086884c81b48000d7e9de14e2 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:14.936868 22211 raft_consensus.cc:385] T 1dd5ef2086884c81b48000d7e9de14e2 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:14.937135 22211 raft_consensus.cc:740] T 1dd5ef2086884c81b48000d7e9de14e2 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:14.937764 22211 consensus_queue.cc:260] T 1dd5ef2086884c81b48000d7e9de14e2 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:14.943219 21574 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet d71049b217a74662b7a9974e776d1448 (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet d71049b217a74662b7a9974e776d1448 consensus state has no leader
I20260925 13:57:14.946058 22211 ts_tablet_manager.cc:1434] T 1dd5ef2086884c81b48000d7e9de14e2 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.017s user 0.006s sys 0.000s
I20260925 13:57:14.946354 22293 consensus_queue.cc:260] T ee060e8412974e3dbb41c1b4fd0180a9 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:14.946856 22211 tablet_bootstrap.cc:492] T ee060e8412974e3dbb41c1b4fd0180a9 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:14.948233 22293 ts_tablet_manager.cc:1434] T ee060e8412974e3dbb41c1b4fd0180a9 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.017s user 0.006s sys 0.001s
I20260925 13:57:14.960163 22212 ts_tablet_manager.cc:1434] T 1dd5ef2086884c81b48000d7e9de14e2 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.097s user 0.006s sys 0.000s
I20260925 13:57:14.961153 22212 tablet_bootstrap.cc:492] T ee060e8412974e3dbb41c1b4fd0180a9 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:14.970242 22105 raft_consensus.cc:2468] T d71049b217a74662b7a9974e776d1448 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:14.972183 21776 tablet_service.cc:1511] Processing CreateTablet for tablet 4ee62073ffa04bd39c688166cfd70677 (DEFAULT_TABLE table=default.table_b17685eaeada404790103bd89711435e [id=df6afcfb1e2c463493b14228e19ade26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:14.972847 22211 tablet_bootstrap.cc:654] T ee060e8412974e3dbb41c1b4fd0180a9 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:14.973493 21776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ee62073ffa04bd39c688166cfd70677. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:14.974224 21583 catalog_manager.cc:5938] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873) does not have the latest schema for tablet 52d8da0caee84c8ea79f382564575b94 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 1 got 0
I20260925 13:57:14.988579 22211 tablet_bootstrap.cc:492] T ee060e8412974e3dbb41c1b4fd0180a9 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:14.988972 22211 ts_tablet_manager.cc:1403] T ee060e8412974e3dbb41c1b4fd0180a9 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.042s user 0.005s sys 0.003s
I20260925 13:57:14.996229 22399 raft_consensus.cc:493] T 1dd5ef2086884c81b48000d7e9de14e2 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:14.996734 22399 raft_consensus.cc:515] T 1dd5ef2086884c81b48000d7e9de14e2 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:14.998333 22211 raft_consensus.cc:359] T ee060e8412974e3dbb41c1b4fd0180a9 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:14.998800 22399 leader_election.cc:290] T 1dd5ef2086884c81b48000d7e9de14e2 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:15.000464 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dd5ef2086884c81b48000d7e9de14e2" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2" is_pre_election: true
I20260925 13:57:15.001149 21796 raft_consensus.cc:2468] T 1dd5ef2086884c81b48000d7e9de14e2 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 0.
I20260925 13:57:15.002473 21893 leader_election.cc:304] T 1dd5ef2086884c81b48000d7e9de14e2 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:14.998916 22211 raft_consensus.cc:385] T ee060e8412974e3dbb41c1b4fd0180a9 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:15.004822 22211 raft_consensus.cc:740] T ee060e8412974e3dbb41c1b4fd0180a9 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:15.005426 22211 consensus_queue.cc:260] T ee060e8412974e3dbb41c1b4fd0180a9 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:15.004531 22399 raft_consensus.cc:2804] T 1dd5ef2086884c81b48000d7e9de14e2 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:15.006316 22399 raft_consensus.cc:493] T 1dd5ef2086884c81b48000d7e9de14e2 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:15.006669 22399 raft_consensus.cc:3060] T 1dd5ef2086884c81b48000d7e9de14e2 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:15.012449 21959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d71049b217a74662b7a9974e776d1448" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
I20260925 13:57:15.013296 21959 raft_consensus.cc:3060] T d71049b217a74662b7a9974e776d1448 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:14.964244 22083 tablet_service.cc:1511] Processing CreateTablet for tablet 4ee62073ffa04bd39c688166cfd70677 (DEFAULT_TABLE table=default.table_b17685eaeada404790103bd89711435e [id=df6afcfb1e2c463493b14228e19ade26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:15.005530 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dd5ef2086884c81b48000d7e9de14e2" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:15.019717 22107 raft_consensus.cc:2468] T 1dd5ef2086884c81b48000d7e9de14e2 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 0.
I20260925 13:57:15.020463 21732 leader_election.cc:304] T d71049b217a74662b7a9974e776d1448 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:15.021248 22375 raft_consensus.cc:2804] T d71049b217a74662b7a9974e776d1448 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:15.021678 22375 raft_consensus.cc:697] T d71049b217a74662b7a9974e776d1448 P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Becoming Leader. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Running, Role: LEADER
I20260925 13:57:15.022121 22083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ee62073ffa04bd39c688166cfd70677. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:15.022397 22375 consensus_queue.cc:237] T d71049b217a74662b7a9974e776d1448 P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:14.892313 22400 tablet.cc:1751] T 52d8da0caee84c8ea79f382564575b94 P ee02f77594864349982492862d668a36: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:57:15.054738 21583 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35994:
name: "default.Table_e0eb6c1d49224d1cb9b4f6d06f0f7a29"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:57:15.026767 22399 raft_consensus.cc:515] T 1dd5ef2086884c81b48000d7e9de14e2 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:15.038084 22401 consensus_queue.cc:1048] T d595ab2f7dc74fdfbf3546e477685a01 P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:15.044216 22211 ts_tablet_manager.cc:1434] T ee060e8412974e3dbb41c1b4fd0180a9 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.055s user 0.007s sys 0.007s
I20260925 13:57:15.058976 22212 tablet_bootstrap.cc:654] T ee060e8412974e3dbb41c1b4fd0180a9 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:15.096051 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dd5ef2086884c81b48000d7e9de14e2" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2"
I20260925 13:57:15.096587 21796 raft_consensus.cc:3060] T 1dd5ef2086884c81b48000d7e9de14e2 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:15.103963 21796 raft_consensus.cc:2468] T 1dd5ef2086884c81b48000d7e9de14e2 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 1.
I20260925 13:57:15.105618 21893 leader_election.cc:304] T 1dd5ef2086884c81b48000d7e9de14e2 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:15.108191 22403 tablet.cc:1751] T 52d8da0caee84c8ea79f382564575b94 P 145e520e0a5a445d943a53cf8c27efee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:57:15.109850 22368 raft_consensus.cc:2804] T 1dd5ef2086884c81b48000d7e9de14e2 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:15.115662 21584 catalog_manager.cc:5938] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291) does not have the latest schema for tablet 52d8da0caee84c8ea79f382564575b94 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 1 got 0
I20260925 13:57:15.119576 22212 tablet_bootstrap.cc:492] T ee060e8412974e3dbb41c1b4fd0180a9 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:15.119959 22212 ts_tablet_manager.cc:1403] T ee060e8412974e3dbb41c1b4fd0180a9 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.159s user 0.007s sys 0.002s
I20260925 13:57:15.122339 22212 raft_consensus.cc:359] T ee060e8412974e3dbb41c1b4fd0180a9 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:15.122925 22212 raft_consensus.cc:385] T ee060e8412974e3dbb41c1b4fd0180a9 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:15.131632 22214 ts_tablet_manager.cc:1929] T 0d216948345a4db488fd0a9ff367bfd3 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:15.131983 22214 log.cc:1199] T 0d216948345a4db488fd0a9ff367bfd3 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/0d216948345a4db488fd0a9ff367bfd3
I20260925 13:57:15.132817 22214 ts_tablet_manager.cc:1950] T 0d216948345a4db488fd0a9ff367bfd3 P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:15.135375 21584 master_service.cc:960] ReplaceTablet: received request to replace tablet d267def3f0864435a0d2c40fd7052eab from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:15.145211 21585 master_service.cc:960] ReplaceTablet: received request to replace tablet d267def3f0864435a0d2c40fd7052eab from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:15.152951 21584 catalog_manager.cc:5938] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881) does not have the latest schema for tablet 52d8da0caee84c8ea79f382564575b94 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 1 got 0
I20260925 13:57:15.157212 22368 raft_consensus.cc:697] T 1dd5ef2086884c81b48000d7e9de14e2 P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: Becoming Leader. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Running, Role: LEADER
I20260925 13:57:15.158721 22293 tablet_bootstrap.cc:492] T 4ee62073ffa04bd39c688166cfd70677 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:15.158195 22368 consensus_queue.cc:237] T 1dd5ef2086884c81b48000d7e9de14e2 P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:15.159847 22402 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.234s user 0.001s sys 0.000s
I20260925 13:57:15.164302 22293 tablet_bootstrap.cc:654] T 4ee62073ffa04bd39c688166cfd70677 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:15.164793 22399 leader_election.cc:290] T 1dd5ef2086884c81b48000d7e9de14e2 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Requested vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:15.112380 21937 tablet_service.cc:1511] Processing CreateTablet for tablet 4ee62073ffa04bd39c688166cfd70677 (DEFAULT_TABLE table=default.table_b17685eaeada404790103bd89711435e [id=df6afcfb1e2c463493b14228e19ade26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:15.165246 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dd5ef2086884c81b48000d7e9de14e2" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:15.166216 22107 raft_consensus.cc:3060] T 1dd5ef2086884c81b48000d7e9de14e2 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:15.166467 21937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ee62073ffa04bd39c688166cfd70677. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:15.167984 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet 0d216948345a4db488fd0a9ff367bfd3 (table default.table_492c567ddda14e77851270f34471730e [id=754c40502e614c5bb18230c852aa0016]) successfully deleted
I20260925 13:57:15.176180 22107 raft_consensus.cc:2468] T 1dd5ef2086884c81b48000d7e9de14e2 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 1.
I20260925 13:57:15.123196 22212 raft_consensus.cc:740] T ee060e8412974e3dbb41c1b4fd0180a9 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:15.178493 22212 consensus_queue.cc:260] T ee060e8412974e3dbb41c1b4fd0180a9 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:15.180868 22293 tablet_bootstrap.cc:492] T 4ee62073ffa04bd39c688166cfd70677 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:15.181380 22293 ts_tablet_manager.cc:1403] T 4ee62073ffa04bd39c688166cfd70677 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.023s user 0.012s sys 0.000s
I20260925 13:57:15.181823 22212 ts_tablet_manager.cc:1434] T ee060e8412974e3dbb41c1b4fd0180a9 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.062s user 0.006s sys 0.000s
I20260925 13:57:15.184273 22293 raft_consensus.cc:359] T 4ee62073ffa04bd39c688166cfd70677 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:15.184850 22293 raft_consensus.cc:385] T 4ee62073ffa04bd39c688166cfd70677 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:15.185092 22293 raft_consensus.cc:740] T 4ee62073ffa04bd39c688166cfd70677 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:15.185010 22214 tablet_replica.cc:333] T 412fd2781eec42eebd232d0c81c644c9 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:15.187224 22214 raft_consensus.cc:2243] T 412fd2781eec42eebd232d0c81c644c9 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:15.190822 22293 consensus_queue.cc:260] T 4ee62073ffa04bd39c688166cfd70677 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:15.191926 22214 raft_consensus.cc:2272] T 412fd2781eec42eebd232d0c81c644c9 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:15.196324 22293 ts_tablet_manager.cc:1434] T 4ee62073ffa04bd39c688166cfd70677 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.015s user 0.008s sys 0.000s
I20260925 13:57:15.209244 22214 ts_tablet_manager.cc:1916] T 412fd2781eec42eebd232d0c81c644c9 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:15.209892 22212 tablet_bootstrap.cc:492] T 4ee62073ffa04bd39c688166cfd70677 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:15.238324 22212 tablet_bootstrap.cc:654] T 4ee62073ffa04bd39c688166cfd70677 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:15.237561 21585 catalog_manager.cc:5838] T d71049b217a74662b7a9974e776d1448 P 5e2731975dbf433aade4a859df9699e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e2731975dbf433aade4a859df9699e2 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "5e2731975dbf433aade4a859df9699e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: UNKNOWN } } }
I20260925 13:57:15.238909 21585 catalog_manager.cc:5938] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881) does not have the latest schema for tablet d71049b217a74662b7a9974e776d1448 (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]). Expected version 1 got 0
I20260925 13:57:15.244738 22375 raft_consensus.cc:493] T ee060e8412974e3dbb41c1b4fd0180a9 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:15.245155 22375 raft_consensus.cc:515] T ee060e8412974e3dbb41c1b4fd0180a9 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:15.247012 22375 leader_election.cc:290] T ee060e8412974e3dbb41c1b4fd0180a9 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:15.251334 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee060e8412974e3dbb41c1b4fd0180a9" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:15.251959 21960 raft_consensus.cc:2468] T ee060e8412974e3dbb41c1b4fd0180a9 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:15.253757 21732 leader_election.cc:304] T ee060e8412974e3dbb41c1b4fd0180a9 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:15.254890 22375 raft_consensus.cc:2804] T ee060e8412974e3dbb41c1b4fd0180a9 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:15.255335 22375 raft_consensus.cc:493] T ee060e8412974e3dbb41c1b4fd0180a9 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:15.255760 22375 raft_consensus.cc:3060] T ee060e8412974e3dbb41c1b4fd0180a9 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:15.260257 22105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee060e8412974e3dbb41c1b4fd0180a9" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:15.260852 22105 raft_consensus.cc:2468] T ee060e8412974e3dbb41c1b4fd0180a9 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:15.238433 21959 raft_consensus.cc:2468] T d71049b217a74662b7a9974e776d1448 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:15.285547 21581 catalog_manager.cc:2213] created HMS catalog entry for table [id=dfd05c11872140cfa86b2628c33ef6d8]
I20260925 13:57:15.288869 22375 raft_consensus.cc:515] T ee060e8412974e3dbb41c1b4fd0180a9 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:15.299419 21585 catalog_manager.cc:5838] T 1dd5ef2086884c81b48000d7e9de14e2 P 145e520e0a5a445d943a53cf8c27efee reported cstate change: term changed from 0 to 1, leader changed from <none> to 145e520e0a5a445d943a53cf8c27efee (127.18.182.194). New cstate: current_term: 1 leader_uuid: "145e520e0a5a445d943a53cf8c27efee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: UNKNOWN } } }
I20260925 13:57:15.304414 21581 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35994 (request call id 12) took 6347 ms. Trace:
I20260925 13:57:15.307210 22214 ts_tablet_manager.cc:1929] T 412fd2781eec42eebd232d0c81c644c9 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:15.307695 22214 log.cc:1199] T 412fd2781eec42eebd232d0c81c644c9 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/412fd2781eec42eebd232d0c81c644c9
I20260925 13:57:15.308400 22214 ts_tablet_manager.cc:1950] T 412fd2781eec42eebd232d0c81c644c9 P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:15.311048 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet 412fd2781eec42eebd232d0c81c644c9 (table default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]) successfully deleted
I20260925 13:57:15.304852 21581 rpcz_store.cc:276] 0925 13:57:08.956700 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:08.956922 (+ 222us) service_pool.cc:224] Handling call
0925 13:57:10.837291 (+1880369us) catalog_manager.cc:2140] Acquired catalog manager lock
0925 13:57:10.840290 (+ 2999us) catalog_manager.cc:2186] Created new table and tablet info
0925 13:57:15.285482 (+4445192us) catalog_manager.cc:2212] Created new table in HMS catalog
0925 13:57:15.303513 (+ 18031us) catalog_manager.cc:2246] Wrote table and tablets to system table
0925 13:57:15.303953 (+ 440us) catalog_manager.cc:2280] Inserted table and tablets into CatalogManager maps
0925 13:57:15.304337 (+ 384us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:57:15.288328 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:57:15.288911 (+ 583us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:57:15.288947 (+ 36us) write_op.cc:435] Acquired schema lock
0925 13:57:15.288969 (+ 22us) tablet.cc:662] Decoding operations
0925 13:57:15.289364 (+ 395us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:57:15.289425 (+ 61us) write_op.cc:641] Partition lock acquired for write op
0925 13:57:15.289459 (+ 34us) tablet.cc:685] Acquiring locks for 2 operations
0925 13:57:15.289759 (+ 300us) tablet.cc:701] Row locks acquired
0925 13:57:15.289777 (+ 18us) write_op.cc:260] PREPARE: finished
0925 13:57:15.289913 (+ 136us) write_op.cc:270] Start()
0925 13:57:15.289984 (+ 71us) write_op.cc:276] Timestamp: P: 1790344635289862 usec, L: 0
0925 13:57:15.289998 (+ 14us) op_driver.cc:348] REPLICATION: starting
0925 13:57:15.290450 (+ 452us) log.cc:844] Serialized 432 byte log entry
0925 13:57:15.299654 (+ 9204us) op_driver.cc:464] REPLICATION: finished
0925 13:57:15.300020 (+ 366us) write_op.cc:301] APPLY: starting
0925 13:57:15.300071 (+ 51us) tablet.cc:1394] starting BulkCheckPresence
0925 13:57:15.300166 (+ 95us) tablet.cc:1397] finished BulkCheckPresence
0925 13:57:15.300246 (+ 80us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:57:15.300632 (+ 386us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:57:15.300756 (+ 124us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 13:57:15.300796 (+ 40us) write_op.cc:312] APPLY: finished
0925 13:57:15.300951 (+ 155us) log.cc:844] Serialized 28 byte log entry
0925 13:57:15.301127 (+ 176us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:15.301192 (+ 65us) write_op.cc:454] Released schema lock
0925 13:57:15.301225 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0925 13:57:15.301429 (+ 204us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:15.301452 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":3964225,"Hive Metastore.run_cpu_time_us":684,"Hive Metastore.run_wall_time_us":479318,"child_traces":[["op",{"apply.queue_time_us":235,"apply.run_cpu_time_us":1368,"apply.run_wall_time_us":1368,"num_ops":2,"prepare.queue_time_us":150,"prepare.run_cpu_time_us":2525,"prepare.run_wall_time_us":2526,"raft.queue_time_us":2550,"raft.run_cpu_time_us":668,"raft.run_wall_time_us":668,"replication_time_us":9575}]]}
I20260925 13:57:15.315527 22212 tablet_bootstrap.cc:492] T 4ee62073ffa04bd39c688166cfd70677 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:15.315924 22212 ts_tablet_manager.cc:1403] T 4ee62073ffa04bd39c688166cfd70677 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.106s user 0.008s sys 0.004s
I20260925 13:57:15.318456 21959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee060e8412974e3dbb41c1b4fd0180a9" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
I20260925 13:57:15.319275 22212 raft_consensus.cc:359] T 4ee62073ffa04bd39c688166cfd70677 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:15.320034 22212 raft_consensus.cc:385] T 4ee62073ffa04bd39c688166cfd70677 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:15.319254 21959 raft_consensus.cc:3060] T ee060e8412974e3dbb41c1b4fd0180a9 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:15.320291 22212 raft_consensus.cc:740] T 4ee62073ffa04bd39c688166cfd70677 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:15.320966 22212 consensus_queue.cc:260] T 4ee62073ffa04bd39c688166cfd70677 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:15.342165 22105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee060e8412974e3dbb41c1b4fd0180a9" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:15.342660 22105 raft_consensus.cc:3060] T ee060e8412974e3dbb41c1b4fd0180a9 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:15.343776 21959 raft_consensus.cc:2468] T ee060e8412974e3dbb41c1b4fd0180a9 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:15.348049 21584 master_service.cc:960] ReplaceTablet: received request to replace tablet ee060e8412974e3dbb41c1b4fd0180a9 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:15.347896 21732 leader_election.cc:304] T ee060e8412974e3dbb41c1b4fd0180a9 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:15.348959 22378 raft_consensus.cc:2804] T ee060e8412974e3dbb41c1b4fd0180a9 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:15.351270 22105 raft_consensus.cc:2468] T ee060e8412974e3dbb41c1b4fd0180a9 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:15.353405 22375 leader_election.cc:290] T ee060e8412974e3dbb41c1b4fd0180a9 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Requested vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:15.366060 22212 ts_tablet_manager.cc:1434] T 4ee62073ffa04bd39c688166cfd70677 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.050s user 0.006s sys 0.000s
I20260925 13:57:15.376936 21584 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet ee060e8412974e3dbb41c1b4fd0180a9
I20260925 13:57:15.379766 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet ee060e8412974e3dbb41c1b4fd0180a9 with delete_type TABLET_DATA_DELETED (replaced by tablet 2cca0cb7cfcc41f8aee144493bd6991a) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:15.379848 22211 tablet_bootstrap.cc:492] T 4ee62073ffa04bd39c688166cfd70677 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:15.380353 21584 catalog_manager.cc:6934] ReplaceTablet: tablet ee060e8412974e3dbb41c1b4fd0180a9 deleted and replaced by tablet 2cca0cb7cfcc41f8aee144493bd6991a
I20260925 13:57:15.381481 21776 tablet_service.cc:1558] Processing DeleteTablet for tablet ee060e8412974e3dbb41c1b4fd0180a9 with delete_type TABLET_DATA_DELETED (replaced by tablet 2cca0cb7cfcc41f8aee144493bd6991a) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:15.386998 22211 tablet_bootstrap.cc:654] T 4ee62073ffa04bd39c688166cfd70677 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:15.388249 21937 tablet_service.cc:1558] Processing DeleteTablet for tablet ee060e8412974e3dbb41c1b4fd0180a9 with delete_type TABLET_DATA_DELETED (replaced by tablet 2cca0cb7cfcc41f8aee144493bd6991a) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:15.399395 22214 tablet_replica.cc:333] T ee060e8412974e3dbb41c1b4fd0180a9 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:15.400303 22214 raft_consensus.cc:2243] T ee060e8412974e3dbb41c1b4fd0180a9 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:15.400818 22214 raft_consensus.cc:2272] T ee060e8412974e3dbb41c1b4fd0180a9 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:15.408474 22416 tablet_replica.cc:333] T ee060e8412974e3dbb41c1b4fd0180a9 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:15.409540 22416 raft_consensus.cc:2243] T ee060e8412974e3dbb41c1b4fd0180a9 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:15.411033 22214 ts_tablet_manager.cc:1916] T ee060e8412974e3dbb41c1b4fd0180a9 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:15.413301 21937 tablet_service.cc:1511] Processing CreateTablet for tablet 044b30ad37d74f3d8aca8cd26520b035 (DEFAULT_TABLE table=default.table_bd714c618f7a4f16a69a69d5932b2014 [id=dfd05c11872140cfa86b2628c33ef6d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:15.414348 21937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 044b30ad37d74f3d8aca8cd26520b035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:15.416098 21777 tablet_service.cc:1511] Processing CreateTablet for tablet 044b30ad37d74f3d8aca8cd26520b035 (DEFAULT_TABLE table=default.table_bd714c618f7a4f16a69a69d5932b2014 [id=dfd05c11872140cfa86b2628c33ef6d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:15.417135 21777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 044b30ad37d74f3d8aca8cd26520b035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:15.418453 22083 tablet_service.cc:1511] Processing CreateTablet for tablet 044b30ad37d74f3d8aca8cd26520b035 (DEFAULT_TABLE table=default.table_bd714c618f7a4f16a69a69d5932b2014 [id=dfd05c11872140cfa86b2628c33ef6d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:15.419389 22083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 044b30ad37d74f3d8aca8cd26520b035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:15.420472 22415 tablet_replica.cc:333] T ee060e8412974e3dbb41c1b4fd0180a9 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:15.429026 22416 raft_consensus.cc:2272] T ee060e8412974e3dbb41c1b4fd0180a9 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:15.437083 22416 ts_tablet_manager.cc:1916] T ee060e8412974e3dbb41c1b4fd0180a9 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:15.425235 22378 raft_consensus.cc:697] T ee060e8412974e3dbb41c1b4fd0180a9 P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Becoming Leader. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Running, Role: LEADER
I20260925 13:57:15.438242 22378 consensus_queue.cc:237] T ee060e8412974e3dbb41c1b4fd0180a9 P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:15.449254 22212 tablet_bootstrap.cc:492] T 044b30ad37d74f3d8aca8cd26520b035 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:15.453997 22212 tablet_bootstrap.cc:654] T 044b30ad37d74f3d8aca8cd26520b035 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:15.437220 22211 tablet_bootstrap.cc:492] T 4ee62073ffa04bd39c688166cfd70677 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:15.454490 22211 ts_tablet_manager.cc:1403] T 4ee62073ffa04bd39c688166cfd70677 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.075s user 0.006s sys 0.007s
I20260925 13:57:15.461544 22211 raft_consensus.cc:359] T 4ee62073ffa04bd39c688166cfd70677 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:15.465301 22293 tablet_bootstrap.cc:492] T 044b30ad37d74f3d8aca8cd26520b035 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:15.470705 22293 tablet_bootstrap.cc:654] T 044b30ad37d74f3d8aca8cd26520b035 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:15.475453 22211 raft_consensus.cc:385] T 4ee62073ffa04bd39c688166cfd70677 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:15.475792 22211 raft_consensus.cc:740] T 4ee62073ffa04bd39c688166cfd70677 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:15.476397 21585 master_service.cc:960] ReplaceTablet: received request to replace tablet 1dd5ef2086884c81b48000d7e9de14e2 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:15.477229 22211 consensus_queue.cc:260] T 4ee62073ffa04bd39c688166cfd70677 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:15.480386 22415 raft_consensus.cc:2243] T ee060e8412974e3dbb41c1b4fd0180a9 P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:15.485121 22415 pending_rounds.cc:70] T ee060e8412974e3dbb41c1b4fd0180a9 P 5e2731975dbf433aade4a859df9699e2: Trying to abort 1 pending ops.
I20260925 13:57:15.485384 22415 pending_rounds.cc:77] T ee060e8412974e3dbb41c1b4fd0180a9 P 5e2731975dbf433aade4a859df9699e2: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251626771693568 op_type: NO_OP noop_request { }
I20260925 13:57:15.488428 22211 ts_tablet_manager.cc:1434] T 4ee62073ffa04bd39c688166cfd70677 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.029s user 0.002s sys 0.002s
I20260925 13:57:15.485765 22415 raft_consensus.cc:2889] T ee060e8412974e3dbb41c1b4fd0180a9 P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:57:15.488395 22293 tablet_bootstrap.cc:492] T 044b30ad37d74f3d8aca8cd26520b035 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:15.491590 22415 raft_consensus.cc:2272] T ee060e8412974e3dbb41c1b4fd0180a9 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:15.491716 22293 ts_tablet_manager.cc:1403] T 044b30ad37d74f3d8aca8cd26520b035 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.027s user 0.006s sys 0.011s
I20260925 13:57:15.492559 22211 tablet_bootstrap.cc:492] T 044b30ad37d74f3d8aca8cd26520b035 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:15.494094 22293 raft_consensus.cc:359] T 044b30ad37d74f3d8aca8cd26520b035 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:15.494657 22293 raft_consensus.cc:385] T 044b30ad37d74f3d8aca8cd26520b035 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:15.494895 22293 raft_consensus.cc:740] T 044b30ad37d74f3d8aca8cd26520b035 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:15.495606 22293 consensus_queue.cc:260] T 044b30ad37d74f3d8aca8cd26520b035 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:15.497438 22293 ts_tablet_manager.cc:1434] T 044b30ad37d74f3d8aca8cd26520b035 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20260925 13:57:15.516944 21585 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet 1dd5ef2086884c81b48000d7e9de14e2
I20260925 13:57:15.519829 21777 tablet_service.cc:1558] Processing DeleteTablet for tablet 1dd5ef2086884c81b48000d7e9de14e2 with delete_type TABLET_DATA_DELETED (replaced by tablet 88f885774038498681d2710e51ff3040) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:15.520978 21585 catalog_manager.cc:6934] ReplaceTablet: tablet 1dd5ef2086884c81b48000d7e9de14e2 deleted and replaced by tablet 88f885774038498681d2710e51ff3040
I20260925 13:57:15.521066 21937 tablet_service.cc:1558] Processing DeleteTablet for tablet 1dd5ef2086884c81b48000d7e9de14e2 with delete_type TABLET_DATA_DELETED (replaced by tablet 88f885774038498681d2710e51ff3040) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:15.521826 21585 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35994:
name: "default.Table_fe2e32c0275e4744b1163ae7b22eaf51"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:57:15.530894 22415 ts_tablet_manager.cc:1916] T ee060e8412974e3dbb41c1b4fd0180a9 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:15.532994 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet 1dd5ef2086884c81b48000d7e9de14e2 with delete_type TABLET_DATA_DELETED (replaced by tablet 88f885774038498681d2710e51ff3040) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:15.535940 21777 tablet_service.cc:1511] Processing CreateTablet for tablet 2cca0cb7cfcc41f8aee144493bd6991a (DEFAULT_TABLE table=default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:15.537207 21777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cca0cb7cfcc41f8aee144493bd6991a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:15.540298 21937 tablet_service.cc:1511] Processing CreateTablet for tablet 2cca0cb7cfcc41f8aee144493bd6991a (DEFAULT_TABLE table=default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:15.541306 21937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cca0cb7cfcc41f8aee144493bd6991a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:15.546321 22083 tablet_service.cc:1511] Processing CreateTablet for tablet 2cca0cb7cfcc41f8aee144493bd6991a (DEFAULT_TABLE table=default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:15.547281 22083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cca0cb7cfcc41f8aee144493bd6991a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:15.553694 22293 tablet_bootstrap.cc:492] T 2cca0cb7cfcc41f8aee144493bd6991a P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:15.558235 22416 ts_tablet_manager.cc:1929] T ee060e8412974e3dbb41c1b4fd0180a9 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:15.558621 22416 log.cc:1199] T ee060e8412974e3dbb41c1b4fd0180a9 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/ee060e8412974e3dbb41c1b4fd0180a9
I20260925 13:57:15.560518 22293 tablet_bootstrap.cc:654] T 2cca0cb7cfcc41f8aee144493bd6991a P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:15.560683 22211 tablet_bootstrap.cc:654] T 044b30ad37d74f3d8aca8cd26520b035 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:15.574210 22293 tablet_bootstrap.cc:492] T 2cca0cb7cfcc41f8aee144493bd6991a P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:15.574577 22293 ts_tablet_manager.cc:1403] T 2cca0cb7cfcc41f8aee144493bd6991a P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.021s user 0.009s sys 0.005s
I20260925 13:57:15.576782 22293 raft_consensus.cc:359] T 2cca0cb7cfcc41f8aee144493bd6991a P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:15.577334 22293 raft_consensus.cc:385] T 2cca0cb7cfcc41f8aee144493bd6991a P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:15.577564 22293 raft_consensus.cc:740] T 2cca0cb7cfcc41f8aee144493bd6991a P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:15.578280 22293 consensus_queue.cc:260] T 2cca0cb7cfcc41f8aee144493bd6991a P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:15.581118 22212 tablet_bootstrap.cc:492] T 044b30ad37d74f3d8aca8cd26520b035 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:15.581552 22212 ts_tablet_manager.cc:1403] T 044b30ad37d74f3d8aca8cd26520b035 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.132s user 0.006s sys 0.004s
I20260925 13:57:15.584599 22293 ts_tablet_manager.cc:1434] T 2cca0cb7cfcc41f8aee144493bd6991a P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20260925 13:57:15.585242 22378 raft_consensus.cc:493] T 4ee62073ffa04bd39c688166cfd70677 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:15.585646 22378 raft_consensus.cc:515] T 4ee62073ffa04bd39c688166cfd70677 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:15.588056 21959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ee62073ffa04bd39c688166cfd70677" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:15.588229 22378 leader_election.cc:290] T 4ee62073ffa04bd39c688166cfd70677 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:15.588682 21959 raft_consensus.cc:2468] T 4ee62073ffa04bd39c688166cfd70677 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:15.590426 22105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ee62073ffa04bd39c688166cfd70677" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:15.591218 22105 raft_consensus.cc:2468] T 4ee62073ffa04bd39c688166cfd70677 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:15.592942 21732 leader_election.cc:304] T 4ee62073ffa04bd39c688166cfd70677 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:15.593783 22378 raft_consensus.cc:2804] T 4ee62073ffa04bd39c688166cfd70677 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:15.594108 22378 raft_consensus.cc:493] T 4ee62073ffa04bd39c688166cfd70677 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:15.594426 22378 raft_consensus.cc:3060] T 4ee62073ffa04bd39c688166cfd70677 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:15.597074 22212 raft_consensus.cc:359] T 044b30ad37d74f3d8aca8cd26520b035 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:15.559298 22416 ts_tablet_manager.cc:1950] T ee060e8412974e3dbb41c1b4fd0180a9 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:15.599452 22212 raft_consensus.cc:385] T 044b30ad37d74f3d8aca8cd26520b035 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:15.599706 22212 raft_consensus.cc:740] T 044b30ad37d74f3d8aca8cd26520b035 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:15.600464 22212 consensus_queue.cc:260] T 044b30ad37d74f3d8aca8cd26520b035 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:15.601027 22378 raft_consensus.cc:515] T 4ee62073ffa04bd39c688166cfd70677 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:15.603585 21959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ee62073ffa04bd39c688166cfd70677" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
I20260925 13:57:15.603734 22378 leader_election.cc:290] T 4ee62073ffa04bd39c688166cfd70677 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Requested vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:15.604110 21959 raft_consensus.cc:3060] T 4ee62073ffa04bd39c688166cfd70677 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:15.605767 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet ee060e8412974e3dbb41c1b4fd0180a9 (table default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]) successfully deleted
I20260925 13:57:15.608361 22416 tablet_replica.cc:333] T 1dd5ef2086884c81b48000d7e9de14e2 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:15.611886 22416 raft_consensus.cc:2243] T 1dd5ef2086884c81b48000d7e9de14e2 P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:15.612239 22212 ts_tablet_manager.cc:1434] T 044b30ad37d74f3d8aca8cd26520b035 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.030s user 0.000s sys 0.007s
I20260925 13:57:15.619160 22416 pending_rounds.cc:70] T 1dd5ef2086884c81b48000d7e9de14e2 P 145e520e0a5a445d943a53cf8c27efee: Trying to abort 1 pending ops.
I20260925 13:57:15.619513 22416 pending_rounds.cc:77] T 1dd5ef2086884c81b48000d7e9de14e2 P 145e520e0a5a445d943a53cf8c27efee: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251625988968448 op_type: NO_OP noop_request { }
I20260925 13:57:15.621958 22416 raft_consensus.cc:2889] T 1dd5ef2086884c81b48000d7e9de14e2 P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:57:15.614933 22105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ee62073ffa04bd39c688166cfd70677" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:15.623680 22416 raft_consensus.cc:2272] T 1dd5ef2086884c81b48000d7e9de14e2 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:15.623703 22105 raft_consensus.cc:3060] T 4ee62073ffa04bd39c688166cfd70677 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:15.634048 22416 ts_tablet_manager.cc:1916] T 1dd5ef2086884c81b48000d7e9de14e2 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:15.638095 22211 tablet_bootstrap.cc:492] T 044b30ad37d74f3d8aca8cd26520b035 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:15.638481 22211 ts_tablet_manager.cc:1403] T 044b30ad37d74f3d8aca8cd26520b035 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.146s user 0.008s sys 0.000s
I20260925 13:57:15.647989 22212 tablet_bootstrap.cc:492] T 2cca0cb7cfcc41f8aee144493bd6991a P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:15.648840 21959 raft_consensus.cc:2468] T 4ee62073ffa04bd39c688166cfd70677 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:15.650440 21732 leader_election.cc:304] T 4ee62073ffa04bd39c688166cfd70677 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:15.651657 22378 raft_consensus.cc:2804] T 4ee62073ffa04bd39c688166cfd70677 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:15.652161 22378 raft_consensus.cc:697] T 4ee62073ffa04bd39c688166cfd70677 P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Becoming Leader. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Running, Role: LEADER
I20260925 13:57:15.652602 21777 tablet_service.cc:1511] Processing CreateTablet for tablet 88f885774038498681d2710e51ff3040 (DEFAULT_TABLE table=default.table_492c567ddda14e77851270f34471730e [id=754c40502e614c5bb18230c852aa0016]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:15.652873 22378 consensus_queue.cc:237] T 4ee62073ffa04bd39c688166cfd70677 P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:15.653543 21777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88f885774038498681d2710e51ff3040. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:15.653422 22211 raft_consensus.cc:359] T 044b30ad37d74f3d8aca8cd26520b035 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:15.654028 22211 raft_consensus.cc:385] T 044b30ad37d74f3d8aca8cd26520b035 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:15.654263 22211 raft_consensus.cc:740] T 044b30ad37d74f3d8aca8cd26520b035 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:15.654206 21937 tablet_service.cc:1511] Processing CreateTablet for tablet 88f885774038498681d2710e51ff3040 (DEFAULT_TABLE table=default.table_492c567ddda14e77851270f34471730e [id=754c40502e614c5bb18230c852aa0016]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:15.655099 21937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88f885774038498681d2710e51ff3040. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:15.664371 22211 consensus_queue.cc:260] T 044b30ad37d74f3d8aca8cd26520b035 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:15.664726 22083 tablet_service.cc:1511] Processing CreateTablet for tablet 88f885774038498681d2710e51ff3040 (DEFAULT_TABLE table=default.table_492c567ddda14e77851270f34471730e [id=754c40502e614c5bb18230c852aa0016]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:15.665619 22083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88f885774038498681d2710e51ff3040. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:15.673365 22293 tablet_bootstrap.cc:492] T 88f885774038498681d2710e51ff3040 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:15.677626 22211 ts_tablet_manager.cc:1434] T 044b30ad37d74f3d8aca8cd26520b035 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.039s user 0.007s sys 0.000s
I20260925 13:57:15.678247 22293 tablet_bootstrap.cc:654] T 88f885774038498681d2710e51ff3040 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:15.684077 22105 raft_consensus.cc:2468] T 4ee62073ffa04bd39c688166cfd70677 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:15.685665 22211 tablet_bootstrap.cc:492] T 2cca0cb7cfcc41f8aee144493bd6991a P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:15.690666 22212 tablet_bootstrap.cc:654] T 2cca0cb7cfcc41f8aee144493bd6991a P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:15.694358 22293 tablet_bootstrap.cc:492] T 88f885774038498681d2710e51ff3040 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:15.694942 22293 ts_tablet_manager.cc:1403] T 88f885774038498681d2710e51ff3040 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.022s user 0.010s sys 0.004s
I20260925 13:57:15.697211 22293 raft_consensus.cc:359] T 88f885774038498681d2710e51ff3040 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:15.697806 22293 raft_consensus.cc:385] T 88f885774038498681d2710e51ff3040 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:15.698050 22293 raft_consensus.cc:740] T 88f885774038498681d2710e51ff3040 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:15.700344 22293 consensus_queue.cc:260] T 88f885774038498681d2710e51ff3040 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:15.703226 22293 ts_tablet_manager.cc:1434] T 88f885774038498681d2710e51ff3040 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20260925 13:57:15.701267 22416 ts_tablet_manager.cc:1929] T 1dd5ef2086884c81b48000d7e9de14e2 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:15.707212 22416 log.cc:1199] T 1dd5ef2086884c81b48000d7e9de14e2 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/1dd5ef2086884c81b48000d7e9de14e2
I20260925 13:57:15.708869 22416 ts_tablet_manager.cc:1950] T 1dd5ef2086884c81b48000d7e9de14e2 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:15.716799 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet 1dd5ef2086884c81b48000d7e9de14e2 (table default.table_492c567ddda14e77851270f34471730e [id=754c40502e614c5bb18230c852aa0016]) successfully deleted
I20260925 13:57:15.718521 22211 tablet_bootstrap.cc:654] T 2cca0cb7cfcc41f8aee144493bd6991a P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:15.734789 22212 tablet_bootstrap.cc:492] T 2cca0cb7cfcc41f8aee144493bd6991a P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:15.735225 22212 ts_tablet_manager.cc:1403] T 2cca0cb7cfcc41f8aee144493bd6991a P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.087s user 0.003s sys 0.007s
I20260925 13:57:15.737473 22212 raft_consensus.cc:359] T 2cca0cb7cfcc41f8aee144493bd6991a P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:15.738087 22212 raft_consensus.cc:385] T 2cca0cb7cfcc41f8aee144493bd6991a P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:15.738336 22212 raft_consensus.cc:740] T 2cca0cb7cfcc41f8aee144493bd6991a P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:15.731330 22211 tablet_bootstrap.cc:492] T 2cca0cb7cfcc41f8aee144493bd6991a P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:15.739394 22211 ts_tablet_manager.cc:1403] T 2cca0cb7cfcc41f8aee144493bd6991a P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.054s user 0.007s sys 0.002s
I20260925 13:57:15.746236 21584 master_service.cc:960] ReplaceTablet: received request to replace tablet d267def3f0864435a0d2c40fd7052eab from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:15.750363 22211 raft_consensus.cc:359] T 2cca0cb7cfcc41f8aee144493bd6991a P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:15.765729 21584 master_service.cc:960] ReplaceTablet: received request to replace tablet 4ee62073ffa04bd39c688166cfd70677 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:15.751031 22211 raft_consensus.cc:385] T 2cca0cb7cfcc41f8aee144493bd6991a P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:15.766402 22211 raft_consensus.cc:740] T 2cca0cb7cfcc41f8aee144493bd6991a P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:15.767196 22211 consensus_queue.cc:260] T 2cca0cb7cfcc41f8aee144493bd6991a P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:15.777560 22105 raft_consensus.cc:1275] T d71049b217a74662b7a9974e776d1448 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Refusing update from remote peer 5e2731975dbf433aade4a859df9699e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:57:15.780015 22378 consensus_queue.cc:1048] T d71049b217a74662b7a9974e776d1448 P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:15.754472 21581 catalog_manager.cc:5838] T 4ee62073ffa04bd39c688166cfd70677 P 5e2731975dbf433aade4a859df9699e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e2731975dbf433aade4a859df9699e2 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "5e2731975dbf433aade4a859df9699e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: UNKNOWN } } }
I20260925 13:57:15.790014 21959 raft_consensus.cc:1275] T d71049b217a74662b7a9974e776d1448 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Refusing update from remote peer 5e2731975dbf433aade4a859df9699e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:57:15.791704 22378 consensus_queue.cc:1048] T d71049b217a74662b7a9974e776d1448 P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:15.792464 22211 ts_tablet_manager.cc:1434] T 2cca0cb7cfcc41f8aee144493bd6991a P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.053s user 0.004s sys 0.001s
I20260925 13:57:15.804169 22211 tablet_bootstrap.cc:492] T 88f885774038498681d2710e51ff3040 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:15.810485 22378 raft_consensus.cc:493] T 044b30ad37d74f3d8aca8cd26520b035 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:15.810981 22378 raft_consensus.cc:515] T 044b30ad37d74f3d8aca8cd26520b035 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:15.816084 22368 raft_consensus.cc:493] T 044b30ad37d74f3d8aca8cd26520b035 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:15.816372 22378 leader_election.cc:290] T 044b30ad37d74f3d8aca8cd26520b035 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:15.816505 22368 raft_consensus.cc:515] T 044b30ad37d74f3d8aca8cd26520b035 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:15.818430 22368 leader_election.cc:290] T 044b30ad37d74f3d8aca8cd26520b035 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:15.819559 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044b30ad37d74f3d8aca8cd26520b035" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:15.820616 22107 raft_consensus.cc:2468] T 044b30ad37d74f3d8aca8cd26520b035 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:15.822064 21732 leader_election.cc:304] T 044b30ad37d74f3d8aca8cd26520b035 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:15.822862 22378 raft_consensus.cc:2804] T 044b30ad37d74f3d8aca8cd26520b035 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:15.823233 22378 raft_consensus.cc:493] T 044b30ad37d74f3d8aca8cd26520b035 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:15.823395 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044b30ad37d74f3d8aca8cd26520b035" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:15.823573 22378 raft_consensus.cc:3060] T 044b30ad37d74f3d8aca8cd26520b035 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:15.823964 21960 raft_consensus.cc:2468] T 044b30ad37d74f3d8aca8cd26520b035 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:15.828573 22212 consensus_queue.cc:260] T 2cca0cb7cfcc41f8aee144493bd6991a P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:15.831725 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044b30ad37d74f3d8aca8cd26520b035" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:15.832458 22107 raft_consensus.cc:2468] T 044b30ad37d74f3d8aca8cd26520b035 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 0.
I20260925 13:57:15.832617 22214 ts_tablet_manager.cc:1929] T ee060e8412974e3dbb41c1b4fd0180a9 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:15.833253 22214 log.cc:1199] T ee060e8412974e3dbb41c1b4fd0180a9 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/ee060e8412974e3dbb41c1b4fd0180a9
I20260925 13:57:15.834190 22214 ts_tablet_manager.cc:1950] T ee060e8412974e3dbb41c1b4fd0180a9 P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:15.834265 21896 leader_election.cc:304] T 044b30ad37d74f3d8aca8cd26520b035 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:15.837992 22368 raft_consensus.cc:2804] T 044b30ad37d74f3d8aca8cd26520b035 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:15.838905 22368 raft_consensus.cc:493] T 044b30ad37d74f3d8aca8cd26520b035 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:15.840011 22368 raft_consensus.cc:3060] T 044b30ad37d74f3d8aca8cd26520b035 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:15.840114 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044b30ad37d74f3d8aca8cd26520b035" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2" is_pre_election: true
I20260925 13:57:15.835210 22378 raft_consensus.cc:515] T 044b30ad37d74f3d8aca8cd26520b035 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:15.842011 21797 raft_consensus.cc:2393] T 044b30ad37d74f3d8aca8cd26520b035 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 145e520e0a5a445d943a53cf8c27efee in current term 1: Already voted for candidate 5e2731975dbf433aade4a859df9699e2 in this term.
I20260925 13:57:15.845695 22378 leader_election.cc:290] T 044b30ad37d74f3d8aca8cd26520b035 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Requested vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:15.849081 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044b30ad37d74f3d8aca8cd26520b035" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
I20260925 13:57:15.850797 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044b30ad37d74f3d8aca8cd26520b035" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:15.851318 22107 raft_consensus.cc:3060] T 044b30ad37d74f3d8aca8cd26520b035 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:15.843223 22212 ts_tablet_manager.cc:1434] T 2cca0cb7cfcc41f8aee144493bd6991a P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.108s user 0.005s sys 0.002s
I20260925 13:57:15.856602 22212 tablet_bootstrap.cc:492] T 88f885774038498681d2710e51ff3040 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:15.861408 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet ee060e8412974e3dbb41c1b4fd0180a9 (table default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]) successfully deleted
I20260925 13:57:15.866767 22214 tablet_replica.cc:333] T 1dd5ef2086884c81b48000d7e9de14e2 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:15.869398 22415 ts_tablet_manager.cc:1929] T ee060e8412974e3dbb41c1b4fd0180a9 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:15.869920 22415 log.cc:1199] T ee060e8412974e3dbb41c1b4fd0180a9 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/ee060e8412974e3dbb41c1b4fd0180a9
I20260925 13:57:15.870819 22415 ts_tablet_manager.cc:1950] T ee060e8412974e3dbb41c1b4fd0180a9 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:15.884063 22214 raft_consensus.cc:2243] T 1dd5ef2086884c81b48000d7e9de14e2 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:15.884580 22214 raft_consensus.cc:2272] T 1dd5ef2086884c81b48000d7e9de14e2 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:15.886566 22214 ts_tablet_manager.cc:1916] T 1dd5ef2086884c81b48000d7e9de14e2 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:15.886896 21584 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet 4ee62073ffa04bd39c688166cfd70677
I20260925 13:57:15.894312 21777 tablet_service.cc:1558] Processing DeleteTablet for tablet 4ee62073ffa04bd39c688166cfd70677 with delete_type TABLET_DATA_DELETED (replaced by tablet 3a8c53e379a9407c920d91b05ebd2afc) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:15.899312 21937 tablet_service.cc:1558] Processing DeleteTablet for tablet 4ee62073ffa04bd39c688166cfd70677 with delete_type TABLET_DATA_DELETED (replaced by tablet 3a8c53e379a9407c920d91b05ebd2afc) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:15.900185 22416 tablet_replica.cc:333] T 4ee62073ffa04bd39c688166cfd70677 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:15.901154 22416 raft_consensus.cc:2243] T 4ee62073ffa04bd39c688166cfd70677 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:15.901683 22416 raft_consensus.cc:2272] T 4ee62073ffa04bd39c688166cfd70677 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:15.901721 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet ee060e8412974e3dbb41c1b4fd0180a9 (table default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]) successfully deleted
I20260925 13:57:15.904446 22416 ts_tablet_manager.cc:1916] T 4ee62073ffa04bd39c688166cfd70677 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:15.917315 22107 raft_consensus.cc:2468] T 044b30ad37d74f3d8aca8cd26520b035 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:15.922111 22368 raft_consensus.cc:515] T 044b30ad37d74f3d8aca8cd26520b035 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:15.923451 21960 raft_consensus.cc:2393] T 044b30ad37d74f3d8aca8cd26520b035 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e2731975dbf433aade4a859df9699e2 in current term 1: Already voted for candidate 145e520e0a5a445d943a53cf8c27efee in this term.
I20260925 13:57:15.928169 22415 tablet_replica.cc:333] T 1dd5ef2086884c81b48000d7e9de14e2 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:15.929013 22415 raft_consensus.cc:2243] T 1dd5ef2086884c81b48000d7e9de14e2 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:15.929476 22415 raft_consensus.cc:2272] T 1dd5ef2086884c81b48000d7e9de14e2 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:15.936380 22435 tablet.cc:1751] T d71049b217a74662b7a9974e776d1448 P 5e2731975dbf433aade4a859df9699e2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:57:15.937366 21584 catalog_manager.cc:6934] ReplaceTablet: tablet 4ee62073ffa04bd39c688166cfd70677 deleted and replaced by tablet 3a8c53e379a9407c920d91b05ebd2afc
I20260925 13:57:15.953825 22415 ts_tablet_manager.cc:1916] T 1dd5ef2086884c81b48000d7e9de14e2 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:15.954859 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044b30ad37d74f3d8aca8cd26520b035" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2"
I20260925 13:57:15.955677 21797 raft_consensus.cc:2393] T 044b30ad37d74f3d8aca8cd26520b035 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 145e520e0a5a445d943a53cf8c27efee in current term 1: Already voted for candidate 5e2731975dbf433aade4a859df9699e2 in this term.
I20260925 13:57:15.968071 22368 leader_election.cc:290] T 044b30ad37d74f3d8aca8cd26520b035 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Requested vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:15.932556 22212 tablet_bootstrap.cc:654] T 88f885774038498681d2710e51ff3040 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:15.940157 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet 4ee62073ffa04bd39c688166cfd70677 with delete_type TABLET_DATA_DELETED (replaced by tablet 3a8c53e379a9407c920d91b05ebd2afc) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:15.940502 22437 tablet.cc:1751] T d71049b217a74662b7a9974e776d1448 P ee02f77594864349982492862d668a36: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:57:15.971092 22378 raft_consensus.cc:493] T 2cca0cb7cfcc41f8aee144493bd6991a P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:15.971532 22378 raft_consensus.cc:515] T 2cca0cb7cfcc41f8aee144493bd6991a P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:15.972904 21732 leader_election.cc:304] T 044b30ad37d74f3d8aca8cd26520b035 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2, ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee
I20260925 13:57:15.973743 22378 leader_election.cc:290] T 2cca0cb7cfcc41f8aee144493bd6991a P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:15.970458 22211 tablet_bootstrap.cc:654] T 88f885774038498681d2710e51ff3040 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:15.974238 22378 raft_consensus.cc:2804] T 044b30ad37d74f3d8aca8cd26520b035 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:15.974602 22378 raft_consensus.cc:697] T 044b30ad37d74f3d8aca8cd26520b035 P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Becoming Leader. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Running, Role: LEADER
I20260925 13:57:15.975427 22378 consensus_queue.cc:237] T 044b30ad37d74f3d8aca8cd26520b035 P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:15.982152 21582 catalog_manager.cc:2213] created HMS catalog entry for table [id=a22f86f8c2404aa69bdeb9883471453c]
I20260925 13:57:15.995584 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cca0cb7cfcc41f8aee144493bd6991a" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:15.996198 21960 raft_consensus.cc:2468] T 2cca0cb7cfcc41f8aee144493bd6991a P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
W20260925 13:57:15.997157 21573 consensus_peers.cc:597] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 -> Peer 3ea1e11bc6d84114b7616815b6af4635 (127.18.182.253:46177): Couldn't send request to peer 3ea1e11bc6d84114b7616815b6af4635. Status: Network error: Client connection negotiation failed: client connection to 127.18.182.253:46177: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260925 13:57:15.997790 21732 leader_election.cc:304] T 2cca0cb7cfcc41f8aee144493bd6991a P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:15.998471 22439 raft_consensus.cc:2804] T 2cca0cb7cfcc41f8aee144493bd6991a P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:15.998790 22439 raft_consensus.cc:493] T 2cca0cb7cfcc41f8aee144493bd6991a P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:15.999069 22439 raft_consensus.cc:3060] T 2cca0cb7cfcc41f8aee144493bd6991a P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:16.003541 22438 tablet.cc:1751] T d71049b217a74662b7a9974e776d1448 P 145e520e0a5a445d943a53cf8c27efee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:57:16.017782 21582 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35994 (request call id 14) took 6925 ms. Trace:
I20260925 13:57:16.018177 22439 raft_consensus.cc:515] T 2cca0cb7cfcc41f8aee144493bd6991a P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.025105 22211 tablet_bootstrap.cc:492] T 88f885774038498681d2710e51ff3040 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:16.025532 22211 ts_tablet_manager.cc:1403] T 88f885774038498681d2710e51ff3040 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.222s user 0.005s sys 0.003s
I20260925 13:57:16.026889 21643 catalog_manager.cc:2811] Deleting table default.table_492c567ddda14e77851270f34471730e [id=754c40502e614c5bb18230c852aa0016] in response to Hive Metastore notification log event 7
I20260925 13:57:16.018163 21582 rpcz_store.cc:276] 0925 13:57:09.092247 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:09.092353 (+ 106us) service_pool.cc:224] Handling call
0925 13:57:10.843204 (+1750851us) catalog_manager.cc:2140] Acquired catalog manager lock
0925 13:57:10.845593 (+ 2389us) catalog_manager.cc:2186] Created new table and tablet info
0925 13:57:15.982107 (+5136514us) catalog_manager.cc:2212] Created new table in HMS catalog
0925 13:57:16.017126 (+ 35019us) catalog_manager.cc:2246] Wrote table and tablets to system table
0925 13:57:16.017433 (+ 307us) catalog_manager.cc:2280] Inserted table and tablets into CatalogManager maps
0925 13:57:16.017719 (+ 286us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:57:15.987562 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:57:15.987926 (+ 364us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:57:15.987948 (+ 22us) write_op.cc:435] Acquired schema lock
0925 13:57:15.987961 (+ 13us) tablet.cc:662] Decoding operations
0925 13:57:15.988208 (+ 247us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:57:15.988241 (+ 33us) write_op.cc:641] Partition lock acquired for write op
0925 13:57:15.988260 (+ 19us) tablet.cc:685] Acquiring locks for 2 operations
0925 13:57:15.988442 (+ 182us) tablet.cc:701] Row locks acquired
0925 13:57:15.988454 (+ 12us) write_op.cc:260] PREPARE: finished
0925 13:57:15.988577 (+ 123us) write_op.cc:270] Start()
0925 13:57:15.988649 (+ 72us) write_op.cc:276] Timestamp: P: 1790344635988497 usec, L: 0
0925 13:57:15.988662 (+ 13us) op_driver.cc:348] REPLICATION: starting
0925 13:57:15.993644 (+ 4982us) log.cc:844] Serialized 432 byte log entry
0925 13:57:16.014459 (+ 20815us) op_driver.cc:464] REPLICATION: finished
0925 13:57:16.014916 (+ 457us) write_op.cc:301] APPLY: starting
0925 13:57:16.015004 (+ 88us) tablet.cc:1394] starting BulkCheckPresence
0925 13:57:16.015144 (+ 140us) tablet.cc:1397] finished BulkCheckPresence
0925 13:57:16.015261 (+ 117us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:57:16.015688 (+ 427us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:57:16.015839 (+ 151us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 13:57:16.015881 (+ 42us) write_op.cc:312] APPLY: finished
0925 13:57:16.016066 (+ 185us) log.cc:844] Serialized 28 byte log entry
0925 13:57:16.016202 (+ 136us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:16.016287 (+ 85us) write_op.cc:454] Released schema lock
0925 13:57:16.016331 (+ 44us) write_op.cc:341] FINISH: Updating metrics
0925 13:57:16.016615 (+ 284us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:16.016649 (+ 34us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":4438385,"Hive Metastore.run_cpu_time_us":822,"Hive Metastore.run_wall_time_us":697119,"child_traces":[["op",{"apply.queue_time_us":283,"apply.run_cpu_time_us":1646,"apply.run_wall_time_us":1646,"num_ops":2,"prepare.queue_time_us":3881,"prepare.run_cpu_time_us":1836,"prepare.run_wall_time_us":6341,"raft.queue_time_us":4680,"raft.run_cpu_time_us":518,"raft.run_wall_time_us":518,"replication_time_us":25690}]]}
I20260925 13:57:16.033895 21582 catalog_manager.cc:5938] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873) does not have the latest schema for tablet d71049b217a74662b7a9974e776d1448 (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]). Expected version 1 got 0
I20260925 13:57:16.037921 21582 master_service.cc:960] ReplaceTablet: received request to replace tablet 88f885774038498681d2710e51ff3040 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:16.037860 22211 raft_consensus.cc:359] T 88f885774038498681d2710e51ff3040 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.038589 22211 raft_consensus.cc:385] T 88f885774038498681d2710e51ff3040 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:16.038874 22211 raft_consensus.cc:740] T 88f885774038498681d2710e51ff3040 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:16.039747 22211 consensus_queue.cc:260] T 88f885774038498681d2710e51ff3040 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.042706 21777 tablet_service.cc:1511] Processing CreateTablet for tablet 3a8c53e379a9407c920d91b05ebd2afc (DEFAULT_TABLE table=default.table_b17685eaeada404790103bd89711435e [id=df6afcfb1e2c463493b14228e19ade26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:16.043314 22211 ts_tablet_manager.cc:1434] T 88f885774038498681d2710e51ff3040 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.018s user 0.006s sys 0.001s
I20260925 13:57:16.043731 21777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a8c53e379a9407c920d91b05ebd2afc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:16.049352 22439 leader_election.cc:290] T 2cca0cb7cfcc41f8aee144493bd6991a P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Requested vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:16.052417 21959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cca0cb7cfcc41f8aee144493bd6991a" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
I20260925 13:57:16.052924 21959 raft_consensus.cc:3060] T 2cca0cb7cfcc41f8aee144493bd6991a P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
W20260925 13:57:16.059968 21572 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 7075ed41f73349aaa7ac85d29950b494 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 7075ed41f73349aaa7ac85d29950b494 consensus state has no leader
I20260925 13:57:16.079147 21937 tablet_service.cc:1558] Processing DeleteTablet for tablet 88f885774038498681d2710e51ff3040 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:57:16 UTC) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:16.079892 21643 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet 88f885774038498681d2710e51ff3040
I20260925 13:57:16.082378 21776 tablet_service.cc:1558] Processing DeleteTablet for tablet 88f885774038498681d2710e51ff3040 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:57:16 UTC) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:16.084607 21937 tablet_service.cc:1558] Processing DeleteTablet for tablet 88f885774038498681d2710e51ff3040 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:57:16 UTC) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:16.087260 21581 catalog_manager.cc:5838] T 044b30ad37d74f3d8aca8cd26520b035 P 5e2731975dbf433aade4a859df9699e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e2731975dbf433aade4a859df9699e2 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "5e2731975dbf433aade4a859df9699e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: UNKNOWN } } }
I20260925 13:57:16.065689 21959 raft_consensus.cc:2468] T 2cca0cb7cfcc41f8aee144493bd6991a P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:16.094002 21732 leader_election.cc:304] T 2cca0cb7cfcc41f8aee144493bd6991a P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:16.096163 22439 raft_consensus.cc:2804] T 2cca0cb7cfcc41f8aee144493bd6991a P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:16.096598 22439 raft_consensus.cc:697] T 2cca0cb7cfcc41f8aee144493bd6991a P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Becoming Leader. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Running, Role: LEADER
I20260925 13:57:16.097335 22439 consensus_queue.cc:237] T 2cca0cb7cfcc41f8aee144493bd6991a P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.098742 21937 tablet_service.cc:1511] Processing CreateTablet for tablet 3a8c53e379a9407c920d91b05ebd2afc (DEFAULT_TABLE table=default.table_b17685eaeada404790103bd89711435e [id=df6afcfb1e2c463493b14228e19ade26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:16.099704 21937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a8c53e379a9407c920d91b05ebd2afc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:16.105762 22416 ts_tablet_manager.cc:1929] T 4ee62073ffa04bd39c688166cfd70677 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:16.106240 22416 log.cc:1199] T 4ee62073ffa04bd39c688166cfd70677 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/4ee62073ffa04bd39c688166cfd70677
I20260925 13:57:16.106968 22416 ts_tablet_manager.cc:1950] T 4ee62073ffa04bd39c688166cfd70677 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:16.109849 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet 4ee62073ffa04bd39c688166cfd70677 (table default.table_b17685eaeada404790103bd89711435e [id=df6afcfb1e2c463493b14228e19ade26]) successfully deleted
I20260925 13:57:16.114241 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044b30ad37d74f3d8aca8cd26520b035" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:16.115028 22107 raft_consensus.cc:2393] T 044b30ad37d74f3d8aca8cd26520b035 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 145e520e0a5a445d943a53cf8c27efee in current term 1: Already voted for candidate 5e2731975dbf433aade4a859df9699e2 in this term.
I20260925 13:57:16.125185 22416 tablet_replica.cc:333] T 88f885774038498681d2710e51ff3040 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:16.126042 21896 leader_election.cc:304] T 044b30ad37d74f3d8aca8cd26520b035 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee; no voters: 5e2731975dbf433aade4a859df9699e2, ee02f77594864349982492862d668a36
I20260925 13:57:16.126807 22432 raft_consensus.cc:2749] T 044b30ad37d74f3d8aca8cd26520b035 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260925 13:57:16.128093 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet 88f885774038498681d2710e51ff3040 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:57:16 UTC) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:16.128880 22087 tablet_service.cc:1511] Processing CreateTablet for tablet 3a8c53e379a9407c920d91b05ebd2afc (DEFAULT_TABLE table=default.table_b17685eaeada404790103bd89711435e [id=df6afcfb1e2c463493b14228e19ade26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:16.130143 22087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a8c53e379a9407c920d91b05ebd2afc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:16.131569 22106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cca0cb7cfcc41f8aee144493bd6991a" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:16.132066 22106 raft_consensus.cc:3060] T 2cca0cb7cfcc41f8aee144493bd6991a P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:16.136633 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cca0cb7cfcc41f8aee144493bd6991a" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:16.138885 22416 raft_consensus.cc:2243] T 88f885774038498681d2710e51ff3040 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:16.143848 22378 raft_consensus.cc:493] T 88f885774038498681d2710e51ff3040 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:16.144301 22378 raft_consensus.cc:515] T 88f885774038498681d2710e51ff3040 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.139621 22416 raft_consensus.cc:2272] T 88f885774038498681d2710e51ff3040 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:16.146206 22378 leader_election.cc:290] T 88f885774038498681d2710e51ff3040 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:16.147601 21959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f885774038498681d2710e51ff3040" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:16.147802 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f885774038498681d2710e51ff3040" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
W20260925 13:57:16.149045 21732 leader_election.cc:343] T 88f885774038498681d2710e51ff3040 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:57:16.149531 22106 raft_consensus.cc:2468] T 2cca0cb7cfcc41f8aee144493bd6991a P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
W20260925 13:57:16.149588 21732 leader_election.cc:343] T 88f885774038498681d2710e51ff3040 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:57:16.150089 21732 leader_election.cc:304] T 88f885774038498681d2710e51ff3040 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
I20260925 13:57:16.150410 22416 ts_tablet_manager.cc:1916] T 88f885774038498681d2710e51ff3040 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:16.154819 22378 raft_consensus.cc:2749] T 88f885774038498681d2710e51ff3040 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:57:16.162837 22415 ts_tablet_manager.cc:1929] T 1dd5ef2086884c81b48000d7e9de14e2 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:16.163031 21643 catalog_manager.cc:2811] Deleting table default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf] in response to Hive Metastore notification log event 9
I20260925 13:57:16.163216 22415 log.cc:1199] T 1dd5ef2086884c81b48000d7e9de14e2 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/1dd5ef2086884c81b48000d7e9de14e2
I20260925 13:57:16.163991 22415 ts_tablet_manager.cc:1950] T 1dd5ef2086884c81b48000d7e9de14e2 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:16.165014 22293 tablet_bootstrap.cc:492] T 3a8c53e379a9407c920d91b05ebd2afc P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:16.167896 22214 ts_tablet_manager.cc:1929] T 1dd5ef2086884c81b48000d7e9de14e2 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:16.169770 22293 tablet_bootstrap.cc:654] T 3a8c53e379a9407c920d91b05ebd2afc P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:16.171221 22415 tablet_replica.cc:333] T 4ee62073ffa04bd39c688166cfd70677 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:16.172047 22415 raft_consensus.cc:2243] T 4ee62073ffa04bd39c688166cfd70677 P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:16.172906 22415 pending_rounds.cc:70] T 4ee62073ffa04bd39c688166cfd70677 P 5e2731975dbf433aade4a859df9699e2: Trying to abort 1 pending ops.
I20260925 13:57:16.173197 22415 pending_rounds.cc:77] T 4ee62073ffa04bd39c688166cfd70677 P 5e2731975dbf433aade4a859df9699e2: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251627642331136 op_type: NO_OP noop_request { }
I20260925 13:57:16.173616 22415 raft_consensus.cc:2889] T 4ee62073ffa04bd39c688166cfd70677 P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:57:16.174026 22415 raft_consensus.cc:2272] T 4ee62073ffa04bd39c688166cfd70677 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:16.179067 22293 tablet_bootstrap.cc:492] T 3a8c53e379a9407c920d91b05ebd2afc P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:16.179476 22293 ts_tablet_manager.cc:1403] T 3a8c53e379a9407c920d91b05ebd2afc P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.004s
I20260925 13:57:16.181155 22415 ts_tablet_manager.cc:1916] T 4ee62073ffa04bd39c688166cfd70677 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:16.181738 22293 raft_consensus.cc:359] T 3a8c53e379a9407c920d91b05ebd2afc P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.182338 22293 raft_consensus.cc:385] T 3a8c53e379a9407c920d91b05ebd2afc P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:16.182591 22293 raft_consensus.cc:740] T 3a8c53e379a9407c920d91b05ebd2afc P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:16.187597 22293 consensus_queue.cc:260] T 3a8c53e379a9407c920d91b05ebd2afc P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.191231 22212 tablet_bootstrap.cc:492] T 88f885774038498681d2710e51ff3040 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:16.194330 22212 ts_tablet_manager.cc:1403] T 88f885774038498681d2710e51ff3040 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.338s user 0.013s sys 0.004s
I20260925 13:57:16.199306 22212 raft_consensus.cc:359] T 88f885774038498681d2710e51ff3040 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.200014 22212 raft_consensus.cc:385] T 88f885774038498681d2710e51ff3040 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:16.200310 22212 raft_consensus.cc:740] T 88f885774038498681d2710e51ff3040 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:16.201144 22212 consensus_queue.cc:260] T 88f885774038498681d2710e51ff3040 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.206875 22212 ts_tablet_manager.cc:1434] T 88f885774038498681d2710e51ff3040 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.012s user 0.005s sys 0.000s
I20260925 13:57:16.208010 22212 tablet_bootstrap.cc:492] T 3a8c53e379a9407c920d91b05ebd2afc P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:16.209874 22293 ts_tablet_manager.cc:1434] T 3a8c53e379a9407c920d91b05ebd2afc P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.030s user 0.004s sys 0.001s
I20260925 13:57:16.210232 22214 log.cc:1199] T 1dd5ef2086884c81b48000d7e9de14e2 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/1dd5ef2086884c81b48000d7e9de14e2
I20260925 13:57:16.210916 22214 ts_tablet_manager.cc:1950] T 1dd5ef2086884c81b48000d7e9de14e2 P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:16.213750 22214 tablet_replica.cc:333] T 4ee62073ffa04bd39c688166cfd70677 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:16.230778 21584 master_service.cc:960] ReplaceTablet: received request to replace tablet 1dd5ef2086884c81b48000d7e9de14e2 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:16.234687 21584 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35994:
name: "default.Table_b54e9b49662b4a8e892615fec1080a87"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:57:16.229682 22416 ts_tablet_manager.cc:1929] T 88f885774038498681d2710e51ff3040 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:16.236855 22416 log.cc:1199] T 88f885774038498681d2710e51ff3040 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/88f885774038498681d2710e51ff3040
I20260925 13:57:16.237666 22416 ts_tablet_manager.cc:1950] T 88f885774038498681d2710e51ff3040 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:16.249923 22211 tablet_bootstrap.cc:492] T 3a8c53e379a9407c920d91b05ebd2afc P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:16.214447 22214 raft_consensus.cc:2243] T 4ee62073ffa04bd39c688166cfd70677 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:16.258562 22214 raft_consensus.cc:2272] T 4ee62073ffa04bd39c688166cfd70677 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:16.260803 22214 ts_tablet_manager.cc:1916] T 4ee62073ffa04bd39c688166cfd70677 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:16.272805 22415 ts_tablet_manager.cc:1929] T 4ee62073ffa04bd39c688166cfd70677 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:16.273124 22212 tablet_bootstrap.cc:654] T 3a8c53e379a9407c920d91b05ebd2afc P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:16.284008 21937 tablet_service.cc:1511] Processing CreateTablet for tablet abadb0b2c7d34451b7b23489e6d866c6 (DEFAULT_TABLE table=default.table_8008d16a92cf4c0dafb258a1122bffb5 [id=a22f86f8c2404aa69bdeb9883471453c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:16.285413 21643 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet 2cca0cb7cfcc41f8aee144493bd6991a
I20260925 13:57:16.285387 21777 tablet_service.cc:1511] Processing CreateTablet for tablet abadb0b2c7d34451b7b23489e6d866c6 (DEFAULT_TABLE table=default.table_8008d16a92cf4c0dafb258a1122bffb5 [id=a22f86f8c2404aa69bdeb9883471453c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:16.286356 21777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abadb0b2c7d34451b7b23489e6d866c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:16.286763 22087 tablet_service.cc:1511] Processing CreateTablet for tablet abadb0b2c7d34451b7b23489e6d866c6 (DEFAULT_TABLE table=default.table_8008d16a92cf4c0dafb258a1122bffb5 [id=a22f86f8c2404aa69bdeb9883471453c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:16.287832 22087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abadb0b2c7d34451b7b23489e6d866c6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:57:16.289963 21575 catalog_manager.cc:5030] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): delete failed for tablet 88f885774038498681d2710e51ff3040 because the tablet was not found. No further retry: Not found: Tablet not found: 88f885774038498681d2710e51ff3040
I20260925 13:57:16.290666 21776 tablet_service.cc:1558] Processing DeleteTablet for tablet 2cca0cb7cfcc41f8aee144493bd6991a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:57:16 UTC) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:16.292114 21938 tablet_service.cc:1558] Processing DeleteTablet for tablet 2cca0cb7cfcc41f8aee144493bd6991a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:57:16 UTC) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:16.297511 22416 tablet_replica.cc:333] T 2cca0cb7cfcc41f8aee144493bd6991a P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:16.298233 22416 raft_consensus.cc:2243] T 2cca0cb7cfcc41f8aee144493bd6991a P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:16.298731 22416 raft_consensus.cc:2272] T 2cca0cb7cfcc41f8aee144493bd6991a P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:16.299387 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet 2cca0cb7cfcc41f8aee144493bd6991a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:57:16 UTC) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:16.301149 22293 tablet_bootstrap.cc:492] T abadb0b2c7d34451b7b23489e6d866c6 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:16.307811 21937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abadb0b2c7d34451b7b23489e6d866c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:16.308918 22416 ts_tablet_manager.cc:1916] T 2cca0cb7cfcc41f8aee144493bd6991a P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:16.310539 22293 tablet_bootstrap.cc:654] T abadb0b2c7d34451b7b23489e6d866c6 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:16.273329 22415 log.cc:1199] T 4ee62073ffa04bd39c688166cfd70677 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/4ee62073ffa04bd39c688166cfd70677
I20260925 13:57:16.311888 21581 master_service.cc:960] ReplaceTablet: received request to replace tablet 2cca0cb7cfcc41f8aee144493bd6991a from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:16.314291 22415 ts_tablet_manager.cc:1950] T 4ee62073ffa04bd39c688166cfd70677 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:16.318030 21586 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:35994 (request call id 8) took 7481 ms. Trace:
W20260925 13:57:16.318660 21580 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_61d48f9061c741fc8aa556e3edd984fb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:57:16.318921 21587 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:35994 (request call id 7) took 7563 ms. Trace:
I20260925 13:57:16.318480 21586 rpcz_store.cc:276] 0925 13:57:08.836129 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:08.836295 (+ 166us) service_pool.cc:224] Handling call
0925 13:57:08.876482 (+ 40187us) spinlock_profiling.cc:243] Waited 38 ms on lock 0x7b54000203d4. stack: 00007fd96d084323 00007fd96d084220 00007fd96cc28574 0000000000516964 00000000005168f9 00007fd9741825c0 00007fd9742216fa 00007fd975f92d8c 00007fd976092876 00007fd973c41b34 00007fd973c41ac2 00007fd973c419f1 00007fd973c4196c 00007fd973c40c22 00007fd971ee709c 00007fd971ee64d6
0925 13:57:10.827514 (+1951032us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 13:57:16.317909 (+5490395us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":2379012,"Hive Metastore.run_cpu_time_us":378,"Hive Metastore.run_wall_time_us":822532,"spinlock_wait_cycles":87390080}
I20260925 13:57:16.319253 21587 rpcz_store.cc:276] 0925 13:57:08.755756 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:08.755907 (+ 151us) service_pool.cc:224] Handling call
0925 13:57:10.264903 (+1508996us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 13:57:16.318863 (+6053960us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":360077,"Hive Metastore.run_cpu_time_us":753,"Hive Metastore.run_wall_time_us":2176405}
W20260925 13:57:16.321794 21585 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_fe2e32c0275e4744b1163ae7b22eaf51 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:57:16.322726 22415 tablet_replica.cc:333] T 88f885774038498681d2710e51ff3040 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:16.323522 22415 raft_consensus.cc:2243] T 88f885774038498681d2710e51ff3040 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260925 13:57:16.323534 21583 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_e0eb6c1d49224d1cb9b4f6d06f0f7a29 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:57:16.323971 22415 raft_consensus.cc:2272] T 88f885774038498681d2710e51ff3040 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260925 13:57:16.324295 21584 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_b54e9b49662b4a8e892615fec1080a87 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:57:16.325769 22415 ts_tablet_manager.cc:1916] T 88f885774038498681d2710e51ff3040 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:16.328182 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet 4ee62073ffa04bd39c688166cfd70677 (table default.table_b17685eaeada404790103bd89711435e [id=df6afcfb1e2c463493b14228e19ade26]) successfully deleted
I20260925 13:57:16.336612 21588 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:35994 (request call id 6) took 7661 ms. Trace:
I20260925 13:57:16.337110 21588 rpcz_store.cc:276] 0925 13:57:08.674829 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:08.674942 (+ 113us) service_pool.cc:224] Handling call
0925 13:57:08.724319 (+ 49377us) spinlock_profiling.cc:243] Waited 42.9 ms on lock 0x7b54000203d4. stack: 00007fd96d084323 00007fd96d084220 00007fd96cc28574 0000000000516964 00000000005168f9 00007fd9741825c0 00007fd9742216fa 00007fd975f92d8c 00007fd976092be6 00007fd973c46634 00007fd973c465c2 00007fd973c464f1 00007fd973c4646c 00007fd973c45722 00007fd971ee709c 00007fd971ee64d6
0925 13:57:10.263613 (+1539294us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 13:57:16.302789 (+6039176us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 13:57:16.335653 (+ 32864us) catalog_manager.cc:3764] Apply alter schema
0925 13:57:16.336548 (+ 895us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":84,"Hive Metastore.run_cpu_time_us":1760,"Hive Metastore.run_wall_time_us":347517,"mutex_wait_us":31378,"spinlock_wait_cycles":98644736}
I20260925 13:57:16.363958 22211 tablet_bootstrap.cc:654] T 3a8c53e379a9407c920d91b05ebd2afc P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:16.371568 22211 tablet_bootstrap.cc:492] T 3a8c53e379a9407c920d91b05ebd2afc P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:16.372081 22211 ts_tablet_manager.cc:1403] T 3a8c53e379a9407c920d91b05ebd2afc P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.122s user 0.006s sys 0.005s
I20260925 13:57:16.374511 22211 raft_consensus.cc:359] T 3a8c53e379a9407c920d91b05ebd2afc P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.375121 22211 raft_consensus.cc:385] T 3a8c53e379a9407c920d91b05ebd2afc P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:16.382560 22211 raft_consensus.cc:740] T 3a8c53e379a9407c920d91b05ebd2afc P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:16.383253 22211 consensus_queue.cc:260] T 3a8c53e379a9407c920d91b05ebd2afc P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.385322 22211 ts_tablet_manager.cc:1434] T 3a8c53e379a9407c920d91b05ebd2afc P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.013s user 0.005s sys 0.000s
I20260925 13:57:16.391216 22212 tablet_bootstrap.cc:492] T 3a8c53e379a9407c920d91b05ebd2afc P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:16.391692 22212 ts_tablet_manager.cc:1403] T 3a8c53e379a9407c920d91b05ebd2afc P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.184s user 0.009s sys 0.000s
I20260925 13:57:16.394142 22212 raft_consensus.cc:359] T 3a8c53e379a9407c920d91b05ebd2afc P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.394824 22212 raft_consensus.cc:385] T 3a8c53e379a9407c920d91b05ebd2afc P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:16.395092 22212 raft_consensus.cc:740] T 3a8c53e379a9407c920d91b05ebd2afc P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:16.395854 22212 consensus_queue.cc:260] T 3a8c53e379a9407c920d91b05ebd2afc P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.398793 22416 ts_tablet_manager.cc:1929] T 2cca0cb7cfcc41f8aee144493bd6991a P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:16.412395 22416 log.cc:1199] T 2cca0cb7cfcc41f8aee144493bd6991a P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/2cca0cb7cfcc41f8aee144493bd6991a
I20260925 13:57:16.413560 22416 ts_tablet_manager.cc:1950] T 2cca0cb7cfcc41f8aee144493bd6991a P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:16.416985 22214 ts_tablet_manager.cc:1929] T 4ee62073ffa04bd39c688166cfd70677 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:16.417340 22214 log.cc:1199] T 4ee62073ffa04bd39c688166cfd70677 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/4ee62073ffa04bd39c688166cfd70677
I20260925 13:57:16.417933 22214 ts_tablet_manager.cc:1950] T 4ee62073ffa04bd39c688166cfd70677 P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:16.418411 22212 ts_tablet_manager.cc:1434] T 3a8c53e379a9407c920d91b05ebd2afc P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.026s user 0.005s sys 0.000s
I20260925 13:57:16.421805 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet 2cca0cb7cfcc41f8aee144493bd6991a (table default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]) successfully deleted
I20260925 13:57:16.427155 22211 tablet_bootstrap.cc:492] T abadb0b2c7d34451b7b23489e6d866c6 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:16.429164 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet 4ee62073ffa04bd39c688166cfd70677 (table default.table_b17685eaeada404790103bd89711435e [id=df6afcfb1e2c463493b14228e19ade26]) successfully deleted
I20260925 13:57:16.430020 22214 tablet_replica.cc:333] T 88f885774038498681d2710e51ff3040 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:16.430749 22214 raft_consensus.cc:2243] T 88f885774038498681d2710e51ff3040 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:16.431242 22214 raft_consensus.cc:2272] T 88f885774038498681d2710e51ff3040 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:16.433271 22214 ts_tablet_manager.cc:1916] T 88f885774038498681d2710e51ff3040 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:16.441004 22415 ts_tablet_manager.cc:1929] T 88f885774038498681d2710e51ff3040 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:16.443310 21582 master_service.cc:960] ReplaceTablet: received request to replace tablet d595ab2f7dc74fdfbf3546e477685a01 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:16.458834 22212 tablet_bootstrap.cc:492] T abadb0b2c7d34451b7b23489e6d866c6 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:16.463630 22293 tablet_bootstrap.cc:492] T abadb0b2c7d34451b7b23489e6d866c6 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:16.463817 22415 log.cc:1199] T 88f885774038498681d2710e51ff3040 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/88f885774038498681d2710e51ff3040
I20260925 13:57:16.465112 22415 ts_tablet_manager.cc:1950] T 88f885774038498681d2710e51ff3040 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:16.472646 22214 ts_tablet_manager.cc:1929] T 88f885774038498681d2710e51ff3040 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:16.472971 22214 log.cc:1199] T 88f885774038498681d2710e51ff3040 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/88f885774038498681d2710e51ff3040
I20260925 13:57:16.473553 22214 ts_tablet_manager.cc:1950] T 88f885774038498681d2710e51ff3040 P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:16.473541 21582 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet d595ab2f7dc74fdfbf3546e477685a01
I20260925 13:57:16.477555 21937 tablet_service.cc:1558] Processing DeleteTablet for tablet d595ab2f7dc74fdfbf3546e477685a01 with delete_type TABLET_DATA_DELETED (replaced by tablet c965fcaf7eb246daa288d6183ce8a6ab) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:16.478318 21582 catalog_manager.cc:6934] ReplaceTablet: tablet d595ab2f7dc74fdfbf3546e477685a01 deleted and replaced by tablet c965fcaf7eb246daa288d6183ce8a6ab
I20260925 13:57:16.478327 22416 tablet_replica.cc:333] T d595ab2f7dc74fdfbf3546e477685a01 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:16.478415 22087 tablet_service.cc:1558] Processing DeleteTablet for tablet d595ab2f7dc74fdfbf3546e477685a01 with delete_type TABLET_DATA_DELETED (replaced by tablet c965fcaf7eb246daa288d6183ce8a6ab) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:16.479084 22416 raft_consensus.cc:2243] T d595ab2f7dc74fdfbf3546e477685a01 P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:16.464102 22293 ts_tablet_manager.cc:1403] T abadb0b2c7d34451b7b23489e6d866c6 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.163s user 0.005s sys 0.029s
I20260925 13:57:16.485926 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet 88f885774038498681d2710e51ff3040 (table default.table_492c567ddda14e77851270f34471730e [id=754c40502e614c5bb18230c852aa0016]) successfully deleted
I20260925 13:57:16.481542 22211 tablet_bootstrap.cc:654] T abadb0b2c7d34451b7b23489e6d866c6 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:16.486817 22214 tablet_replica.cc:333] T 2cca0cb7cfcc41f8aee144493bd6991a P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:16.487519 22214 raft_consensus.cc:2243] T 2cca0cb7cfcc41f8aee144493bd6991a P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:16.487783 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet 88f885774038498681d2710e51ff3040 (table default.table_492c567ddda14e77851270f34471730e [id=754c40502e614c5bb18230c852aa0016]) successfully deleted
I20260925 13:57:16.487507 22293 raft_consensus.cc:359] T abadb0b2c7d34451b7b23489e6d866c6 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.487936 22214 raft_consensus.cc:2272] T 2cca0cb7cfcc41f8aee144493bd6991a P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:16.488121 22293 raft_consensus.cc:385] T abadb0b2c7d34451b7b23489e6d866c6 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:16.488417 22293 raft_consensus.cc:740] T abadb0b2c7d34451b7b23489e6d866c6 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:16.475950 21777 tablet_service.cc:1558] Processing DeleteTablet for tablet d595ab2f7dc74fdfbf3546e477685a01 with delete_type TABLET_DATA_DELETED (replaced by tablet c965fcaf7eb246daa288d6183ce8a6ab) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:16.486182 22416 raft_consensus.cc:2272] T d595ab2f7dc74fdfbf3546e477685a01 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:16.489328 22293 consensus_queue.cc:260] T abadb0b2c7d34451b7b23489e6d866c6 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.489775 22214 ts_tablet_manager.cc:1916] T 2cca0cb7cfcc41f8aee144493bd6991a P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:16.494033 22293 ts_tablet_manager.cc:1434] T abadb0b2c7d34451b7b23489e6d866c6 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20260925 13:57:16.495393 22415 tablet_replica.cc:333] T 2cca0cb7cfcc41f8aee144493bd6991a P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:16.496115 22415 raft_consensus.cc:2243] T 2cca0cb7cfcc41f8aee144493bd6991a P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:16.496759 22415 pending_rounds.cc:70] T 2cca0cb7cfcc41f8aee144493bd6991a P 5e2731975dbf433aade4a859df9699e2: Trying to abort 1 pending ops.
I20260925 13:57:16.502074 22415 pending_rounds.cc:77] T 2cca0cb7cfcc41f8aee144493bd6991a P 5e2731975dbf433aade4a859df9699e2: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251629465866240 op_type: NO_OP noop_request { }
I20260925 13:57:16.502437 22415 raft_consensus.cc:2889] T 2cca0cb7cfcc41f8aee144493bd6991a P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:57:16.502767 22415 raft_consensus.cc:2272] T 2cca0cb7cfcc41f8aee144493bd6991a P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:16.473802 22212 tablet_bootstrap.cc:654] T abadb0b2c7d34451b7b23489e6d866c6 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:16.510713 22416 ts_tablet_manager.cc:1916] T d595ab2f7dc74fdfbf3546e477685a01 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:16.524827 21581 master_service.cc:960] ReplaceTablet: received request to replace tablet 044b30ad37d74f3d8aca8cd26520b035 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:16.524482 22214 ts_tablet_manager.cc:1929] T 2cca0cb7cfcc41f8aee144493bd6991a P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:16.528644 22214 log.cc:1199] T 2cca0cb7cfcc41f8aee144493bd6991a P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/2cca0cb7cfcc41f8aee144493bd6991a
I20260925 13:57:16.529635 22214 ts_tablet_manager.cc:1950] T 2cca0cb7cfcc41f8aee144493bd6991a P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:16.537448 22214 tablet_replica.cc:333] T d595ab2f7dc74fdfbf3546e477685a01 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:16.538074 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet 2cca0cb7cfcc41f8aee144493bd6991a (table default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]) successfully deleted
I20260925 13:57:16.541638 21588 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35994:
table { table_name: "default.Table_8b5db7d66475449d9772db568c9ded56" } modify_external_catalogs: true
I20260925 13:57:16.542361 21588 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35994:
table { table_name: "default.Table_8b5db7d66475449d9772db568c9ded56" } modify_external_catalogs: true
I20260925 13:57:16.545356 21586 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35994:
table { table_name: "default.Table_01a6dec392d24e9f89578f6f4274c704" } modify_external_catalogs: true
I20260925 13:57:16.542136 22415 ts_tablet_manager.cc:1916] T 2cca0cb7cfcc41f8aee144493bd6991a P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:16.546020 21586 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35994:
table { table_name: "default.Table_01a6dec392d24e9f89578f6f4274c704" } modify_external_catalogs: true
I20260925 13:57:16.548806 22212 tablet_bootstrap.cc:492] T abadb0b2c7d34451b7b23489e6d866c6 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:16.549268 22212 ts_tablet_manager.cc:1403] T abadb0b2c7d34451b7b23489e6d866c6 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.091s user 0.012s sys 0.006s
I20260925 13:57:16.551994 22212 raft_consensus.cc:359] T abadb0b2c7d34451b7b23489e6d866c6 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.553308 22212 raft_consensus.cc:385] T abadb0b2c7d34451b7b23489e6d866c6 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:16.553593 22212 raft_consensus.cc:740] T abadb0b2c7d34451b7b23489e6d866c6 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:16.553009 22214 raft_consensus.cc:2243] T d595ab2f7dc74fdfbf3546e477685a01 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:16.555189 22214 raft_consensus.cc:2272] T d595ab2f7dc74fdfbf3546e477685a01 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:16.557350 22214 ts_tablet_manager.cc:1916] T d595ab2f7dc74fdfbf3546e477685a01 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:16.557643 22212 consensus_queue.cc:260] T abadb0b2c7d34451b7b23489e6d866c6 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.571278 22212 ts_tablet_manager.cc:1434] T abadb0b2c7d34451b7b23489e6d866c6 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.022s user 0.004s sys 0.000s
I20260925 13:57:16.572261 21587 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35994:
table { table_name: "default.Table_b17685eaeada404790103bd89711435e" } new_table_name: "default.Table_05ac73cb4d774c0fbd89f99093663e76" modify_external_catalogs: true
I20260925 13:57:16.579222 22439 raft_consensus.cc:493] T 3a8c53e379a9407c920d91b05ebd2afc P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:16.579702 22211 tablet_bootstrap.cc:492] T abadb0b2c7d34451b7b23489e6d866c6 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:16.580127 22211 ts_tablet_manager.cc:1403] T abadb0b2c7d34451b7b23489e6d866c6 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.153s user 0.008s sys 0.004s
I20260925 13:57:16.579819 22439 raft_consensus.cc:515] T 3a8c53e379a9407c920d91b05ebd2afc P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.582103 22439 leader_election.cc:290] T 3a8c53e379a9407c920d91b05ebd2afc P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:16.582397 22211 raft_consensus.cc:359] T abadb0b2c7d34451b7b23489e6d866c6 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.583016 22211 raft_consensus.cc:385] T abadb0b2c7d34451b7b23489e6d866c6 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:16.584465 21959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8c53e379a9407c920d91b05ebd2afc" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:16.585139 21959 raft_consensus.cc:2468] T 3a8c53e379a9407c920d91b05ebd2afc P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:16.586339 22106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8c53e379a9407c920d91b05ebd2afc" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:16.586848 22106 raft_consensus.cc:2468] T 3a8c53e379a9407c920d91b05ebd2afc P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:16.588189 21732 leader_election.cc:304] T 3a8c53e379a9407c920d91b05ebd2afc P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:16.589129 22378 raft_consensus.cc:2804] T 3a8c53e379a9407c920d91b05ebd2afc P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:16.589465 22378 raft_consensus.cc:493] T 3a8c53e379a9407c920d91b05ebd2afc P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:16.589318 22214 ts_tablet_manager.cc:1929] T d595ab2f7dc74fdfbf3546e477685a01 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:16.589775 22378 raft_consensus.cc:3060] T 3a8c53e379a9407c920d91b05ebd2afc P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:16.590087 22214 log.cc:1199] T d595ab2f7dc74fdfbf3546e477685a01 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/d595ab2f7dc74fdfbf3546e477685a01
I20260925 13:57:16.590951 22214 ts_tablet_manager.cc:1950] T d595ab2f7dc74fdfbf3546e477685a01 P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:16.583273 22211 raft_consensus.cc:740] T abadb0b2c7d34451b7b23489e6d866c6 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:16.597286 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet d595ab2f7dc74fdfbf3546e477685a01 (table default.table_8b5db7d66475449d9772db568c9ded56 [id=7fb6c448c78344719796980b4be1cda9]) successfully deleted
I20260925 13:57:16.594233 22211 consensus_queue.cc:260] T abadb0b2c7d34451b7b23489e6d866c6 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.601065 21777 tablet_service.cc:1511] Processing CreateTablet for tablet c965fcaf7eb246daa288d6183ce8a6ab (DEFAULT_TABLE table=default.table_8b5db7d66475449d9772db568c9ded56 [id=7fb6c448c78344719796980b4be1cda9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:16.602281 21777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c965fcaf7eb246daa288d6183ce8a6ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:16.602531 21581 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet 044b30ad37d74f3d8aca8cd26520b035
I20260925 13:57:16.604229 21581 catalog_manager.cc:6934] ReplaceTablet: tablet 044b30ad37d74f3d8aca8cd26520b035 deleted and replaced by tablet fdf04c2fba51472883ec9141d42a6585
I20260925 13:57:16.608763 21776 tablet_service.cc:1558] Processing DeleteTablet for tablet 044b30ad37d74f3d8aca8cd26520b035 with delete_type TABLET_DATA_DELETED (replaced by tablet fdf04c2fba51472883ec9141d42a6585) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:16.602149 21937 tablet_service.cc:1511] Processing CreateTablet for tablet c965fcaf7eb246daa288d6183ce8a6ab (DEFAULT_TABLE table=default.table_8b5db7d66475449d9772db568c9ded56 [id=7fb6c448c78344719796980b4be1cda9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:16.610250 21937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c965fcaf7eb246daa288d6183ce8a6ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:16.611238 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet 044b30ad37d74f3d8aca8cd26520b035 with delete_type TABLET_DATA_DELETED (replaced by tablet fdf04c2fba51472883ec9141d42a6585) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:16.612444 22211 ts_tablet_manager.cc:1434] T abadb0b2c7d34451b7b23489e6d866c6 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.032s user 0.006s sys 0.000s
I20260925 13:57:16.617367 21938 tablet_service.cc:1558] Processing DeleteTablet for tablet 044b30ad37d74f3d8aca8cd26520b035 with delete_type TABLET_DATA_DELETED (replaced by tablet fdf04c2fba51472883ec9141d42a6585) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:16.618613 22087 tablet_service.cc:1511] Processing CreateTablet for tablet c965fcaf7eb246daa288d6183ce8a6ab (DEFAULT_TABLE table=default.table_8b5db7d66475449d9772db568c9ded56 [id=7fb6c448c78344719796980b4be1cda9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:16.619681 22087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c965fcaf7eb246daa288d6183ce8a6ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:16.616416 22378 raft_consensus.cc:515] T 3a8c53e379a9407c920d91b05ebd2afc P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.632893 22214 tablet_replica.cc:333] T 044b30ad37d74f3d8aca8cd26520b035 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:16.633652 22214 raft_consensus.cc:2243] T 044b30ad37d74f3d8aca8cd26520b035 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:16.634085 22214 raft_consensus.cc:2272] T 044b30ad37d74f3d8aca8cd26520b035 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:16.635017 22106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8c53e379a9407c920d91b05ebd2afc" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:16.636380 22106 raft_consensus.cc:3060] T 3a8c53e379a9407c920d91b05ebd2afc P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:16.636564 22293 tablet_bootstrap.cc:492] T c965fcaf7eb246daa288d6183ce8a6ab P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:16.632316 22378 leader_election.cc:290] T 3a8c53e379a9407c920d91b05ebd2afc P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Requested vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:16.636170 21959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8c53e379a9407c920d91b05ebd2afc" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
I20260925 13:57:16.643569 22293 tablet_bootstrap.cc:654] T c965fcaf7eb246daa288d6183ce8a6ab P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:16.643770 21959 raft_consensus.cc:3060] T 3a8c53e379a9407c920d91b05ebd2afc P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:16.650378 21959 raft_consensus.cc:2468] T 3a8c53e379a9407c920d91b05ebd2afc P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:16.651494 22214 ts_tablet_manager.cc:1916] T 044b30ad37d74f3d8aca8cd26520b035 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:16.651842 21732 leader_election.cc:304] T 3a8c53e379a9407c920d91b05ebd2afc P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:16.652616 22378 raft_consensus.cc:2804] T 3a8c53e379a9407c920d91b05ebd2afc P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:16.652163 22106 raft_consensus.cc:2468] T 3a8c53e379a9407c920d91b05ebd2afc P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:16.654462 22293 tablet_bootstrap.cc:492] T c965fcaf7eb246daa288d6183ce8a6ab P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:16.654856 22293 ts_tablet_manager.cc:1403] T c965fcaf7eb246daa288d6183ce8a6ab P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.009s
I20260925 13:57:16.655068 22378 raft_consensus.cc:697] T 3a8c53e379a9407c920d91b05ebd2afc P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Becoming Leader. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Running, Role: LEADER
I20260925 13:57:16.656967 22378 consensus_queue.cc:237] T 3a8c53e379a9407c920d91b05ebd2afc P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.631192 22416 ts_tablet_manager.cc:1929] T d595ab2f7dc74fdfbf3546e477685a01 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:16.658797 22416 log.cc:1199] T d595ab2f7dc74fdfbf3546e477685a01 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/d595ab2f7dc74fdfbf3546e477685a01
I20260925 13:57:16.659703 22416 ts_tablet_manager.cc:1950] T d595ab2f7dc74fdfbf3546e477685a01 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:16.664517 22293 raft_consensus.cc:359] T c965fcaf7eb246daa288d6183ce8a6ab P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.669766 22293 raft_consensus.cc:385] T c965fcaf7eb246daa288d6183ce8a6ab P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:16.670047 22293 raft_consensus.cc:740] T c965fcaf7eb246daa288d6183ce8a6ab P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:16.668043 22415 ts_tablet_manager.cc:1929] T 2cca0cb7cfcc41f8aee144493bd6991a P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:16.670871 22415 log.cc:1199] T 2cca0cb7cfcc41f8aee144493bd6991a P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/2cca0cb7cfcc41f8aee144493bd6991a
I20260925 13:57:16.671602 22415 ts_tablet_manager.cc:1950] T 2cca0cb7cfcc41f8aee144493bd6991a P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:16.674218 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet 2cca0cb7cfcc41f8aee144493bd6991a (table default.table_63ea1ad4e7904f8791170dd4da08d74e [id=af7e917a89ce4ca88be6ee9f143481cf]) successfully deleted
I20260925 13:57:16.676203 22293 consensus_queue.cc:260] T c965fcaf7eb246daa288d6183ce8a6ab P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.679263 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet d595ab2f7dc74fdfbf3546e477685a01 (table default.table_8b5db7d66475449d9772db568c9ded56 [id=7fb6c448c78344719796980b4be1cda9]) successfully deleted
I20260925 13:57:16.684935 22416 tablet_replica.cc:333] T 044b30ad37d74f3d8aca8cd26520b035 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:16.686673 22211 tablet_bootstrap.cc:492] T c965fcaf7eb246daa288d6183ce8a6ab P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:16.688179 22415 tablet_replica.cc:333] T d595ab2f7dc74fdfbf3546e477685a01 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:16.689332 22415 raft_consensus.cc:2243] T d595ab2f7dc74fdfbf3546e477685a01 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:16.690048 22415 raft_consensus.cc:2272] T d595ab2f7dc74fdfbf3546e477685a01 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:16.691738 22293 ts_tablet_manager.cc:1434] T c965fcaf7eb246daa288d6183ce8a6ab P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.037s user 0.003s sys 0.004s
I20260925 13:57:16.694756 22212 tablet_bootstrap.cc:492] T c965fcaf7eb246daa288d6183ce8a6ab P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:16.700558 22212 tablet_bootstrap.cc:654] T c965fcaf7eb246daa288d6183ce8a6ab P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:16.700929 22415 ts_tablet_manager.cc:1916] T d595ab2f7dc74fdfbf3546e477685a01 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:16.701519 22214 ts_tablet_manager.cc:1929] T 044b30ad37d74f3d8aca8cd26520b035 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:16.701848 22214 log.cc:1199] T 044b30ad37d74f3d8aca8cd26520b035 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/044b30ad37d74f3d8aca8cd26520b035
I20260925 13:57:16.702440 22214 ts_tablet_manager.cc:1950] T 044b30ad37d74f3d8aca8cd26520b035 P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:16.704942 22211 tablet_bootstrap.cc:654] T c965fcaf7eb246daa288d6183ce8a6ab P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:16.720311 21582 catalog_manager.cc:5838] T 3a8c53e379a9407c920d91b05ebd2afc P 5e2731975dbf433aade4a859df9699e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e2731975dbf433aade4a859df9699e2 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "5e2731975dbf433aade4a859df9699e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: UNKNOWN } } }
I20260925 13:57:16.722844 21937 tablet_service.cc:1511] Processing CreateTablet for tablet fdf04c2fba51472883ec9141d42a6585 (DEFAULT_TABLE table=default.table_bd714c618f7a4f16a69a69d5932b2014 [id=dfd05c11872140cfa86b2628c33ef6d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:16.723835 21937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdf04c2fba51472883ec9141d42a6585. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:16.724957 21777 tablet_service.cc:1511] Processing CreateTablet for tablet fdf04c2fba51472883ec9141d42a6585 (DEFAULT_TABLE table=default.table_bd714c618f7a4f16a69a69d5932b2014 [id=dfd05c11872140cfa86b2628c33ef6d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:16.725926 21777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdf04c2fba51472883ec9141d42a6585. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:16.729094 22087 tablet_service.cc:1511] Processing CreateTablet for tablet fdf04c2fba51472883ec9141d42a6585 (DEFAULT_TABLE table=default.table_bd714c618f7a4f16a69a69d5932b2014 [id=dfd05c11872140cfa86b2628c33ef6d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:16.730047 22087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdf04c2fba51472883ec9141d42a6585. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:16.732163 21581 master_service.cc:960] ReplaceTablet: received request to replace tablet 044b30ad37d74f3d8aca8cd26520b035 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:16.734692 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet 044b30ad37d74f3d8aca8cd26520b035 (table default.table_bd714c618f7a4f16a69a69d5932b2014 [id=dfd05c11872140cfa86b2628c33ef6d8]) successfully deleted
I20260925 13:57:16.756641 22416 raft_consensus.cc:2243] T 044b30ad37d74f3d8aca8cd26520b035 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:16.757083 22212 tablet_bootstrap.cc:492] T c965fcaf7eb246daa288d6183ce8a6ab P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:16.757521 22212 ts_tablet_manager.cc:1403] T c965fcaf7eb246daa288d6183ce8a6ab P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.063s user 0.008s sys 0.000s
I20260925 13:57:16.757632 21579 catalog_manager.cc:3541] renamed table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb] in HMS: new name default.table_d6100b0f2b294ab099c179a7681cb4ed
I20260925 13:57:16.764200 22293 tablet_bootstrap.cc:492] T fdf04c2fba51472883ec9141d42a6585 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:16.768201 22212 raft_consensus.cc:359] T c965fcaf7eb246daa288d6183ce8a6ab P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.768827 22212 raft_consensus.cc:385] T c965fcaf7eb246daa288d6183ce8a6ab P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:16.769100 22212 raft_consensus.cc:740] T c965fcaf7eb246daa288d6183ce8a6ab P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:16.770455 22293 tablet_bootstrap.cc:654] T fdf04c2fba51472883ec9141d42a6585 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:16.776613 22293 tablet_bootstrap.cc:492] T fdf04c2fba51472883ec9141d42a6585 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:16.777021 22293 ts_tablet_manager.cc:1403] T fdf04c2fba51472883ec9141d42a6585 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20260925 13:57:16.769824 22212 consensus_queue.cc:260] T c965fcaf7eb246daa288d6183ce8a6ab P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.757150 22416 raft_consensus.cc:2272] T 044b30ad37d74f3d8aca8cd26520b035 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:16.784634 22212 ts_tablet_manager.cc:1434] T c965fcaf7eb246daa288d6183ce8a6ab P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.027s user 0.007s sys 0.000s
I20260925 13:57:16.785707 22416 ts_tablet_manager.cc:1916] T 044b30ad37d74f3d8aca8cd26520b035 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:16.779155 22293 raft_consensus.cc:359] T fdf04c2fba51472883ec9141d42a6585 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.790472 22293 raft_consensus.cc:385] T fdf04c2fba51472883ec9141d42a6585 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:16.790716 22293 raft_consensus.cc:740] T fdf04c2fba51472883ec9141d42a6585 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:16.785571 22212 tablet_bootstrap.cc:492] T fdf04c2fba51472883ec9141d42a6585 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:16.795667 22211 tablet_bootstrap.cc:492] T c965fcaf7eb246daa288d6183ce8a6ab P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:16.796252 22211 ts_tablet_manager.cc:1403] T c965fcaf7eb246daa288d6183ce8a6ab P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.110s user 0.010s sys 0.000s
I20260925 13:57:16.800009 22211 raft_consensus.cc:359] T c965fcaf7eb246daa288d6183ce8a6ab P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.800639 22211 raft_consensus.cc:385] T c965fcaf7eb246daa288d6183ce8a6ab P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:16.800923 22211 raft_consensus.cc:740] T c965fcaf7eb246daa288d6183ce8a6ab P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:16.804020 22293 consensus_queue.cc:260] T fdf04c2fba51472883ec9141d42a6585 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.805752 22211 consensus_queue.cc:260] T c965fcaf7eb246daa288d6183ce8a6ab P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.807595 22211 ts_tablet_manager.cc:1434] T c965fcaf7eb246daa288d6183ce8a6ab P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.011s user 0.008s sys 0.000s
I20260925 13:57:16.811586 22212 tablet_bootstrap.cc:654] T fdf04c2fba51472883ec9141d42a6585 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:16.818925 22415 ts_tablet_manager.cc:1929] T d595ab2f7dc74fdfbf3546e477685a01 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:16.820443 22293 ts_tablet_manager.cc:1434] T fdf04c2fba51472883ec9141d42a6585 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.043s user 0.006s sys 0.000s
I20260925 13:57:16.833521 21581 master_service.cc:960] ReplaceTablet: received request to replace tablet fdf04c2fba51472883ec9141d42a6585 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:16.833649 22211 tablet_bootstrap.cc:492] T fdf04c2fba51472883ec9141d42a6585 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:16.819316 22415 log.cc:1199] T d595ab2f7dc74fdfbf3546e477685a01 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/d595ab2f7dc74fdfbf3546e477685a01
I20260925 13:57:16.843115 22415 ts_tablet_manager.cc:1950] T d595ab2f7dc74fdfbf3546e477685a01 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:16.849560 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet d595ab2f7dc74fdfbf3546e477685a01 (table default.table_8b5db7d66475449d9772db568c9ded56 [id=7fb6c448c78344719796980b4be1cda9]) successfully deleted
I20260925 13:57:16.858716 21777 tablet_service.cc:1558] Processing DeleteTablet for tablet fdf04c2fba51472883ec9141d42a6585 with delete_type TABLET_DATA_DELETED (replaced by tablet dc9cc9c6d85446b786e8ea4396b33a14) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:16.859090 22212 tablet_bootstrap.cc:492] T fdf04c2fba51472883ec9141d42a6585 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:16.859627 22212 ts_tablet_manager.cc:1403] T fdf04c2fba51472883ec9141d42a6585 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.074s user 0.012s sys 0.002s
I20260925 13:57:16.860720 21581 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet fdf04c2fba51472883ec9141d42a6585
I20260925 13:57:16.862416 21777 tablet_service.cc:1558] Processing DeleteTablet for tablet fdf04c2fba51472883ec9141d42a6585 with delete_type TABLET_DATA_DELETED (replaced by tablet dc9cc9c6d85446b786e8ea4396b33a14) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:16.862366 21581 catalog_manager.cc:6934] ReplaceTablet: tablet fdf04c2fba51472883ec9141d42a6585 deleted and replaced by tablet dc9cc9c6d85446b786e8ea4396b33a14
I20260925 13:57:16.863654 22087 tablet_service.cc:1558] Processing DeleteTablet for tablet fdf04c2fba51472883ec9141d42a6585 with delete_type TABLET_DATA_DELETED (replaced by tablet dc9cc9c6d85446b786e8ea4396b33a14) from {username='slave'} at 127.0.0.1:50052
W20260925 13:57:16.866111 21575 catalog_manager.cc:5043] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): delete failed for tablet fdf04c2fba51472883ec9141d42a6585 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fdf04c2fba51472883ec9141d42a6585 already in progress: creating tablet
I20260925 13:57:16.866846 21937 tablet_service.cc:1558] Processing DeleteTablet for tablet fdf04c2fba51472883ec9141d42a6585 with delete_type TABLET_DATA_DELETED (replaced by tablet dc9cc9c6d85446b786e8ea4396b33a14) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:16.876130 22212 raft_consensus.cc:359] T fdf04c2fba51472883ec9141d42a6585 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.876999 22212 raft_consensus.cc:385] T fdf04c2fba51472883ec9141d42a6585 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:16.877257 22212 raft_consensus.cc:740] T fdf04c2fba51472883ec9141d42a6585 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:16.877882 22212 consensus_queue.cc:260] T fdf04c2fba51472883ec9141d42a6585 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.886857 22415 tablet_replica.cc:333] T 044b30ad37d74f3d8aca8cd26520b035 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:16.888099 22211 tablet_bootstrap.cc:654] T fdf04c2fba51472883ec9141d42a6585 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:16.895850 22429 raft_consensus.cc:493] T c965fcaf7eb246daa288d6183ce8a6ab P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:16.896306 22429 raft_consensus.cc:515] T c965fcaf7eb246daa288d6183ce8a6ab P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.897629 22415 raft_consensus.cc:2243] T 044b30ad37d74f3d8aca8cd26520b035 P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:16.898219 22429 leader_election.cc:290] T c965fcaf7eb246daa288d6183ce8a6ab P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:16.898346 22415 pending_rounds.cc:70] T 044b30ad37d74f3d8aca8cd26520b035 P 5e2731975dbf433aade4a859df9699e2: Trying to abort 1 pending ops.
I20260925 13:57:16.898583 22415 pending_rounds.cc:77] T 044b30ad37d74f3d8aca8cd26520b035 P 5e2731975dbf433aade4a859df9699e2: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251628964515840 op_type: NO_OP noop_request { }
I20260925 13:57:16.898902 22415 raft_consensus.cc:2889] T 044b30ad37d74f3d8aca8cd26520b035 P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:57:16.899490 22378 raft_consensus.cc:493] T c965fcaf7eb246daa288d6183ce8a6ab P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:16.899899 22378 raft_consensus.cc:515] T c965fcaf7eb246daa288d6183ce8a6ab P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.901247 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c965fcaf7eb246daa288d6183ce8a6ab" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2" is_pre_election: true
I20260925 13:57:16.901768 21797 raft_consensus.cc:2468] T c965fcaf7eb246daa288d6183ce8a6ab P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee02f77594864349982492862d668a36 in term 0.
I20260925 13:57:16.901762 22378 leader_election.cc:290] T c965fcaf7eb246daa288d6183ce8a6ab P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:16.903118 22040 leader_election.cc:304] T c965fcaf7eb246daa288d6183ce8a6ab P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:16.904947 22106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c965fcaf7eb246daa288d6183ce8a6ab" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:16.905475 22106 raft_consensus.cc:2468] T c965fcaf7eb246daa288d6183ce8a6ab P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:16.907032 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c965fcaf7eb246daa288d6183ce8a6ab" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:16.907769 22415 raft_consensus.cc:2272] T 044b30ad37d74f3d8aca8cd26520b035 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:16.907740 21960 raft_consensus.cc:2468] T c965fcaf7eb246daa288d6183ce8a6ab P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:16.909235 21959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c965fcaf7eb246daa288d6183ce8a6ab" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:16.909811 21959 raft_consensus.cc:2468] T c965fcaf7eb246daa288d6183ce8a6ab P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee02f77594864349982492862d668a36 in term 0.
I20260925 13:57:16.910933 22415 ts_tablet_manager.cc:1916] T 044b30ad37d74f3d8aca8cd26520b035 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:16.912590 21732 leader_election.cc:304] T c965fcaf7eb246daa288d6183ce8a6ab P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:16.912914 22087 tablet_service.cc:1558] Processing DeleteTablet for tablet fdf04c2fba51472883ec9141d42a6585 with delete_type TABLET_DATA_DELETED (replaced by tablet dc9cc9c6d85446b786e8ea4396b33a14) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:16.913347 22378 raft_consensus.cc:2804] T c965fcaf7eb246daa288d6183ce8a6ab P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:16.913647 22378 raft_consensus.cc:493] T c965fcaf7eb246daa288d6183ce8a6ab P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:16.913975 22378 raft_consensus.cc:3060] T c965fcaf7eb246daa288d6183ce8a6ab P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:16.916231 22211 tablet_bootstrap.cc:492] T fdf04c2fba51472883ec9141d42a6585 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:16.916616 22211 ts_tablet_manager.cc:1403] T fdf04c2fba51472883ec9141d42a6585 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.084s user 0.010s sys 0.000s
W20260925 13:57:16.921515 21575 catalog_manager.cc:5043] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): delete failed for tablet fdf04c2fba51472883ec9141d42a6585 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fdf04c2fba51472883ec9141d42a6585 already in progress: creating tablet
I20260925 13:57:16.925292 22378 raft_consensus.cc:515] T c965fcaf7eb246daa288d6183ce8a6ab P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.927649 22429 raft_consensus.cc:2804] T c965fcaf7eb246daa288d6183ce8a6ab P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:16.928020 22429 raft_consensus.cc:493] T c965fcaf7eb246daa288d6183ce8a6ab P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:16.928347 22429 raft_consensus.cc:3060] T c965fcaf7eb246daa288d6183ce8a6ab P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:16.935961 22416 ts_tablet_manager.cc:1929] T 044b30ad37d74f3d8aca8cd26520b035 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:16.936321 22416 log.cc:1199] T 044b30ad37d74f3d8aca8cd26520b035 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/044b30ad37d74f3d8aca8cd26520b035
I20260925 13:57:16.936946 22416 ts_tablet_manager.cc:1950] T 044b30ad37d74f3d8aca8cd26520b035 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:16.936815 21937 tablet_service.cc:1511] Processing CreateTablet for tablet dc9cc9c6d85446b786e8ea4396b33a14 (DEFAULT_TABLE table=default.table_bd714c618f7a4f16a69a69d5932b2014 [id=dfd05c11872140cfa86b2628c33ef6d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:16.938315 21937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc9cc9c6d85446b786e8ea4396b33a14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:16.944340 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet 044b30ad37d74f3d8aca8cd26520b035 (table default.table_bd714c618f7a4f16a69a69d5932b2014 [id=dfd05c11872140cfa86b2628c33ef6d8]) successfully deleted
W20260925 13:57:16.945154 21575 catalog_manager.cc:5043] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): delete failed for tablet fdf04c2fba51472883ec9141d42a6585 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fdf04c2fba51472883ec9141d42a6585 already in progress: creating tablet
I20260925 13:57:16.947124 22087 tablet_service.cc:1511] Processing CreateTablet for tablet dc9cc9c6d85446b786e8ea4396b33a14 (DEFAULT_TABLE table=default.table_bd714c618f7a4f16a69a69d5932b2014 [id=dfd05c11872140cfa86b2628c33ef6d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:16.948098 22087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc9cc9c6d85446b786e8ea4396b33a14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:16.934829 21777 tablet_service.cc:1511] Processing CreateTablet for tablet dc9cc9c6d85446b786e8ea4396b33a14 (DEFAULT_TABLE table=default.table_bd714c618f7a4f16a69a69d5932b2014 [id=dfd05c11872140cfa86b2628c33ef6d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:16.942755 22439 raft_consensus.cc:493] T abadb0b2c7d34451b7b23489e6d866c6 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:16.955643 22439 raft_consensus.cc:515] T abadb0b2c7d34451b7b23489e6d866c6 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.956077 21777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc9cc9c6d85446b786e8ea4396b33a14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:16.957527 22439 leader_election.cc:290] T abadb0b2c7d34451b7b23489e6d866c6 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:16.959282 21959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abadb0b2c7d34451b7b23489e6d866c6" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:16.959926 21959 raft_consensus.cc:2468] T abadb0b2c7d34451b7b23489e6d866c6 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:16.961088 21732 leader_election.cc:304] T abadb0b2c7d34451b7b23489e6d866c6 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:16.961826 22439 raft_consensus.cc:2804] T abadb0b2c7d34451b7b23489e6d866c6 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:16.962129 22439 raft_consensus.cc:493] T abadb0b2c7d34451b7b23489e6d866c6 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:16.962419 22439 raft_consensus.cc:3060] T abadb0b2c7d34451b7b23489e6d866c6 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:16.968206 22106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abadb0b2c7d34451b7b23489e6d866c6" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:16.968776 22106 raft_consensus.cc:2468] T abadb0b2c7d34451b7b23489e6d866c6 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:16.970371 22378 leader_election.cc:290] T c965fcaf7eb246daa288d6183ce8a6ab P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Requested vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:16.971918 22106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c965fcaf7eb246daa288d6183ce8a6ab" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:16.972769 21959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c965fcaf7eb246daa288d6183ce8a6ab" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
I20260925 13:57:16.973297 21959 raft_consensus.cc:3060] T c965fcaf7eb246daa288d6183ce8a6ab P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:16.977640 22429 raft_consensus.cc:515] T c965fcaf7eb246daa288d6183ce8a6ab P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.919313 22211 raft_consensus.cc:359] T fdf04c2fba51472883ec9141d42a6585 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.979655 22106 raft_consensus.cc:2393] T c965fcaf7eb246daa288d6183ce8a6ab P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e2731975dbf433aade4a859df9699e2 in current term 1: Already voted for candidate ee02f77594864349982492862d668a36 in this term.
I20260925 13:57:16.980456 21580 catalog_manager.cc:2213] created HMS catalog entry for table [id=7377fa3ea9664b47a930d9c5f88c60fa]
I20260925 13:57:16.981540 22211 raft_consensus.cc:385] T fdf04c2fba51472883ec9141d42a6585 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:16.981875 22211 raft_consensus.cc:740] T fdf04c2fba51472883ec9141d42a6585 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:16.991272 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c965fcaf7eb246daa288d6183ce8a6ab" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2"
I20260925 13:57:16.992022 21797 raft_consensus.cc:2393] T c965fcaf7eb246daa288d6183ce8a6ab P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ee02f77594864349982492862d668a36 in current term 1: Already voted for candidate 5e2731975dbf433aade4a859df9699e2 in this term.
I20260925 13:57:16.993395 22211 consensus_queue.cc:260] T fdf04c2fba51472883ec9141d42a6585 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:16.994649 22293 tablet_bootstrap.cc:492] T dc9cc9c6d85446b786e8ea4396b33a14 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:16.994650 22429 leader_election.cc:290] T c965fcaf7eb246daa288d6183ce8a6ab P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 election: Requested vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:16.995968 21580 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35994 (request call id 46) took 3519 ms. Trace:
I20260925 13:57:16.999660 22293 tablet_bootstrap.cc:654] T dc9cc9c6d85446b786e8ea4396b33a14 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:16.996250 21580 rpcz_store.cc:276] 0925 13:57:13.476410 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:13.513935 (+ 37525us) service_pool.cc:224] Handling call
0925 13:57:16.319072 (+2805137us) catalog_manager.cc:2140] Acquired catalog manager lock
0925 13:57:16.322178 (+ 3106us) catalog_manager.cc:2186] Created new table and tablet info
0925 13:57:16.980309 (+658131us) catalog_manager.cc:2212] Created new table in HMS catalog
0925 13:57:16.995277 (+ 14968us) catalog_manager.cc:2246] Wrote table and tablets to system table
0925 13:57:16.995573 (+ 296us) catalog_manager.cc:2280] Inserted table and tablets into CatalogManager maps
0925 13:57:16.995908 (+ 335us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:57:16.982199 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:57:16.982599 (+ 400us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:57:16.982622 (+ 23us) write_op.cc:435] Acquired schema lock
0925 13:57:16.982634 (+ 12us) tablet.cc:662] Decoding operations
0925 13:57:16.982904 (+ 270us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:57:16.982943 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0925 13:57:16.982965 (+ 22us) tablet.cc:685] Acquiring locks for 2 operations
0925 13:57:16.983182 (+ 217us) tablet.cc:701] Row locks acquired
0925 13:57:16.983196 (+ 14us) write_op.cc:260] PREPARE: finished
0925 13:57:16.983308 (+ 112us) write_op.cc:270] Start()
0925 13:57:16.983418 (+ 110us) write_op.cc:276] Timestamp: P: 1790344636983204 usec, L: 0
0925 13:57:16.983432 (+ 14us) op_driver.cc:348] REPLICATION: starting
0925 13:57:16.983949 (+ 517us) log.cc:844] Serialized 432 byte log entry
0925 13:57:16.993511 (+ 9562us) op_driver.cc:464] REPLICATION: finished
0925 13:57:16.993880 (+ 369us) write_op.cc:301] APPLY: starting
0925 13:57:16.993922 (+ 42us) tablet.cc:1394] starting BulkCheckPresence
0925 13:57:16.994004 (+ 82us) tablet.cc:1397] finished BulkCheckPresence
0925 13:57:16.994070 (+ 66us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:57:16.994405 (+ 335us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:57:16.994522 (+ 117us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 13:57:16.994553 (+ 31us) write_op.cc:312] APPLY: finished
0925 13:57:16.994691 (+ 138us) log.cc:844] Serialized 28 byte log entry
0925 13:57:16.994801 (+ 110us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:16.994854 (+ 53us) write_op.cc:454] Released schema lock
0925 13:57:16.994879 (+ 25us) write_op.cc:341] FINISH: Updating metrics
0925 13:57:16.995049 (+ 170us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:16.995066 (+ 17us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":434192,"Hive Metastore.run_cpu_time_us":802,"Hive Metastore.run_wall_time_us":223336,"child_traces":[["op",{"apply.queue_time_us":235,"apply.run_cpu_time_us":1151,"apply.run_wall_time_us":1152,"num_ops":2,"prepare.queue_time_us":117,"prepare.run_cpu_time_us":2131,"prepare.run_wall_time_us":2131,"raft.queue_time_us":3080,"raft.run_cpu_time_us":750,"raft.run_wall_time_us":750,"replication_time_us":9976}]]}
I20260925 13:57:17.005683 22439 raft_consensus.cc:515] T abadb0b2c7d34451b7b23489e6d866c6 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:17.014204 21580 master_service.cc:960] ReplaceTablet: received request to replace tablet 044b30ad37d74f3d8aca8cd26520b035 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:17.016255 22212 ts_tablet_manager.cc:1434] T fdf04c2fba51472883ec9141d42a6585 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.156s user 0.005s sys 0.001s
I20260925 13:57:17.017216 22212 tablet_bootstrap.cc:492] T dc9cc9c6d85446b786e8ea4396b33a14 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:17.024627 22211 ts_tablet_manager.cc:1434] T fdf04c2fba51472883ec9141d42a6585 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.108s user 0.006s sys 0.000s
I20260925 13:57:17.026180 22106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abadb0b2c7d34451b7b23489e6d866c6" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:17.026634 22106 raft_consensus.cc:3060] T abadb0b2c7d34451b7b23489e6d866c6 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:17.029078 22415 ts_tablet_manager.cc:1929] T 044b30ad37d74f3d8aca8cd26520b035 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:17.029392 22415 log.cc:1199] T 044b30ad37d74f3d8aca8cd26520b035 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/044b30ad37d74f3d8aca8cd26520b035
I20260925 13:57:17.030072 22415 ts_tablet_manager.cc:1950] T 044b30ad37d74f3d8aca8cd26520b035 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:17.033766 22106 raft_consensus.cc:2468] T abadb0b2c7d34451b7b23489e6d866c6 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:17.035393 21732 leader_election.cc:304] T abadb0b2c7d34451b7b23489e6d866c6 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:17.036141 22378 raft_consensus.cc:2804] T abadb0b2c7d34451b7b23489e6d866c6 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:16.979336 21959 raft_consensus.cc:2468] T c965fcaf7eb246daa288d6183ce8a6ab P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:17.038403 21732 leader_election.cc:304] T c965fcaf7eb246daa288d6183ce8a6ab P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters: ee02f77594864349982492862d668a36
I20260925 13:57:17.044292 22293 tablet_bootstrap.cc:492] T dc9cc9c6d85446b786e8ea4396b33a14 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:17.044850 22293 ts_tablet_manager.cc:1403] T dc9cc9c6d85446b786e8ea4396b33a14 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.050s user 0.013s sys 0.000s
I20260925 13:57:17.048768 22293 raft_consensus.cc:359] T dc9cc9c6d85446b786e8ea4396b33a14 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:17.049371 22293 raft_consensus.cc:385] T dc9cc9c6d85446b786e8ea4396b33a14 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:17.049612 22293 raft_consensus.cc:740] T dc9cc9c6d85446b786e8ea4396b33a14 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:17.049916 22462 raft_consensus.cc:2804] T c965fcaf7eb246daa288d6183ce8a6ab P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:17.050469 22462 raft_consensus.cc:697] T c965fcaf7eb246daa288d6183ce8a6ab P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Becoming Leader. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Running, Role: LEADER
I20260925 13:57:17.051447 22462 consensus_queue.cc:237] T c965fcaf7eb246daa288d6183ce8a6ab P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:17.053249 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet 044b30ad37d74f3d8aca8cd26520b035 (table default.table_bd714c618f7a4f16a69a69d5932b2014 [id=dfd05c11872140cfa86b2628c33ef6d8]) successfully deleted
I20260925 13:57:17.053547 22439 leader_election.cc:290] T abadb0b2c7d34451b7b23489e6d866c6 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Requested vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:17.056869 22415 tablet_replica.cc:333] T fdf04c2fba51472883ec9141d42a6585 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:17.057570 22415 raft_consensus.cc:2243] T fdf04c2fba51472883ec9141d42a6585 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:17.058060 22415 raft_consensus.cc:2272] T fdf04c2fba51472883ec9141d42a6585 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:17.054337 22293 consensus_queue.cc:260] T dc9cc9c6d85446b786e8ea4396b33a14 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:17.061146 22378 raft_consensus.cc:697] T abadb0b2c7d34451b7b23489e6d866c6 P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Becoming Leader. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Running, Role: LEADER
I20260925 13:57:17.061944 22378 consensus_queue.cc:237] T abadb0b2c7d34451b7b23489e6d866c6 P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:17.063016 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c965fcaf7eb246daa288d6183ce8a6ab" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
I20260925 13:57:17.063760 21960 raft_consensus.cc:2393] T c965fcaf7eb246daa288d6183ce8a6ab P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ee02f77594864349982492862d668a36 in current term 1: Already voted for candidate 5e2731975dbf433aade4a859df9699e2 in this term.
I20260925 13:57:17.067395 22293 ts_tablet_manager.cc:1434] T dc9cc9c6d85446b786e8ea4396b33a14 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.022s user 0.006s sys 0.000s
I20260925 13:57:17.072963 22043 leader_election.cc:304] T c965fcaf7eb246daa288d6183ce8a6ab P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:17.066144 21958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abadb0b2c7d34451b7b23489e6d866c6" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
I20260925 13:57:17.069254 22211 tablet_bootstrap.cc:492] T dc9cc9c6d85446b786e8ea4396b33a14 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:17.074050 21958 raft_consensus.cc:3060] T abadb0b2c7d34451b7b23489e6d866c6 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:17.075150 22429 raft_consensus.cc:2749] T c965fcaf7eb246daa288d6183ce8a6ab P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260925 13:57:17.081177 22211 tablet_bootstrap.cc:654] T dc9cc9c6d85446b786e8ea4396b33a14 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:17.082753 21937 tablet_service.cc:1558] Processing DeleteTablet for tablet fdf04c2fba51472883ec9141d42a6585 with delete_type TABLET_DATA_DELETED (replaced by tablet dc9cc9c6d85446b786e8ea4396b33a14) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:17.083307 22416 tablet_replica.cc:333] T fdf04c2fba51472883ec9141d42a6585 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:17.083972 22416 raft_consensus.cc:2243] T fdf04c2fba51472883ec9141d42a6585 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:17.084388 22416 raft_consensus.cc:2272] T fdf04c2fba51472883ec9141d42a6585 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:17.084990 22087 tablet_service.cc:1558] Processing DeleteTablet for tablet fdf04c2fba51472883ec9141d42a6585 with delete_type TABLET_DATA_DELETED (replaced by tablet dc9cc9c6d85446b786e8ea4396b33a14) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:17.085908 22214 tablet_replica.cc:333] T fdf04c2fba51472883ec9141d42a6585 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:17.086525 22214 raft_consensus.cc:2243] T fdf04c2fba51472883ec9141d42a6585 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:17.086896 22214 raft_consensus.cc:2272] T fdf04c2fba51472883ec9141d42a6585 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:17.088181 22415 ts_tablet_manager.cc:1916] T fdf04c2fba51472883ec9141d42a6585 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:17.088719 22214 ts_tablet_manager.cc:1916] T fdf04c2fba51472883ec9141d42a6585 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:17.094086 22416 ts_tablet_manager.cc:1916] T fdf04c2fba51472883ec9141d42a6585 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:17.097007 22212 tablet_bootstrap.cc:654] T dc9cc9c6d85446b786e8ea4396b33a14 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:17.109899 22415 ts_tablet_manager.cc:1929] T fdf04c2fba51472883ec9141d42a6585 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:17.110214 22415 log.cc:1199] T fdf04c2fba51472883ec9141d42a6585 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/fdf04c2fba51472883ec9141d42a6585
I20260925 13:57:17.110839 22415 ts_tablet_manager.cc:1950] T fdf04c2fba51472883ec9141d42a6585 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:17.120316 21958 raft_consensus.cc:2468] T abadb0b2c7d34451b7b23489e6d866c6 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:17.125025 21582 master_service.cc:960] ReplaceTablet: received request to replace tablet c965fcaf7eb246daa288d6183ce8a6ab from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:17.126022 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet fdf04c2fba51472883ec9141d42a6585 (table default.table_bd714c618f7a4f16a69a69d5932b2014 [id=dfd05c11872140cfa86b2628c33ef6d8]) successfully deleted
W20260925 13:57:17.126783 21572 catalog_manager.cc:5030] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): delete failed for tablet fdf04c2fba51472883ec9141d42a6585 because the tablet was not found. No further retry: Not found: Tablet not found: fdf04c2fba51472883ec9141d42a6585
I20260925 13:57:17.141808 21581 catalog_manager.cc:5838] T abadb0b2c7d34451b7b23489e6d866c6 P 5e2731975dbf433aade4a859df9699e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e2731975dbf433aade4a859df9699e2 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "5e2731975dbf433aade4a859df9699e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: UNKNOWN } } }
I20260925 13:57:17.148340 21777 tablet_service.cc:1511] Processing CreateTablet for tablet e4fe2c4187b34b5dba98873ab0d5abdb (DEFAULT_TABLE table=default.table_61d48f9061c741fc8aa556e3edd984fb [id=7377fa3ea9664b47a930d9c5f88c60fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:17.148671 22087 tablet_service.cc:1511] Processing CreateTablet for tablet e4fe2c4187b34b5dba98873ab0d5abdb (DEFAULT_TABLE table=default.table_61d48f9061c741fc8aa556e3edd984fb [id=7377fa3ea9664b47a930d9c5f88c60fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:17.149324 21777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4fe2c4187b34b5dba98873ab0d5abdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:17.150192 22087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4fe2c4187b34b5dba98873ab0d5abdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:17.156564 21937 tablet_service.cc:1511] Processing CreateTablet for tablet e4fe2c4187b34b5dba98873ab0d5abdb (DEFAULT_TABLE table=default.table_61d48f9061c741fc8aa556e3edd984fb [id=7377fa3ea9664b47a930d9c5f88c60fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:17.165494 21937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4fe2c4187b34b5dba98873ab0d5abdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:17.199165 21582 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet c965fcaf7eb246daa288d6183ce8a6ab
I20260925 13:57:17.202667 21938 tablet_service.cc:1558] Processing DeleteTablet for tablet c965fcaf7eb246daa288d6183ce8a6ab with delete_type TABLET_DATA_DELETED (replaced by tablet edfa0efda2a24efd89c38ef688c917d5) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:17.203979 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet c965fcaf7eb246daa288d6183ce8a6ab with delete_type TABLET_DATA_DELETED (replaced by tablet edfa0efda2a24efd89c38ef688c917d5) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:17.207686 21776 tablet_service.cc:1558] Processing DeleteTablet for tablet c965fcaf7eb246daa288d6183ce8a6ab with delete_type TABLET_DATA_DELETED (replaced by tablet edfa0efda2a24efd89c38ef688c917d5) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:17.208635 22415 tablet_replica.cc:333] T c965fcaf7eb246daa288d6183ce8a6ab P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:17.209599 22415 raft_consensus.cc:2243] T c965fcaf7eb246daa288d6183ce8a6ab P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:17.210510 22415 pending_rounds.cc:70] T c965fcaf7eb246daa288d6183ce8a6ab P 5e2731975dbf433aade4a859df9699e2: Trying to abort 1 pending ops.
I20260925 13:57:17.210784 22415 pending_rounds.cc:77] T c965fcaf7eb246daa288d6183ce8a6ab P 5e2731975dbf433aade4a859df9699e2: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251633587601408 op_type: NO_OP noop_request { }
I20260925 13:57:17.211259 22415 raft_consensus.cc:2889] T c965fcaf7eb246daa288d6183ce8a6ab P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:57:17.212602 22415 raft_consensus.cc:2272] T c965fcaf7eb246daa288d6183ce8a6ab P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:17.214867 22416 ts_tablet_manager.cc:1929] T fdf04c2fba51472883ec9141d42a6585 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:17.217777 21582 catalog_manager.cc:6934] ReplaceTablet: tablet c965fcaf7eb246daa288d6183ce8a6ab deleted and replaced by tablet edfa0efda2a24efd89c38ef688c917d5
I20260925 13:57:17.218108 22214 ts_tablet_manager.cc:1929] T fdf04c2fba51472883ec9141d42a6585 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:17.218459 22214 log.cc:1199] T fdf04c2fba51472883ec9141d42a6585 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/fdf04c2fba51472883ec9141d42a6585
I20260925 13:57:17.219072 22214 ts_tablet_manager.cc:1950] T fdf04c2fba51472883ec9141d42a6585 P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:17.219002 21580 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35994:
name: "default.Table_fce30b75d46f47089d7a31e04841dcdf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:57:17.221941 22415 ts_tablet_manager.cc:1916] T c965fcaf7eb246daa288d6183ce8a6ab P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:17.240908 22211 tablet_bootstrap.cc:492] T dc9cc9c6d85446b786e8ea4396b33a14 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:17.241345 22211 ts_tablet_manager.cc:1403] T dc9cc9c6d85446b786e8ea4396b33a14 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.172s user 0.007s sys 0.003s
I20260925 13:57:17.243747 22293 tablet_bootstrap.cc:492] T e4fe2c4187b34b5dba98873ab0d5abdb P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:17.215227 22416 log.cc:1199] T fdf04c2fba51472883ec9141d42a6585 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/fdf04c2fba51472883ec9141d42a6585
I20260925 13:57:17.247681 22416 ts_tablet_manager.cc:1950] T fdf04c2fba51472883ec9141d42a6585 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:17.249851 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet fdf04c2fba51472883ec9141d42a6585 (table default.table_bd714c618f7a4f16a69a69d5932b2014 [id=dfd05c11872140cfa86b2628c33ef6d8]) successfully deleted
I20260925 13:57:17.250582 22416 tablet_replica.cc:333] T c965fcaf7eb246daa288d6183ce8a6ab P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:17.251171 22416 raft_consensus.cc:2243] T c965fcaf7eb246daa288d6183ce8a6ab P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:17.251552 22416 raft_consensus.cc:2272] T c965fcaf7eb246daa288d6183ce8a6ab P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:17.253297 22293 tablet_bootstrap.cc:654] T e4fe2c4187b34b5dba98873ab0d5abdb P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:17.270921 22293 tablet_bootstrap.cc:492] T e4fe2c4187b34b5dba98873ab0d5abdb P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:17.271312 22293 ts_tablet_manager.cc:1403] T e4fe2c4187b34b5dba98873ab0d5abdb P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.028s user 0.007s sys 0.004s
I20260925 13:57:17.274641 22293 raft_consensus.cc:359] T e4fe2c4187b34b5dba98873ab0d5abdb P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:17.275229 22293 raft_consensus.cc:385] T e4fe2c4187b34b5dba98873ab0d5abdb P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:17.275501 22293 raft_consensus.cc:740] T e4fe2c4187b34b5dba98873ab0d5abdb P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:17.276077 22293 consensus_queue.cc:260] T e4fe2c4187b34b5dba98873ab0d5abdb P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:17.277825 22293 ts_tablet_manager.cc:1434] T e4fe2c4187b34b5dba98873ab0d5abdb P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260925 13:57:17.284909 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet fdf04c2fba51472883ec9141d42a6585 (table default.table_bd714c618f7a4f16a69a69d5932b2014 [id=dfd05c11872140cfa86b2628c33ef6d8]) successfully deleted
I20260925 13:57:17.285657 22214 tablet_replica.cc:333] T c965fcaf7eb246daa288d6183ce8a6ab P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:17.286363 22214 raft_consensus.cc:2243] T c965fcaf7eb246daa288d6183ce8a6ab P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:17.286751 22214 raft_consensus.cc:2272] T c965fcaf7eb246daa288d6183ce8a6ab P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:17.288319 22214 ts_tablet_manager.cc:1916] T c965fcaf7eb246daa288d6183ce8a6ab P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:17.290166 22211 raft_consensus.cc:359] T dc9cc9c6d85446b786e8ea4396b33a14 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:17.291452 22211 raft_consensus.cc:385] T dc9cc9c6d85446b786e8ea4396b33a14 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:17.291713 22211 raft_consensus.cc:740] T dc9cc9c6d85446b786e8ea4396b33a14 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:17.292390 22211 consensus_queue.cc:260] T dc9cc9c6d85446b786e8ea4396b33a14 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:17.296602 22211 ts_tablet_manager.cc:1434] T dc9cc9c6d85446b786e8ea4396b33a14 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.055s user 0.007s sys 0.001s
I20260925 13:57:17.306555 22416 ts_tablet_manager.cc:1916] T c965fcaf7eb246daa288d6183ce8a6ab P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:17.293183 21581 master_service.cc:960] ReplaceTablet: received request to replace tablet d267def3f0864435a0d2c40fd7052eab from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:17.329012 21583 catalog_manager.cc:2213] created HMS catalog entry for table [id=420cd09cc1184f6e9fb38b2cbc3f6fce]
I20260925 13:57:17.347371 22212 tablet_bootstrap.cc:492] T dc9cc9c6d85446b786e8ea4396b33a14 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:17.347769 22212 ts_tablet_manager.cc:1403] T dc9cc9c6d85446b786e8ea4396b33a14 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.331s user 0.011s sys 0.004s
I20260925 13:57:17.374527 21583 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35994 (request call id 55) took 2320 ms. Trace:
I20260925 13:57:17.374821 21583 rpcz_store.cc:276] 0925 13:57:15.054164 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:15.054391 (+ 227us) service_pool.cc:224] Handling call
0925 13:57:16.323900 (+1269509us) catalog_manager.cc:2140] Acquired catalog manager lock
0925 13:57:16.325486 (+ 1586us) catalog_manager.cc:2186] Created new table and tablet info
0925 13:57:17.328863 (+1003377us) catalog_manager.cc:2212] Created new table in HMS catalog
0925 13:57:17.373771 (+ 44908us) catalog_manager.cc:2246] Wrote table and tablets to system table
0925 13:57:17.374102 (+ 331us) catalog_manager.cc:2280] Inserted table and tablets into CatalogManager maps
0925 13:57:17.374468 (+ 366us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:57:17.349156 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:57:17.349754 (+ 598us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:57:17.349786 (+ 32us) write_op.cc:435] Acquired schema lock
0925 13:57:17.349805 (+ 19us) tablet.cc:662] Decoding operations
0925 13:57:17.350148 (+ 343us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:57:17.350205 (+ 57us) write_op.cc:641] Partition lock acquired for write op
0925 13:57:17.350234 (+ 29us) tablet.cc:685] Acquiring locks for 2 operations
0925 13:57:17.350490 (+ 256us) tablet.cc:701] Row locks acquired
0925 13:57:17.350508 (+ 18us) write_op.cc:260] PREPARE: finished
0925 13:57:17.350636 (+ 128us) write_op.cc:270] Start()
0925 13:57:17.350731 (+ 95us) write_op.cc:276] Timestamp: P: 1790344637350564 usec, L: 0
0925 13:57:17.350752 (+ 21us) op_driver.cc:348] REPLICATION: starting
0925 13:57:17.351361 (+ 609us) log.cc:844] Serialized 432 byte log entry
0925 13:57:17.371567 (+ 20206us) op_driver.cc:464] REPLICATION: finished
0925 13:57:17.372586 (+ 1019us) write_op.cc:301] APPLY: starting
0925 13:57:17.372631 (+ 45us) tablet.cc:1394] starting BulkCheckPresence
0925 13:57:17.372713 (+ 82us) tablet.cc:1397] finished BulkCheckPresence
0925 13:57:17.372787 (+ 74us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:57:17.373093 (+ 306us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:57:17.373210 (+ 117us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 13:57:17.373246 (+ 36us) write_op.cc:312] APPLY: finished
0925 13:57:17.373391 (+ 145us) log.cc:844] Serialized 28 byte log entry
0925 13:57:17.373510 (+ 119us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:17.373561 (+ 51us) write_op.cc:454] Released schema lock
0925 13:57:17.373587 (+ 26us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"Hive Metastore.queue_time_us":654233,"Hive Metastore.run_cpu_time_us":790,"Hive Metastore.run_wall_time_us":348410,"spinlock_wait_cycles":5888,"child_traces":[["op",{"apply.queue_time_us":891,"apply.run_cpu_time_us":1175,"apply.run_wall_time_us":1175,"num_ops":2,"prepare.queue_time_us":18547,"replication_time_us":20734}]]}
I20260925 13:57:17.380146 22212 raft_consensus.cc:359] T dc9cc9c6d85446b786e8ea4396b33a14 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:17.380827 22212 raft_consensus.cc:385] T dc9cc9c6d85446b786e8ea4396b33a14 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:17.381130 22212 raft_consensus.cc:740] T dc9cc9c6d85446b786e8ea4396b33a14 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:17.402639 22212 consensus_queue.cc:260] T dc9cc9c6d85446b786e8ea4396b33a14 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:17.409257 22212 ts_tablet_manager.cc:1434] T dc9cc9c6d85446b786e8ea4396b33a14 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.061s user 0.005s sys 0.000s
I20260925 13:57:17.423518 22462 consensus_queue.cc:1048] T 3a8c53e379a9407c920d91b05ebd2afc P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:17.424386 21938 tablet_service.cc:1511] Processing CreateTablet for tablet edfa0efda2a24efd89c38ef688c917d5 (DEFAULT_TABLE table=default.table_8b5db7d66475449d9772db568c9ded56 [id=7fb6c448c78344719796980b4be1cda9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:17.425432 21938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edfa0efda2a24efd89c38ef688c917d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:17.428930 22083 tablet_service.cc:1511] Processing CreateTablet for tablet edfa0efda2a24efd89c38ef688c917d5 (DEFAULT_TABLE table=default.table_8b5db7d66475449d9772db568c9ded56 [id=7fb6c448c78344719796980b4be1cda9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:17.429903 22083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edfa0efda2a24efd89c38ef688c917d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:17.431919 21777 tablet_service.cc:1511] Processing CreateTablet for tablet edfa0efda2a24efd89c38ef688c917d5 (DEFAULT_TABLE table=default.table_8b5db7d66475449d9772db568c9ded56 [id=7fb6c448c78344719796980b4be1cda9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:17.432767 21777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edfa0efda2a24efd89c38ef688c917d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:17.433169 21583 master_service.cc:960] ReplaceTablet: received request to replace tablet 52d8da0caee84c8ea79f382564575b94 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:17.433683 21776 tablet_service.cc:1511] Processing CreateTablet for tablet 456a6d0a631a4c9aa8d343eb014555ce (DEFAULT_TABLE table=default.table_e0eb6c1d49224d1cb9b4f6d06f0f7a29 [id=420cd09cc1184f6e9fb38b2cbc3f6fce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:17.434612 21776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 456a6d0a631a4c9aa8d343eb014555ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:17.437326 22085 tablet_service.cc:1511] Processing CreateTablet for tablet 456a6d0a631a4c9aa8d343eb014555ce (DEFAULT_TABLE table=default.table_e0eb6c1d49224d1cb9b4f6d06f0f7a29 [id=420cd09cc1184f6e9fb38b2cbc3f6fce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:17.438248 22085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 456a6d0a631a4c9aa8d343eb014555ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:17.440546 21940 tablet_service.cc:1511] Processing CreateTablet for tablet 456a6d0a631a4c9aa8d343eb014555ce (DEFAULT_TABLE table=default.table_e0eb6c1d49224d1cb9b4f6d06f0f7a29 [id=420cd09cc1184f6e9fb38b2cbc3f6fce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:17.441509 21940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 456a6d0a631a4c9aa8d343eb014555ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:17.456421 22416 ts_tablet_manager.cc:1929] T c965fcaf7eb246daa288d6183ce8a6ab P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:17.456774 22416 log.cc:1199] T c965fcaf7eb246daa288d6183ce8a6ab P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/c965fcaf7eb246daa288d6183ce8a6ab
I20260925 13:57:17.457402 22416 ts_tablet_manager.cc:1950] T c965fcaf7eb246daa288d6183ce8a6ab P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:17.459654 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet c965fcaf7eb246daa288d6183ce8a6ab (table default.table_8b5db7d66475449d9772db568c9ded56 [id=7fb6c448c78344719796980b4be1cda9]) successfully deleted
I20260925 13:57:17.480767 22293 tablet_bootstrap.cc:492] T 456a6d0a631a4c9aa8d343eb014555ce P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:17.487264 22293 tablet_bootstrap.cc:654] T 456a6d0a631a4c9aa8d343eb014555ce P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:17.488621 21583 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet 52d8da0caee84c8ea79f382564575b94
I20260925 13:57:17.490528 21775 tablet_service.cc:1558] Processing DeleteTablet for tablet 52d8da0caee84c8ea79f382564575b94 with delete_type TABLET_DATA_DELETED (replaced by tablet 761ede873f5648ccb5216a4c0a8afa2e) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:17.492892 21583 catalog_manager.cc:6934] ReplaceTablet: tablet 52d8da0caee84c8ea79f382564575b94 deleted and replaced by tablet 761ede873f5648ccb5216a4c0a8afa2e
I20260925 13:57:17.494499 22084 tablet_service.cc:1558] Processing DeleteTablet for tablet 52d8da0caee84c8ea79f382564575b94 with delete_type TABLET_DATA_DELETED (replaced by tablet 761ede873f5648ccb5216a4c0a8afa2e) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:17.495858 21939 tablet_service.cc:1558] Processing DeleteTablet for tablet 52d8da0caee84c8ea79f382564575b94 with delete_type TABLET_DATA_DELETED (replaced by tablet 761ede873f5648ccb5216a4c0a8afa2e) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:17.496577 22416 tablet_replica.cc:333] T 52d8da0caee84c8ea79f382564575b94 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:17.497187 22416 raft_consensus.cc:2243] T 52d8da0caee84c8ea79f382564575b94 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:17.497611 22416 raft_consensus.cc:2272] T 52d8da0caee84c8ea79f382564575b94 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:17.499240 22416 ts_tablet_manager.cc:1916] T 52d8da0caee84c8ea79f382564575b94 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:17.507920 22212 tablet_bootstrap.cc:492] T e4fe2c4187b34b5dba98873ab0d5abdb P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:17.514075 22214 ts_tablet_manager.cc:1929] T c965fcaf7eb246daa288d6183ce8a6ab P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:17.514595 22214 log.cc:1199] T c965fcaf7eb246daa288d6183ce8a6ab P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/c965fcaf7eb246daa288d6183ce8a6ab
I20260925 13:57:17.515259 22214 ts_tablet_manager.cc:1950] T c965fcaf7eb246daa288d6183ce8a6ab P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:17.521662 21583 master_service.cc:960] ReplaceTablet: received request to replace tablet dc9cc9c6d85446b786e8ea4396b33a14 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:17.539314 22211 tablet_bootstrap.cc:492] T e4fe2c4187b34b5dba98873ab0d5abdb P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:17.547827 22211 tablet_bootstrap.cc:654] T e4fe2c4187b34b5dba98873ab0d5abdb P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:17.549823 22293 tablet_bootstrap.cc:492] T 456a6d0a631a4c9aa8d343eb014555ce P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:17.550289 22293 ts_tablet_manager.cc:1403] T 456a6d0a631a4c9aa8d343eb014555ce P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.070s user 0.003s sys 0.019s
I20260925 13:57:17.553395 22293 raft_consensus.cc:359] T 456a6d0a631a4c9aa8d343eb014555ce P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:17.554004 22293 raft_consensus.cc:385] T 456a6d0a631a4c9aa8d343eb014555ce P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:17.554250 22293 raft_consensus.cc:740] T 456a6d0a631a4c9aa8d343eb014555ce P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:17.554903 22293 consensus_queue.cc:260] T 456a6d0a631a4c9aa8d343eb014555ce P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:17.533782 22415 ts_tablet_manager.cc:1929] T c965fcaf7eb246daa288d6183ce8a6ab P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:17.556205 22415 log.cc:1199] T c965fcaf7eb246daa288d6183ce8a6ab P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/c965fcaf7eb246daa288d6183ce8a6ab
I20260925 13:57:17.556921 22415 ts_tablet_manager.cc:1950] T c965fcaf7eb246daa288d6183ce8a6ab P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:17.564587 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet c965fcaf7eb246daa288d6183ce8a6ab (table default.table_8b5db7d66475449d9772db568c9ded56 [id=7fb6c448c78344719796980b4be1cda9]) successfully deleted
I20260925 13:57:17.566165 22415 tablet_replica.cc:333] T 52d8da0caee84c8ea79f382564575b94 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:17.566903 22415 raft_consensus.cc:2243] T 52d8da0caee84c8ea79f382564575b94 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:17.574558 22293 ts_tablet_manager.cc:1434] T 456a6d0a631a4c9aa8d343eb014555ce P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.024s user 0.001s sys 0.004s
I20260925 13:57:17.576818 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet c965fcaf7eb246daa288d6183ce8a6ab (table default.table_8b5db7d66475449d9772db568c9ded56 [id=7fb6c448c78344719796980b4be1cda9]) successfully deleted
I20260925 13:57:17.577771 22214 tablet_replica.cc:333] T 52d8da0caee84c8ea79f382564575b94 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:17.578476 22214 raft_consensus.cc:2243] T 52d8da0caee84c8ea79f382564575b94 P ee02f77594864349982492862d668a36 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:17.579407 22214 raft_consensus.cc:2272] T 52d8da0caee84c8ea79f382564575b94 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:17.581248 22214 ts_tablet_manager.cc:1916] T 52d8da0caee84c8ea79f382564575b94 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:17.584474 22211 tablet_bootstrap.cc:492] T e4fe2c4187b34b5dba98873ab0d5abdb P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:17.584887 22211 ts_tablet_manager.cc:1403] T e4fe2c4187b34b5dba98873ab0d5abdb P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.046s user 0.006s sys 0.004s
I20260925 13:57:17.587064 22211 raft_consensus.cc:359] T e4fe2c4187b34b5dba98873ab0d5abdb P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:17.587736 22211 raft_consensus.cc:385] T e4fe2c4187b34b5dba98873ab0d5abdb P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:17.588311 22368 raft_consensus.cc:493] T dc9cc9c6d85446b786e8ea4396b33a14 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:17.588716 22368 raft_consensus.cc:515] T dc9cc9c6d85446b786e8ea4396b33a14 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:17.590591 22368 leader_election.cc:290] T dc9cc9c6d85446b786e8ea4396b33a14 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:17.588037 22211 raft_consensus.cc:740] T e4fe2c4187b34b5dba98873ab0d5abdb P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:17.597118 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc9cc9c6d85446b786e8ea4396b33a14" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2" is_pre_election: true
I20260925 13:57:17.597688 21797 raft_consensus.cc:2468] T dc9cc9c6d85446b786e8ea4396b33a14 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 0.
I20260925 13:57:17.597770 22211 consensus_queue.cc:260] T e4fe2c4187b34b5dba98873ab0d5abdb P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:17.599037 21893 leader_election.cc:304] T dc9cc9c6d85446b786e8ea4396b33a14 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:17.604574 22293 tablet_bootstrap.cc:492] T edfa0efda2a24efd89c38ef688c917d5 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:17.607110 22368 raft_consensus.cc:2804] T dc9cc9c6d85446b786e8ea4396b33a14 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:17.607461 22368 raft_consensus.cc:493] T dc9cc9c6d85446b786e8ea4396b33a14 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:17.607761 22368 raft_consensus.cc:3060] T dc9cc9c6d85446b786e8ea4396b33a14 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:17.614494 22368 raft_consensus.cc:515] T dc9cc9c6d85446b786e8ea4396b33a14 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:17.617708 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc9cc9c6d85446b786e8ea4396b33a14" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2"
I20260925 13:57:17.618244 21797 raft_consensus.cc:3060] T dc9cc9c6d85446b786e8ea4396b33a14 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:17.619532 22368 leader_election.cc:290] T dc9cc9c6d85446b786e8ea4396b33a14 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Requested vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:17.625411 22416 ts_tablet_manager.cc:1929] T 52d8da0caee84c8ea79f382564575b94 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:57:17.625751 22416 log.cc:1199] T 52d8da0caee84c8ea79f382564575b94 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/52d8da0caee84c8ea79f382564575b94
I20260925 13:57:17.626423 22416 ts_tablet_manager.cc:1950] T 52d8da0caee84c8ea79f382564575b94 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:17.628657 21797 raft_consensus.cc:2468] T dc9cc9c6d85446b786e8ea4396b33a14 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 1.
I20260925 13:57:17.629879 21893 leader_election.cc:304] T dc9cc9c6d85446b786e8ea4396b33a14 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:17.631484 21583 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet dc9cc9c6d85446b786e8ea4396b33a14
I20260925 13:57:17.633029 21583 catalog_manager.cc:6934] ReplaceTablet: tablet dc9cc9c6d85446b786e8ea4396b33a14 deleted and replaced by tablet f3b0e015d5dd4a668b6a04b49f628070
I20260925 13:57:17.633215 21777 tablet_service.cc:1558] Processing DeleteTablet for tablet dc9cc9c6d85446b786e8ea4396b33a14 with delete_type TABLET_DATA_DELETED (replaced by tablet f3b0e015d5dd4a668b6a04b49f628070) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:17.634007 21583 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35994:
name: "default.Table_2138b0cdfa7c4afda3a919dd0945a2d1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:57:17.634532 22293 tablet_bootstrap.cc:654] T edfa0efda2a24efd89c38ef688c917d5 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:17.635249 21940 tablet_service.cc:1558] Processing DeleteTablet for tablet dc9cc9c6d85446b786e8ea4396b33a14 with delete_type TABLET_DATA_DELETED (replaced by tablet f3b0e015d5dd4a668b6a04b49f628070) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:17.637004 22211 ts_tablet_manager.cc:1434] T e4fe2c4187b34b5dba98873ab0d5abdb P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.052s user 0.005s sys 0.000s
I20260925 13:57:17.594347 22106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc9cc9c6d85446b786e8ea4396b33a14" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:17.637773 22106 raft_consensus.cc:2468] T dc9cc9c6d85446b786e8ea4396b33a14 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 0.
I20260925 13:57:17.618158 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc9cc9c6d85446b786e8ea4396b33a14" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:17.638815 22107 raft_consensus.cc:3060] T dc9cc9c6d85446b786e8ea4396b33a14 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:17.640265 22211 tablet_bootstrap.cc:492] T edfa0efda2a24efd89c38ef688c917d5 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:17.644783 22415 raft_consensus.cc:2272] T 52d8da0caee84c8ea79f382564575b94 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:17.645758 21777 tablet_service.cc:1511] Processing CreateTablet for tablet 761ede873f5648ccb5216a4c0a8afa2e (DEFAULT_TABLE table=default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:17.647123 21777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 761ede873f5648ccb5216a4c0a8afa2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:17.649789 22478 raft_consensus.cc:2804] T dc9cc9c6d85446b786e8ea4396b33a14 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:17.650199 22478 raft_consensus.cc:697] T dc9cc9c6d85446b786e8ea4396b33a14 P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: Becoming Leader. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Running, Role: LEADER
I20260925 13:57:17.654417 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet 52d8da0caee84c8ea79f382564575b94 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]) successfully deleted
I20260925 13:57:17.656424 22415 ts_tablet_manager.cc:1916] T 52d8da0caee84c8ea79f382564575b94 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:17.657243 22378 consensus_queue.cc:1048] T 3a8c53e379a9407c920d91b05ebd2afc P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:17.666229 22083 tablet_service.cc:1511] Processing CreateTablet for tablet 761ede873f5648ccb5216a4c0a8afa2e (DEFAULT_TABLE table=default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:17.667263 22083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 761ede873f5648ccb5216a4c0a8afa2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:17.673537 22107 raft_consensus.cc:2468] T dc9cc9c6d85446b786e8ea4396b33a14 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 1.
I20260925 13:57:17.634644 22085 tablet_service.cc:1558] Processing DeleteTablet for tablet dc9cc9c6d85446b786e8ea4396b33a14 with delete_type TABLET_DATA_DELETED (replaced by tablet f3b0e015d5dd4a668b6a04b49f628070) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:17.564152 22212 tablet_bootstrap.cc:654] T e4fe2c4187b34b5dba98873ab0d5abdb P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:17.676316 22416 tablet_replica.cc:333] T dc9cc9c6d85446b786e8ea4396b33a14 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:17.679904 21940 tablet_service.cc:1511] Processing CreateTablet for tablet 761ede873f5648ccb5216a4c0a8afa2e (DEFAULT_TABLE table=default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:17.683926 21940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 761ede873f5648ccb5216a4c0a8afa2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:17.685773 22293 tablet_bootstrap.cc:492] T edfa0efda2a24efd89c38ef688c917d5 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:17.686190 22293 ts_tablet_manager.cc:1403] T edfa0efda2a24efd89c38ef688c917d5 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.095s user 0.006s sys 0.004s
I20260925 13:57:17.650933 22478 consensus_queue.cc:237] T dc9cc9c6d85446b786e8ea4396b33a14 P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:17.688642 22293 raft_consensus.cc:359] T edfa0efda2a24efd89c38ef688c917d5 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:17.689249 22293 raft_consensus.cc:385] T edfa0efda2a24efd89c38ef688c917d5 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:17.689505 22293 raft_consensus.cc:740] T edfa0efda2a24efd89c38ef688c917d5 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:17.704756 22212 tablet_bootstrap.cc:492] T e4fe2c4187b34b5dba98873ab0d5abdb P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:17.705340 22212 ts_tablet_manager.cc:1403] T e4fe2c4187b34b5dba98873ab0d5abdb P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.198s user 0.007s sys 0.006s
I20260925 13:57:17.710858 21776 tablet_service.cc:1511] Processing CreateTablet for tablet f3b0e015d5dd4a668b6a04b49f628070 (DEFAULT_TABLE table=default.table_bd714c618f7a4f16a69a69d5932b2014 [id=dfd05c11872140cfa86b2628c33ef6d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:17.711793 21776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3b0e015d5dd4a668b6a04b49f628070. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:17.716162 22416 raft_consensus.cc:2243] T dc9cc9c6d85446b786e8ea4396b33a14 P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:17.719599 21582 master_service.cc:960] ReplaceTablet: received request to replace tablet e4fe2c4187b34b5dba98873ab0d5abdb from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:17.720922 22212 raft_consensus.cc:359] T e4fe2c4187b34b5dba98873ab0d5abdb P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:17.721629 22212 raft_consensus.cc:385] T e4fe2c4187b34b5dba98873ab0d5abdb P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:17.721905 22212 raft_consensus.cc:740] T e4fe2c4187b34b5dba98873ab0d5abdb P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:17.722604 22212 consensus_queue.cc:260] T e4fe2c4187b34b5dba98873ab0d5abdb P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:17.723174 21937 tablet_service.cc:1511] Processing CreateTablet for tablet f3b0e015d5dd4a668b6a04b49f628070 (DEFAULT_TABLE table=default.table_bd714c618f7a4f16a69a69d5932b2014 [id=dfd05c11872140cfa86b2628c33ef6d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:17.724128 21937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3b0e015d5dd4a668b6a04b49f628070. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:17.727329 22416 pending_rounds.cc:70] T dc9cc9c6d85446b786e8ea4396b33a14 P 145e520e0a5a445d943a53cf8c27efee: Trying to abort 1 pending ops.
I20260925 13:57:17.727605 22416 pending_rounds.cc:77] T dc9cc9c6d85446b786e8ea4396b33a14 P 145e520e0a5a445d943a53cf8c27efee: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251635983872000 op_type: NO_OP noop_request { }
I20260925 13:57:17.728351 22416 raft_consensus.cc:2889] T dc9cc9c6d85446b786e8ea4396b33a14 P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:57:17.730129 22416 raft_consensus.cc:2272] T dc9cc9c6d85446b786e8ea4396b33a14 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:17.733963 21582 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet e4fe2c4187b34b5dba98873ab0d5abdb
I20260925 13:57:17.735292 21582 catalog_manager.cc:6934] ReplaceTablet: tablet e4fe2c4187b34b5dba98873ab0d5abdb deleted and replaced by tablet e07f41572a914b8a86c063c65e73e072
I20260925 13:57:17.737640 21938 tablet_service.cc:1558] Processing DeleteTablet for tablet e4fe2c4187b34b5dba98873ab0d5abdb with delete_type TABLET_DATA_DELETED (replaced by tablet e07f41572a914b8a86c063c65e73e072) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:17.737708 22085 tablet_service.cc:1558] Processing DeleteTablet for tablet e4fe2c4187b34b5dba98873ab0d5abdb with delete_type TABLET_DATA_DELETED (replaced by tablet e07f41572a914b8a86c063c65e73e072) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:17.739295 22416 ts_tablet_manager.cc:1916] T dc9cc9c6d85446b786e8ea4396b33a14 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:17.740150 21775 tablet_service.cc:1558] Processing DeleteTablet for tablet e4fe2c4187b34b5dba98873ab0d5abdb with delete_type TABLET_DATA_DELETED (replaced by tablet e07f41572a914b8a86c063c65e73e072) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:17.694958 22293 consensus_queue.cc:260] T edfa0efda2a24efd89c38ef688c917d5 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:17.750671 22085 tablet_service.cc:1511] Processing CreateTablet for tablet f3b0e015d5dd4a668b6a04b49f628070 (DEFAULT_TABLE table=default.table_bd714c618f7a4f16a69a69d5932b2014 [id=dfd05c11872140cfa86b2628c33ef6d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:17.753371 22293 ts_tablet_manager.cc:1434] T edfa0efda2a24efd89c38ef688c917d5 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.067s user 0.007s sys 0.000s
I20260925 13:57:17.751819 22085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3b0e015d5dd4a668b6a04b49f628070. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:17.749688 22212 ts_tablet_manager.cc:1434] T e4fe2c4187b34b5dba98873ab0d5abdb P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.044s user 0.001s sys 0.005s
I20260925 13:57:17.760423 21584 catalog_manager.cc:2213] created HMS catalog entry for table [id=722129e8c51a45f8ae31843d0d3f0a47]
I20260925 13:57:17.768039 22212 tablet_bootstrap.cc:492] T edfa0efda2a24efd89c38ef688c917d5 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:17.770531 22293 tablet_bootstrap.cc:492] T 761ede873f5648ccb5216a4c0a8afa2e P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:17.739486 22211 tablet_bootstrap.cc:654] T edfa0efda2a24efd89c38ef688c917d5 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:17.749985 22478 raft_consensus.cc:493] T e4fe2c4187b34b5dba98873ab0d5abdb P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:17.773419 22478 raft_consensus.cc:515] T e4fe2c4187b34b5dba98873ab0d5abdb P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:17.775298 22478 leader_election.cc:290] T e4fe2c4187b34b5dba98873ab0d5abdb P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:17.777192 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4fe2c4187b34b5dba98873ab0d5abdb" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2" is_pre_election: true
I20260925 13:57:17.777807 22293 tablet_bootstrap.cc:654] T 761ede873f5648ccb5216a4c0a8afa2e P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:17.777781 21797 raft_consensus.cc:2468] T e4fe2c4187b34b5dba98873ab0d5abdb P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 0.
I20260925 13:57:17.779067 22106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4fe2c4187b34b5dba98873ab0d5abdb" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:17.779109 21893 leader_election.cc:304] T e4fe2c4187b34b5dba98873ab0d5abdb P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:17.779872 22478 raft_consensus.cc:2804] T e4fe2c4187b34b5dba98873ab0d5abdb P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:17.780189 22478 raft_consensus.cc:493] T e4fe2c4187b34b5dba98873ab0d5abdb P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:17.780486 22478 raft_consensus.cc:3060] T e4fe2c4187b34b5dba98873ab0d5abdb P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:17.779672 22106 raft_consensus.cc:2468] T e4fe2c4187b34b5dba98873ab0d5abdb P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 0.
I20260925 13:57:17.785894 21584 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35994 (request call id 65) took 1551 ms. Trace:
I20260925 13:57:17.788686 22478 raft_consensus.cc:515] T e4fe2c4187b34b5dba98873ab0d5abdb P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:17.786212 21584 rpcz_store.cc:276] 0925 13:57:16.234225 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:16.234351 (+ 126us) service_pool.cc:224] Handling call
0925 13:57:16.324695 (+ 90344us) catalog_manager.cc:2140] Acquired catalog manager lock
0925 13:57:16.326507 (+ 1812us) catalog_manager.cc:2186] Created new table and tablet info
0925 13:57:17.760269 (+1433762us) catalog_manager.cc:2212] Created new table in HMS catalog
0925 13:57:17.785109 (+ 24840us) catalog_manager.cc:2246] Wrote table and tablets to system table
0925 13:57:17.785438 (+ 329us) catalog_manager.cc:2280] Inserted table and tablets into CatalogManager maps
0925 13:57:17.785837 (+ 399us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:57:17.762105 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:57:17.762476 (+ 371us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:57:17.762499 (+ 23us) write_op.cc:435] Acquired schema lock
0925 13:57:17.762512 (+ 13us) tablet.cc:662] Decoding operations
0925 13:57:17.762751 (+ 239us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:57:17.762787 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0925 13:57:17.762807 (+ 20us) tablet.cc:685] Acquiring locks for 2 operations
0925 13:57:17.762979 (+ 172us) tablet.cc:701] Row locks acquired
0925 13:57:17.762991 (+ 12us) write_op.cc:260] PREPARE: finished
0925 13:57:17.763222 (+ 231us) write_op.cc:270] Start()
0925 13:57:17.763303 (+ 81us) write_op.cc:276] Timestamp: P: 1790344637763161 usec, L: 0
0925 13:57:17.763317 (+ 14us) op_driver.cc:348] REPLICATION: starting
0925 13:57:17.766987 (+ 3670us) log.cc:844] Serialized 432 byte log entry
0925 13:57:17.783105 (+ 16118us) op_driver.cc:464] REPLICATION: finished
0925 13:57:17.783554 (+ 449us) write_op.cc:301] APPLY: starting
0925 13:57:17.783603 (+ 49us) tablet.cc:1394] starting BulkCheckPresence
0925 13:57:17.783699 (+ 96us) tablet.cc:1397] finished BulkCheckPresence
0925 13:57:17.783781 (+ 82us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:57:17.784121 (+ 340us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:57:17.784239 (+ 118us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 13:57:17.784272 (+ 33us) write_op.cc:312] APPLY: finished
0925 13:57:17.784426 (+ 154us) log.cc:844] Serialized 28 byte log entry
0925 13:57:17.784560 (+ 134us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:17.784620 (+ 60us) write_op.cc:454] Released schema lock
0925 13:57:17.784647 (+ 27us) write_op.cc:341] FINISH: Updating metrics
0925 13:57:17.784830 (+ 183us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:17.784849 (+ 19us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":1001839,"Hive Metastore.run_cpu_time_us":1002,"Hive Metastore.run_wall_time_us":431233,"child_traces":[["op",{"apply.queue_time_us":305,"apply.run_cpu_time_us":1255,"apply.run_wall_time_us":1255,"num_ops":2,"prepare.queue_time_us":111,"prepare.run_cpu_time_us":1952,"prepare.run_wall_time_us":5191,"raft.queue_time_us":311,"raft.run_cpu_time_us":569,"raft.run_wall_time_us":568,"replication_time_us":19694,"spinlock_wait_cycles":17152}]]}
I20260925 13:57:17.791280 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4fe2c4187b34b5dba98873ab0d5abdb" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2"
I20260925 13:57:17.791869 21797 raft_consensus.cc:3060] T e4fe2c4187b34b5dba98873ab0d5abdb P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:17.794721 22106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4fe2c4187b34b5dba98873ab0d5abdb" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:17.795256 22106 raft_consensus.cc:3060] T e4fe2c4187b34b5dba98873ab0d5abdb P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:17.798424 21797 raft_consensus.cc:2468] T e4fe2c4187b34b5dba98873ab0d5abdb P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 1.
I20260925 13:57:17.799439 22478 leader_election.cc:290] T e4fe2c4187b34b5dba98873ab0d5abdb P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Requested vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:17.799642 21893 leader_election.cc:304] T e4fe2c4187b34b5dba98873ab0d5abdb P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:17.806520 22212 tablet_bootstrap.cc:654] T edfa0efda2a24efd89c38ef688c917d5 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:17.816224 22415 ts_tablet_manager.cc:1929] T 52d8da0caee84c8ea79f382564575b94 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:57:17.816668 22415 log.cc:1199] T 52d8da0caee84c8ea79f382564575b94 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/52d8da0caee84c8ea79f382564575b94
I20260925 13:57:17.817492 22415 ts_tablet_manager.cc:1950] T 52d8da0caee84c8ea79f382564575b94 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:17.818167 22106 raft_consensus.cc:2468] T e4fe2c4187b34b5dba98873ab0d5abdb P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 1.
I20260925 13:57:17.830555 22211 tablet_bootstrap.cc:492] T edfa0efda2a24efd89c38ef688c917d5 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:17.830963 22211 ts_tablet_manager.cc:1403] T edfa0efda2a24efd89c38ef688c917d5 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.191s user 0.008s sys 0.000s
I20260925 13:57:17.833153 22478 raft_consensus.cc:2804] T e4fe2c4187b34b5dba98873ab0d5abdb P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:17.833848 22478 raft_consensus.cc:697] T e4fe2c4187b34b5dba98873ab0d5abdb P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: Becoming Leader. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Running, Role: LEADER
I20260925 13:57:17.834609 22478 consensus_queue.cc:237] T e4fe2c4187b34b5dba98873ab0d5abdb P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:17.842895 22214 ts_tablet_manager.cc:1929] T 52d8da0caee84c8ea79f382564575b94 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:57:17.846001 21581 master_service.cc:960] ReplaceTablet: received request to replace tablet d71049b217a74662b7a9974e776d1448 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:17.849802 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet 52d8da0caee84c8ea79f382564575b94 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]) successfully deleted
I20260925 13:57:17.851468 22212 tablet_bootstrap.cc:492] T edfa0efda2a24efd89c38ef688c917d5 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:17.851871 22212 ts_tablet_manager.cc:1403] T edfa0efda2a24efd89c38ef688c917d5 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.084s user 0.006s sys 0.003s
I20260925 13:57:17.844352 22211 raft_consensus.cc:359] T edfa0efda2a24efd89c38ef688c917d5 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:17.855757 22415 tablet_replica.cc:333] T dc9cc9c6d85446b786e8ea4396b33a14 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:17.855822 22211 raft_consensus.cc:385] T edfa0efda2a24efd89c38ef688c917d5 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:17.856098 22211 raft_consensus.cc:740] T edfa0efda2a24efd89c38ef688c917d5 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:17.858268 22415 raft_consensus.cc:2243] T dc9cc9c6d85446b786e8ea4396b33a14 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:17.858687 22415 raft_consensus.cc:2272] T dc9cc9c6d85446b786e8ea4396b33a14 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:17.865587 21581 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet d71049b217a74662b7a9974e776d1448
I20260925 13:57:17.866868 21581 catalog_manager.cc:6934] ReplaceTablet: tablet d71049b217a74662b7a9974e776d1448 deleted and replaced by tablet 7efa9e6cb797487bb11bf47ad3c320ba
I20260925 13:57:17.869051 22211 consensus_queue.cc:260] T edfa0efda2a24efd89c38ef688c917d5 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:17.870085 21937 tablet_service.cc:1558] Processing DeleteTablet for tablet d71049b217a74662b7a9974e776d1448 with delete_type TABLET_DATA_DELETED (replaced by tablet 7efa9e6cb797487bb11bf47ad3c320ba) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:17.870745 22085 tablet_service.cc:1558] Processing DeleteTablet for tablet d71049b217a74662b7a9974e776d1448 with delete_type TABLET_DATA_DELETED (replaced by tablet 7efa9e6cb797487bb11bf47ad3c320ba) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:17.872447 21776 tablet_service.cc:1558] Processing DeleteTablet for tablet d71049b217a74662b7a9974e776d1448 with delete_type TABLET_DATA_DELETED (replaced by tablet 7efa9e6cb797487bb11bf47ad3c320ba) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:17.873714 22211 ts_tablet_manager.cc:1434] T edfa0efda2a24efd89c38ef688c917d5 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.042s user 0.006s sys 0.001s
I20260925 13:57:17.874096 22415 ts_tablet_manager.cc:1916] T dc9cc9c6d85446b786e8ea4396b33a14 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:17.876678 22211 tablet_bootstrap.cc:492] T 456a6d0a631a4c9aa8d343eb014555ce P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:17.877878 22293 tablet_bootstrap.cc:492] T 761ede873f5648ccb5216a4c0a8afa2e P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:17.878274 22293 ts_tablet_manager.cc:1403] T 761ede873f5648ccb5216a4c0a8afa2e P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.108s user 0.007s sys 0.002s
I20260925 13:57:17.894482 22212 raft_consensus.cc:359] T edfa0efda2a24efd89c38ef688c917d5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:17.895073 22212 raft_consensus.cc:385] T edfa0efda2a24efd89c38ef688c917d5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:17.880589 22293 raft_consensus.cc:359] T 761ede873f5648ccb5216a4c0a8afa2e P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:17.900887 22293 raft_consensus.cc:385] T 761ede873f5648ccb5216a4c0a8afa2e P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:17.901170 22293 raft_consensus.cc:740] T 761ede873f5648ccb5216a4c0a8afa2e P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:17.903746 22293 consensus_queue.cc:260] T 761ede873f5648ccb5216a4c0a8afa2e P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:17.905884 22415 ts_tablet_manager.cc:1929] T dc9cc9c6d85446b786e8ea4396b33a14 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:17.906353 22415 log.cc:1199] T dc9cc9c6d85446b786e8ea4396b33a14 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/dc9cc9c6d85446b786e8ea4396b33a14
I20260925 13:57:17.907006 22415 ts_tablet_manager.cc:1950] T dc9cc9c6d85446b786e8ea4396b33a14 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:17.908850 22416 ts_tablet_manager.cc:1929] T dc9cc9c6d85446b786e8ea4396b33a14 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:17.909209 22416 log.cc:1199] T dc9cc9c6d85446b786e8ea4396b33a14 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/dc9cc9c6d85446b786e8ea4396b33a14
I20260925 13:57:17.909849 22416 ts_tablet_manager.cc:1950] T dc9cc9c6d85446b786e8ea4396b33a14 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:17.843222 22214 log.cc:1199] T 52d8da0caee84c8ea79f382564575b94 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/52d8da0caee84c8ea79f382564575b94
I20260925 13:57:17.911957 22214 ts_tablet_manager.cc:1950] T 52d8da0caee84c8ea79f382564575b94 P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:17.895318 22212 raft_consensus.cc:740] T edfa0efda2a24efd89c38ef688c917d5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:17.922375 22415 tablet_replica.cc:333] T e4fe2c4187b34b5dba98873ab0d5abdb P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:17.923095 22415 raft_consensus.cc:2243] T e4fe2c4187b34b5dba98873ab0d5abdb P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:17.923421 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet 52d8da0caee84c8ea79f382564575b94 (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]) successfully deleted
I20260925 13:57:17.923569 22415 raft_consensus.cc:2272] T e4fe2c4187b34b5dba98873ab0d5abdb P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:17.923863 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet dc9cc9c6d85446b786e8ea4396b33a14 (table default.table_bd714c618f7a4f16a69a69d5932b2014 [id=dfd05c11872140cfa86b2628c33ef6d8]) successfully deleted
I20260925 13:57:17.924312 22214 tablet_replica.cc:333] T dc9cc9c6d85446b786e8ea4396b33a14 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:17.925009 22214 raft_consensus.cc:2243] T dc9cc9c6d85446b786e8ea4396b33a14 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:17.925395 22214 raft_consensus.cc:2272] T dc9cc9c6d85446b786e8ea4396b33a14 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:17.926669 22293 ts_tablet_manager.cc:1434] T 761ede873f5648ccb5216a4c0a8afa2e P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.048s user 0.006s sys 0.001s
I20260925 13:57:17.926913 22214 ts_tablet_manager.cc:1916] T dc9cc9c6d85446b786e8ea4396b33a14 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:17.927485 22415 ts_tablet_manager.cc:1916] T e4fe2c4187b34b5dba98873ab0d5abdb P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:17.914326 22212 consensus_queue.cc:260] T edfa0efda2a24efd89c38ef688c917d5 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:17.939206 22293 tablet_bootstrap.cc:492] T f3b0e015d5dd4a668b6a04b49f628070 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:17.939642 22212 ts_tablet_manager.cc:1434] T edfa0efda2a24efd89c38ef688c917d5 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.088s user 0.006s sys 0.000s
I20260925 13:57:17.943007 22211 tablet_bootstrap.cc:654] T 456a6d0a631a4c9aa8d343eb014555ce P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:17.948179 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet dc9cc9c6d85446b786e8ea4396b33a14 (table default.table_bd714c618f7a4f16a69a69d5932b2014 [id=dfd05c11872140cfa86b2628c33ef6d8]) successfully deleted
I20260925 13:57:17.948992 22416 tablet_replica.cc:333] T e4fe2c4187b34b5dba98873ab0d5abdb P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:17.949631 22416 raft_consensus.cc:2243] T e4fe2c4187b34b5dba98873ab0d5abdb P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:17.950212 22416 pending_rounds.cc:70] T e4fe2c4187b34b5dba98873ab0d5abdb P 145e520e0a5a445d943a53cf8c27efee: Trying to abort 1 pending ops.
I20260925 13:57:17.950412 22416 pending_rounds.cc:77] T e4fe2c4187b34b5dba98873ab0d5abdb P 145e520e0a5a445d943a53cf8c27efee: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251636582838272 op_type: NO_OP noop_request { }
I20260925 13:57:17.950690 22416 raft_consensus.cc:2889] T e4fe2c4187b34b5dba98873ab0d5abdb P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:57:17.950943 22416 raft_consensus.cc:2272] T e4fe2c4187b34b5dba98873ab0d5abdb P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:17.952414 21581 catalog_manager.cc:5938] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881) does not have the latest schema for tablet 761ede873f5648ccb5216a4c0a8afa2e (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 1 got 0
I20260925 13:57:17.953303 22212 tablet_bootstrap.cc:492] T 456a6d0a631a4c9aa8d343eb014555ce P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:17.954082 22478 raft_consensus.cc:493] T edfa0efda2a24efd89c38ef688c917d5 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:17.955307 22478 raft_consensus.cc:515] T edfa0efda2a24efd89c38ef688c917d5 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:17.957175 21581 catalog_manager.cc:6038] Failed to send AlterTable RPC for tablet 761ede873f5648ccb5216a4c0a8afa2e (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 761ede873f5648ccb5216a4c0a8afa2e consensus state has no leader
I20260925 13:57:17.959107 21581 master_service.cc:960] ReplaceTablet: received request to replace tablet 456a6d0a631a4c9aa8d343eb014555ce from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:17.959301 22478 leader_election.cc:290] T edfa0efda2a24efd89c38ef688c917d5 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:17.964350 22106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfa0efda2a24efd89c38ef688c917d5" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:17.964926 22106 raft_consensus.cc:2468] T edfa0efda2a24efd89c38ef688c917d5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 0.
I20260925 13:57:17.965232 21940 tablet_service.cc:1511] Processing CreateTablet for tablet f68ccd8a7f4a4e518c2584e579de0c87 (DEFAULT_TABLE table=default.table_b54e9b49662b4a8e892615fec1080a87 [id=722129e8c51a45f8ae31843d0d3f0a47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:17.966284 21940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f68ccd8a7f4a4e518c2584e579de0c87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:17.967653 21896 leader_election.cc:304] T edfa0efda2a24efd89c38ef688c917d5 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:17.970486 22416 ts_tablet_manager.cc:1916] T e4fe2c4187b34b5dba98873ab0d5abdb P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:17.975001 21582 catalog_manager.cc:5938] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881) does not have the latest schema for tablet 761ede873f5648ccb5216a4c0a8afa2e (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 1 got 0
I20260925 13:57:17.977614 22478 raft_consensus.cc:2804] T edfa0efda2a24efd89c38ef688c917d5 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:17.977948 22478 raft_consensus.cc:493] T edfa0efda2a24efd89c38ef688c917d5 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:17.978277 22478 raft_consensus.cc:3060] T edfa0efda2a24efd89c38ef688c917d5 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:17.958096 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfa0efda2a24efd89c38ef688c917d5" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2" is_pre_election: true
I20260925 13:57:17.973836 21776 tablet_service.cc:1511] Processing CreateTablet for tablet f68ccd8a7f4a4e518c2584e579de0c87 (DEFAULT_TABLE table=default.table_b54e9b49662b4a8e892615fec1080a87 [id=722129e8c51a45f8ae31843d0d3f0a47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:17.979730 21797 raft_consensus.cc:2468] T edfa0efda2a24efd89c38ef688c917d5 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 0.
I20260925 13:57:17.980077 21776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f68ccd8a7f4a4e518c2584e579de0c87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:17.986091 22085 tablet_service.cc:1511] Processing CreateTablet for tablet f68ccd8a7f4a4e518c2584e579de0c87 (DEFAULT_TABLE table=default.table_b54e9b49662b4a8e892615fec1080a87 [id=722129e8c51a45f8ae31843d0d3f0a47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:17.987073 22085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f68ccd8a7f4a4e518c2584e579de0c87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:17.987403 21777 tablet_service.cc:1511] Processing CreateTablet for tablet e07f41572a914b8a86c063c65e73e072 (DEFAULT_TABLE table=default.table_61d48f9061c741fc8aa556e3edd984fb [id=7377fa3ea9664b47a930d9c5f88c60fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:17.988308 21777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e07f41572a914b8a86c063c65e73e072. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:17.993974 21937 tablet_service.cc:1511] Processing CreateTablet for tablet e07f41572a914b8a86c063c65e73e072 (DEFAULT_TABLE table=default.table_61d48f9061c741fc8aa556e3edd984fb [id=7377fa3ea9664b47a930d9c5f88c60fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:17.995021 21937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e07f41572a914b8a86c063c65e73e072. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:17.995193 22416 ts_tablet_manager.cc:1929] T e4fe2c4187b34b5dba98873ab0d5abdb P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:17.995530 22416 log.cc:1199] T e4fe2c4187b34b5dba98873ab0d5abdb P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/e4fe2c4187b34b5dba98873ab0d5abdb
I20260925 13:57:17.996141 22416 ts_tablet_manager.cc:1950] T e4fe2c4187b34b5dba98873ab0d5abdb P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:17.996244 22083 tablet_service.cc:1511] Processing CreateTablet for tablet e07f41572a914b8a86c063c65e73e072 (DEFAULT_TABLE table=default.table_61d48f9061c741fc8aa556e3edd984fb [id=7377fa3ea9664b47a930d9c5f88c60fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:17.997107 22083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e07f41572a914b8a86c063c65e73e072. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:17.998422 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet e4fe2c4187b34b5dba98873ab0d5abdb (table default.table_61d48f9061c741fc8aa556e3edd984fb [id=7377fa3ea9664b47a930d9c5f88c60fa]) successfully deleted
I20260925 13:57:18.004801 22416 tablet_replica.cc:333] T d71049b217a74662b7a9974e776d1448 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
W20260925 13:57:18.004902 21573 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 761ede873f5648ccb5216a4c0a8afa2e (table default.table_51d65314582841419e34561a4a8d6bbb [id=74ee67384ecf4220b7d08fdf0637aefb], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 761ede873f5648ccb5216a4c0a8afa2e consensus state has no leader
I20260925 13:57:18.005573 22416 raft_consensus.cc:2243] T d71049b217a74662b7a9974e776d1448 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:18.004454 21584 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35994:
name: "default.Table_dc864d7428e5405da360d2f379eef37b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:57:18.006088 22416 raft_consensus.cc:2272] T d71049b217a74662b7a9974e776d1448 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:18.007668 22293 tablet_bootstrap.cc:654] T f3b0e015d5dd4a668b6a04b49f628070 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:18.007779 22416 ts_tablet_manager.cc:1916] T d71049b217a74662b7a9974e776d1448 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:18.014914 22212 tablet_bootstrap.cc:654] T 456a6d0a631a4c9aa8d343eb014555ce P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:18.029728 21581 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet 456a6d0a631a4c9aa8d343eb014555ce
I20260925 13:57:18.031605 21775 tablet_service.cc:1558] Processing DeleteTablet for tablet 456a6d0a631a4c9aa8d343eb014555ce with delete_type TABLET_DATA_DELETED (replaced by tablet 72fc427f45934cba91e52ad84275c461) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:18.032696 21938 tablet_service.cc:1558] Processing DeleteTablet for tablet 456a6d0a631a4c9aa8d343eb014555ce with delete_type TABLET_DATA_DELETED (replaced by tablet 72fc427f45934cba91e52ad84275c461) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:18.033373 22416 ts_tablet_manager.cc:1929] T d71049b217a74662b7a9974e776d1448 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:57:18.033697 22416 log.cc:1199] T d71049b217a74662b7a9974e776d1448 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/d71049b217a74662b7a9974e776d1448
I20260925 13:57:18.034348 22416 ts_tablet_manager.cc:1950] T d71049b217a74662b7a9974e776d1448 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:18.034843 22087 tablet_service.cc:1558] Processing DeleteTablet for tablet 456a6d0a631a4c9aa8d343eb014555ce with delete_type TABLET_DATA_DELETED (replaced by tablet 72fc427f45934cba91e52ad84275c461) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:18.035485 21581 catalog_manager.cc:6934] ReplaceTablet: tablet 456a6d0a631a4c9aa8d343eb014555ce deleted and replaced by tablet 72fc427f45934cba91e52ad84275c461
I20260925 13:57:18.048141 22478 raft_consensus.cc:515] T edfa0efda2a24efd89c38ef688c917d5 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.050175 22478 leader_election.cc:290] T edfa0efda2a24efd89c38ef688c917d5 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Requested vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:18.051299 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfa0efda2a24efd89c38ef688c917d5" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2"
I20260925 13:57:18.051894 21797 raft_consensus.cc:3060] T edfa0efda2a24efd89c38ef688c917d5 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:18.051851 22106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfa0efda2a24efd89c38ef688c917d5" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:18.052385 22106 raft_consensus.cc:3060] T edfa0efda2a24efd89c38ef688c917d5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:18.043090 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet d71049b217a74662b7a9974e776d1448 (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]) successfully deleted
W20260925 13:57:18.057837 21575 catalog_manager.cc:5043] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): delete failed for tablet 456a6d0a631a4c9aa8d343eb014555ce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 456a6d0a631a4c9aa8d343eb014555ce already in progress: creating tablet
I20260925 13:57:18.058760 21797 raft_consensus.cc:2468] T edfa0efda2a24efd89c38ef688c917d5 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 1.
I20260925 13:57:18.061003 21893 leader_election.cc:304] T edfa0efda2a24efd89c38ef688c917d5 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:18.064523 22106 raft_consensus.cc:2468] T edfa0efda2a24efd89c38ef688c917d5 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 1.
I20260925 13:57:18.068663 22211 tablet_bootstrap.cc:492] T 456a6d0a631a4c9aa8d343eb014555ce P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:18.069134 22211 ts_tablet_manager.cc:1403] T 456a6d0a631a4c9aa8d343eb014555ce P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.193s user 0.003s sys 0.005s
I20260925 13:57:18.072420 22415 ts_tablet_manager.cc:1929] T e4fe2c4187b34b5dba98873ab0d5abdb P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:18.073453 22415 log.cc:1199] T e4fe2c4187b34b5dba98873ab0d5abdb P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/e4fe2c4187b34b5dba98873ab0d5abdb
I20260925 13:57:18.074296 22415 ts_tablet_manager.cc:1950] T e4fe2c4187b34b5dba98873ab0d5abdb P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:18.075894 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet e4fe2c4187b34b5dba98873ab0d5abdb (table default.table_61d48f9061c741fc8aa556e3edd984fb [id=7377fa3ea9664b47a930d9c5f88c60fa]) successfully deleted
I20260925 13:57:18.083453 22211 raft_consensus.cc:359] T 456a6d0a631a4c9aa8d343eb014555ce P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.084100 22211 raft_consensus.cc:385] T 456a6d0a631a4c9aa8d343eb014555ce P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:18.084336 22211 raft_consensus.cc:740] T 456a6d0a631a4c9aa8d343eb014555ce P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:18.084924 22211 consensus_queue.cc:260] T 456a6d0a631a4c9aa8d343eb014555ce P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.085796 22368 raft_consensus.cc:2804] T edfa0efda2a24efd89c38ef688c917d5 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:18.081422 21581 master_service.cc:960] ReplaceTablet: received request to replace tablet d267def3f0864435a0d2c40fd7052eab from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:18.097819 22415 tablet_replica.cc:333] T d71049b217a74662b7a9974e776d1448 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:18.098524 22415 raft_consensus.cc:2243] T d71049b217a74662b7a9974e776d1448 P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:18.100678 22212 tablet_bootstrap.cc:492] T 456a6d0a631a4c9aa8d343eb014555ce P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:18.103793 22212 ts_tablet_manager.cc:1403] T 456a6d0a631a4c9aa8d343eb014555ce P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.151s user 0.010s sys 0.000s
I20260925 13:57:18.104712 22462 raft_consensus.cc:493] T 456a6d0a631a4c9aa8d343eb014555ce P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:18.105129 22462 raft_consensus.cc:515] T 456a6d0a631a4c9aa8d343eb014555ce P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.106968 22462 leader_election.cc:290] T 456a6d0a631a4c9aa8d343eb014555ce P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:18.108428 22211 ts_tablet_manager.cc:1434] T 456a6d0a631a4c9aa8d343eb014555ce P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.039s user 0.005s sys 0.001s
I20260925 13:57:18.109367 22368 raft_consensus.cc:697] T edfa0efda2a24efd89c38ef688c917d5 P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: Becoming Leader. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Running, Role: LEADER
I20260925 13:57:18.109696 21958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "456a6d0a631a4c9aa8d343eb014555ce" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:18.110280 21958 raft_consensus.cc:2468] T 456a6d0a631a4c9aa8d343eb014555ce P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:18.110285 22106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "456a6d0a631a4c9aa8d343eb014555ce" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:18.111462 22415 raft_consensus.cc:2272] T d71049b217a74662b7a9974e776d1448 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 13:57:18.111538 21732 leader_election.cc:343] T 456a6d0a631a4c9aa8d343eb014555ce P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:57:18.112178 21732 leader_election.cc:304] T 456a6d0a631a4c9aa8d343eb014555ce P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters: ee02f77594864349982492862d668a36
I20260925 13:57:18.112694 22293 tablet_bootstrap.cc:492] T f3b0e015d5dd4a668b6a04b49f628070 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:18.113061 22462 raft_consensus.cc:2804] T 456a6d0a631a4c9aa8d343eb014555ce P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:18.113226 22415 ts_tablet_manager.cc:1916] T d71049b217a74662b7a9974e776d1448 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:18.113380 22462 raft_consensus.cc:493] T 456a6d0a631a4c9aa8d343eb014555ce P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:18.113672 22462 raft_consensus.cc:3060] T 456a6d0a631a4c9aa8d343eb014555ce P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:18.114936 21777 tablet_service.cc:1511] Processing CreateTablet for tablet 7efa9e6cb797487bb11bf47ad3c320ba (DEFAULT_TABLE table=default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:18.115911 21777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7efa9e6cb797487bb11bf47ad3c320ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:18.120325 22462 raft_consensus.cc:515] T 456a6d0a631a4c9aa8d343eb014555ce P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.110271 22368 consensus_queue.cc:237] T edfa0efda2a24efd89c38ef688c917d5 P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.129467 22212 raft_consensus.cc:359] T 456a6d0a631a4c9aa8d343eb014555ce P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.131006 22211 tablet_bootstrap.cc:492] T f3b0e015d5dd4a668b6a04b49f628070 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:18.134778 21958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "456a6d0a631a4c9aa8d343eb014555ce" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
I20260925 13:57:18.135378 21958 raft_consensus.cc:3060] T 456a6d0a631a4c9aa8d343eb014555ce P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:18.136765 21940 tablet_service.cc:1511] Processing CreateTablet for tablet 7efa9e6cb797487bb11bf47ad3c320ba (DEFAULT_TABLE table=default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:18.137866 21940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7efa9e6cb797487bb11bf47ad3c320ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:18.138207 22106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "456a6d0a631a4c9aa8d343eb014555ce" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:18.139801 22212 raft_consensus.cc:385] T 456a6d0a631a4c9aa8d343eb014555ce P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:18.146452 21958 raft_consensus.cc:2468] T 456a6d0a631a4c9aa8d343eb014555ce P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:18.147836 21732 leader_election.cc:304] T 456a6d0a631a4c9aa8d343eb014555ce P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:18.148563 22378 raft_consensus.cc:2804] T 456a6d0a631a4c9aa8d343eb014555ce P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:18.143165 22085 tablet_service.cc:1511] Processing CreateTablet for tablet 7efa9e6cb797487bb11bf47ad3c320ba (DEFAULT_TABLE table=default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:18.153245 22212 raft_consensus.cc:740] T 456a6d0a631a4c9aa8d343eb014555ce P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:18.154244 22085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7efa9e6cb797487bb11bf47ad3c320ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:18.140128 22462 leader_election.cc:290] T 456a6d0a631a4c9aa8d343eb014555ce P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Requested vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:18.156219 22415 ts_tablet_manager.cc:1929] T d71049b217a74662b7a9974e776d1448 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:57:18.156569 22415 log.cc:1199] T d71049b217a74662b7a9974e776d1448 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/d71049b217a74662b7a9974e776d1448
I20260925 13:57:18.157239 22415 ts_tablet_manager.cc:1950] T d71049b217a74662b7a9974e776d1448 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:18.147219 22214 ts_tablet_manager.cc:1929] T dc9cc9c6d85446b786e8ea4396b33a14 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:18.161892 22214 log.cc:1199] T dc9cc9c6d85446b786e8ea4396b33a14 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/dc9cc9c6d85446b786e8ea4396b33a14
I20260925 13:57:18.162626 22214 ts_tablet_manager.cc:1950] T dc9cc9c6d85446b786e8ea4396b33a14 P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:18.166603 22415 tablet_replica.cc:333] T 456a6d0a631a4c9aa8d343eb014555ce P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:18.167173 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet d71049b217a74662b7a9974e776d1448 (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]) successfully deleted
I20260925 13:57:18.169143 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet dc9cc9c6d85446b786e8ea4396b33a14 (table default.table_bd714c618f7a4f16a69a69d5932b2014 [id=dfd05c11872140cfa86b2628c33ef6d8]) successfully deleted
I20260925 13:57:18.140142 22293 ts_tablet_manager.cc:1403] T f3b0e015d5dd4a668b6a04b49f628070 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.201s user 0.014s sys 0.005s
I20260925 13:57:18.174705 22378 raft_consensus.cc:697] T 456a6d0a631a4c9aa8d343eb014555ce P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Becoming Leader. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Running, Role: LEADER
I20260925 13:57:18.154067 22212 consensus_queue.cc:260] T 456a6d0a631a4c9aa8d343eb014555ce P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.175522 22106 raft_consensus.cc:3060] T 456a6d0a631a4c9aa8d343eb014555ce P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:18.175518 22378 consensus_queue.cc:237] T 456a6d0a631a4c9aa8d343eb014555ce P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.184079 22106 raft_consensus.cc:2468] T 456a6d0a631a4c9aa8d343eb014555ce P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:18.192675 21585 catalog_manager.cc:2213] created HMS catalog entry for table [id=9e8b522f92404f2ba5505e6fa689ea5a]
I20260925 13:57:18.194347 22214 tablet_replica.cc:333] T e4fe2c4187b34b5dba98873ab0d5abdb P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:18.195241 22214 raft_consensus.cc:2243] T e4fe2c4187b34b5dba98873ab0d5abdb P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:18.195765 22214 raft_consensus.cc:2272] T e4fe2c4187b34b5dba98873ab0d5abdb P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:18.198766 21581 catalog_manager.cc:5838] T edfa0efda2a24efd89c38ef688c917d5 P 145e520e0a5a445d943a53cf8c27efee reported cstate change: term changed from 0 to 1, leader changed from <none> to 145e520e0a5a445d943a53cf8c27efee (127.18.182.194). New cstate: current_term: 1 leader_uuid: "145e520e0a5a445d943a53cf8c27efee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: UNKNOWN } } }
I20260925 13:57:18.213974 22211 tablet_bootstrap.cc:654] T f3b0e015d5dd4a668b6a04b49f628070 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:18.207306 22212 ts_tablet_manager.cc:1434] T 456a6d0a631a4c9aa8d343eb014555ce P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.101s user 0.005s sys 0.000s
I20260925 13:57:18.216104 22293 raft_consensus.cc:359] T f3b0e015d5dd4a668b6a04b49f628070 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.222270 22212 tablet_bootstrap.cc:492] T f3b0e015d5dd4a668b6a04b49f628070 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:18.228000 22293 raft_consensus.cc:385] T f3b0e015d5dd4a668b6a04b49f628070 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:18.231470 22293 raft_consensus.cc:740] T f3b0e015d5dd4a668b6a04b49f628070 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:18.232270 22293 consensus_queue.cc:260] T f3b0e015d5dd4a668b6a04b49f628070 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.245913 22462 consensus_queue.cc:1048] T abadb0b2c7d34451b7b23489e6d866c6 P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:18.248176 21585 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35994 (request call id 60) took 2730 ms. Trace:
I20260925 13:57:18.251647 21581 master_service.cc:960] ReplaceTablet: received request to replace tablet e07f41572a914b8a86c063c65e73e072 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:18.248476 21585 rpcz_store.cc:276] 0925 13:57:15.518265 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:15.521502 (+ 3237us) service_pool.cc:224] Handling call
0925 13:57:16.322175 (+800673us) catalog_manager.cc:2140] Acquired catalog manager lock
0925 13:57:16.331121 (+ 8946us) catalog_manager.cc:2186] Created new table and tablet info
0925 13:57:18.192514 (+1861393us) catalog_manager.cc:2212] Created new table in HMS catalog
0925 13:57:18.220983 (+ 28469us) catalog_manager.cc:2246] Wrote table and tablets to system table
0925 13:57:18.247824 (+ 26841us) catalog_manager.cc:2280] Inserted table and tablets into CatalogManager maps
0925 13:57:18.248123 (+ 299us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:57:18.200915 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:57:18.201351 (+ 436us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:57:18.201373 (+ 22us) write_op.cc:435] Acquired schema lock
0925 13:57:18.201386 (+ 13us) tablet.cc:662] Decoding operations
0925 13:57:18.201624 (+ 238us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:57:18.201662 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0925 13:57:18.201683 (+ 21us) tablet.cc:685] Acquiring locks for 2 operations
0925 13:57:18.201859 (+ 176us) tablet.cc:701] Row locks acquired
0925 13:57:18.201872 (+ 13us) write_op.cc:260] PREPARE: finished
0925 13:57:18.201971 (+ 99us) write_op.cc:270] Start()
0925 13:57:18.202047 (+ 76us) write_op.cc:276] Timestamp: P: 1790344638201924 usec, L: 0
0925 13:57:18.202061 (+ 14us) op_driver.cc:348] REPLICATION: starting
0925 13:57:18.202526 (+ 465us) log.cc:844] Serialized 432 byte log entry
0925 13:57:18.211709 (+ 9183us) op_driver.cc:464] REPLICATION: finished
0925 13:57:18.212122 (+ 413us) write_op.cc:301] APPLY: starting
0925 13:57:18.212167 (+ 45us) tablet.cc:1394] starting BulkCheckPresence
0925 13:57:18.212255 (+ 88us) tablet.cc:1397] finished BulkCheckPresence
0925 13:57:18.212331 (+ 76us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:57:18.212623 (+ 292us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:57:18.212732 (+ 109us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 13:57:18.212766 (+ 34us) write_op.cc:312] APPLY: finished
0925 13:57:18.212912 (+ 146us) log.cc:844] Serialized 28 byte log entry
0925 13:57:18.213030 (+ 118us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:18.213086 (+ 56us) write_op.cc:454] Released schema lock
0925 13:57:18.213111 (+ 25us) write_op.cc:341] FINISH: Updating metrics
0925 13:57:18.213286 (+ 175us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:18.213305 (+ 19us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":1428630,"Hive Metastore.run_cpu_time_us":683,"Hive Metastore.run_wall_time_us":429904,"child_traces":[["op",{"apply.queue_time_us":280,"apply.run_cpu_time_us":1141,"apply.run_wall_time_us":1141,"num_ops":2,"prepare.queue_time_us":6737,"prepare.run_cpu_time_us":1968,"prepare.run_wall_time_us":3172,"raft.queue_time_us":129,"raft.run_cpu_time_us":645,"raft.run_wall_time_us":647,"replication_time_us":9567,"spinlock_wait_cycles":35200}]]}
W20260925 13:57:18.254467 21582 catalog_manager.cc:5671] Ignoring report from unknown tablet 456a6d0a631a4c9aa8d343eb014555ce
I20260925 13:57:18.267633 22293 ts_tablet_manager.cc:1434] T f3b0e015d5dd4a668b6a04b49f628070 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.094s user 0.001s sys 0.004s
I20260925 13:57:18.268456 22293 tablet_bootstrap.cc:492] T f68ccd8a7f4a4e518c2584e579de0c87 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:18.272352 21581 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet e07f41572a914b8a86c063c65e73e072
I20260925 13:57:18.273916 22212 tablet_bootstrap.cc:654] T f3b0e015d5dd4a668b6a04b49f628070 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:18.275380 21581 catalog_manager.cc:6934] ReplaceTablet: tablet e07f41572a914b8a86c063c65e73e072 deleted and replaced by tablet f481b714a5f34fff8f74ba30bdd5dd35
I20260925 13:57:18.275468 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet e07f41572a914b8a86c063c65e73e072 with delete_type TABLET_DATA_DELETED (replaced by tablet f481b714a5f34fff8f74ba30bdd5dd35) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:18.277755 21777 tablet_service.cc:1558] Processing DeleteTablet for tablet e07f41572a914b8a86c063c65e73e072 with delete_type TABLET_DATA_DELETED (replaced by tablet f481b714a5f34fff8f74ba30bdd5dd35) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:18.277911 21937 tablet_service.cc:1558] Processing DeleteTablet for tablet 456a6d0a631a4c9aa8d343eb014555ce with delete_type TABLET_DATA_DELETED (replaced by tablet 72fc427f45934cba91e52ad84275c461) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:18.278535 21938 tablet_service.cc:1558] Processing DeleteTablet for tablet e07f41572a914b8a86c063c65e73e072 with delete_type TABLET_DATA_DELETED (replaced by tablet f481b714a5f34fff8f74ba30bdd5dd35) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:18.281108 22416 tablet_replica.cc:333] T 456a6d0a631a4c9aa8d343eb014555ce P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:18.281766 22416 raft_consensus.cc:2243] T 456a6d0a631a4c9aa8d343eb014555ce P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:18.282162 22416 raft_consensus.cc:2272] T 456a6d0a631a4c9aa8d343eb014555ce P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:18.284278 22416 ts_tablet_manager.cc:1916] T 456a6d0a631a4c9aa8d343eb014555ce P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:18.291491 22214 ts_tablet_manager.cc:1916] T e4fe2c4187b34b5dba98873ab0d5abdb P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:18.296847 21585 master_service.cc:960] ReplaceTablet: received request to replace tablet edfa0efda2a24efd89c38ef688c917d5 from {username='slave'} at 127.0.0.1:35994
W20260925 13:57:18.300695 21581 catalog_manager.cc:5671] Ignoring report from unknown tablet 456a6d0a631a4c9aa8d343eb014555ce
I20260925 13:57:18.301396 22415 raft_consensus.cc:2243] T 456a6d0a631a4c9aa8d343eb014555ce P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:18.302472 22415 pending_rounds.cc:70] T 456a6d0a631a4c9aa8d343eb014555ce P 5e2731975dbf433aade4a859df9699e2: Trying to abort 1 pending ops.
I20260925 13:57:18.302686 22415 pending_rounds.cc:77] T 456a6d0a631a4c9aa8d343eb014555ce P 5e2731975dbf433aade4a859df9699e2: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251637975457792 op_type: NO_OP noop_request { }
I20260925 13:57:18.302984 22415 raft_consensus.cc:2889] T 456a6d0a631a4c9aa8d343eb014555ce P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:57:18.303290 22415 raft_consensus.cc:2272] T 456a6d0a631a4c9aa8d343eb014555ce P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:18.307181 22415 ts_tablet_manager.cc:1916] T 456a6d0a631a4c9aa8d343eb014555ce P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 13:57:18.315418 21581 catalog_manager.cc:5671] Ignoring report from unknown tablet 456a6d0a631a4c9aa8d343eb014555ce
I20260925 13:57:18.322160 22378 consensus_queue.cc:1048] T abadb0b2c7d34451b7b23489e6d866c6 P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:18.350018 21585 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet edfa0efda2a24efd89c38ef688c917d5
I20260925 13:57:18.351554 21585 catalog_manager.cc:6934] ReplaceTablet: tablet edfa0efda2a24efd89c38ef688c917d5 deleted and replaced by tablet 60256642160a44a2bab77b97d78c27a8
I20260925 13:57:18.351724 21777 tablet_service.cc:1558] Processing DeleteTablet for tablet edfa0efda2a24efd89c38ef688c917d5 with delete_type TABLET_DATA_DELETED (replaced by tablet 60256642160a44a2bab77b97d78c27a8) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:18.353410 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet edfa0efda2a24efd89c38ef688c917d5 with delete_type TABLET_DATA_DELETED (replaced by tablet 60256642160a44a2bab77b97d78c27a8) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:18.355782 22211 tablet_bootstrap.cc:492] T f3b0e015d5dd4a668b6a04b49f628070 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:18.358830 22211 ts_tablet_manager.cc:1403] T f3b0e015d5dd4a668b6a04b49f628070 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.228s user 0.009s sys 0.002s
I20260925 13:57:18.367131 22293 tablet_bootstrap.cc:654] T f68ccd8a7f4a4e518c2584e579de0c87 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:18.401054 21938 tablet_service.cc:1511] Processing CreateTablet for tablet 72fc427f45934cba91e52ad84275c461 (DEFAULT_TABLE table=default.table_e0eb6c1d49224d1cb9b4f6d06f0f7a29 [id=420cd09cc1184f6e9fb38b2cbc3f6fce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:18.402052 21938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72fc427f45934cba91e52ad84275c461. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:18.407912 22083 tablet_service.cc:1511] Processing CreateTablet for tablet 72fc427f45934cba91e52ad84275c461 (DEFAULT_TABLE table=default.table_e0eb6c1d49224d1cb9b4f6d06f0f7a29 [id=420cd09cc1184f6e9fb38b2cbc3f6fce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:18.408804 22083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72fc427f45934cba91e52ad84275c461. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:57:18.423732 21573 consensus_peers.cc:597] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 -> Peer 3ea1e11bc6d84114b7616815b6af4635 (127.18.182.253:46177): Couldn't send request to peer 3ea1e11bc6d84114b7616815b6af4635. Status: Network error: Client connection negotiation failed: client connection to 127.18.182.253:46177: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260925 13:57:18.438297 21940 tablet_service.cc:1558] Processing DeleteTablet for tablet edfa0efda2a24efd89c38ef688c917d5 with delete_type TABLET_DATA_DELETED (replaced by tablet 60256642160a44a2bab77b97d78c27a8) from {username='slave'} at 127.0.0.1:55588
W20260925 13:57:18.442305 21643 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 761ede873f5648ccb5216a4c0a8afa2e consensus state has no leader
I20260925 13:57:18.444628 22416 ts_tablet_manager.cc:1929] T 456a6d0a631a4c9aa8d343eb014555ce P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:18.445017 22416 log.cc:1199] T 456a6d0a631a4c9aa8d343eb014555ce P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/456a6d0a631a4c9aa8d343eb014555ce
I20260925 13:57:18.445638 22416 ts_tablet_manager.cc:1950] T 456a6d0a631a4c9aa8d343eb014555ce P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:18.446558 22211 raft_consensus.cc:359] T f3b0e015d5dd4a668b6a04b49f628070 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.401629 22462 raft_consensus.cc:493] T 761ede873f5648ccb5216a4c0a8afa2e P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:18.447424 22462 raft_consensus.cc:515] T 761ede873f5648ccb5216a4c0a8afa2e P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.449316 22462 leader_election.cc:290] T 761ede873f5648ccb5216a4c0a8afa2e P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:18.406792 21777 tablet_service.cc:1511] Processing CreateTablet for tablet 72fc427f45934cba91e52ad84275c461 (DEFAULT_TABLE table=default.table_e0eb6c1d49224d1cb9b4f6d06f0f7a29 [id=420cd09cc1184f6e9fb38b2cbc3f6fce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:18.450306 21777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72fc427f45934cba91e52ad84275c461. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:18.456686 22212 tablet_bootstrap.cc:492] T f3b0e015d5dd4a668b6a04b49f628070 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:18.457091 22212 ts_tablet_manager.cc:1403] T f3b0e015d5dd4a668b6a04b49f628070 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.235s user 0.007s sys 0.001s
I20260925 13:57:18.457578 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet 456a6d0a631a4c9aa8d343eb014555ce (table default.table_e0eb6c1d49224d1cb9b4f6d06f0f7a29 [id=420cd09cc1184f6e9fb38b2cbc3f6fce]) successfully deleted
W20260925 13:57:18.458251 21575 catalog_manager.cc:5043] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): delete failed for tablet e07f41572a914b8a86c063c65e73e072 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e07f41572a914b8a86c063c65e73e072 already in progress: creating tablet
I20260925 13:57:18.460242 22211 raft_consensus.cc:385] T f3b0e015d5dd4a668b6a04b49f628070 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:18.459883 21582 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35994:
name: "default.Table_d8ce7cda4ecf4422a5e74c80d7ebf4da"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:57:18.460521 22211 raft_consensus.cc:740] T f3b0e015d5dd4a668b6a04b49f628070 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:18.461174 22211 consensus_queue.cc:260] T f3b0e015d5dd4a668b6a04b49f628070 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.469771 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ede873f5648ccb5216a4c0a8afa2e" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
W20260925 13:57:18.471261 21732 leader_election.cc:343] T 761ede873f5648ccb5216a4c0a8afa2e P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:57:18.479761 22214 ts_tablet_manager.cc:1929] T e4fe2c4187b34b5dba98873ab0d5abdb P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:18.480129 22214 log.cc:1199] T e4fe2c4187b34b5dba98873ab0d5abdb P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/e4fe2c4187b34b5dba98873ab0d5abdb
I20260925 13:57:18.480775 22214 ts_tablet_manager.cc:1950] T e4fe2c4187b34b5dba98873ab0d5abdb P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:18.483049 22214 tablet_replica.cc:333] T d71049b217a74662b7a9974e776d1448 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:18.483784 22415 ts_tablet_manager.cc:1929] T 456a6d0a631a4c9aa8d343eb014555ce P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:18.484126 22415 log.cc:1199] T 456a6d0a631a4c9aa8d343eb014555ce P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/456a6d0a631a4c9aa8d343eb014555ce
I20260925 13:57:18.484812 22415 ts_tablet_manager.cc:1950] T 456a6d0a631a4c9aa8d343eb014555ce P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:18.486652 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet e4fe2c4187b34b5dba98873ab0d5abdb (table default.table_61d48f9061c741fc8aa556e3edd984fb [id=7377fa3ea9664b47a930d9c5f88c60fa]) successfully deleted
I20260925 13:57:18.495294 21581 master_service.cc:960] ReplaceTablet: received request to replace tablet e07f41572a914b8a86c063c65e73e072 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:18.496580 22211 ts_tablet_manager.cc:1434] T f3b0e015d5dd4a668b6a04b49f628070 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.133s user 0.005s sys 0.001s
I20260925 13:57:18.455740 22416 tablet_replica.cc:333] T edfa0efda2a24efd89c38ef688c917d5 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:18.499730 22416 raft_consensus.cc:2243] T edfa0efda2a24efd89c38ef688c917d5 P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:18.500363 22212 raft_consensus.cc:359] T f3b0e015d5dd4a668b6a04b49f628070 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.501196 22212 raft_consensus.cc:385] T f3b0e015d5dd4a668b6a04b49f628070 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:18.503769 22416 pending_rounds.cc:70] T edfa0efda2a24efd89c38ef688c917d5 P 145e520e0a5a445d943a53cf8c27efee: Trying to abort 1 pending ops.
I20260925 13:57:18.504000 22416 pending_rounds.cc:77] T edfa0efda2a24efd89c38ef688c917d5 P 145e520e0a5a445d943a53cf8c27efee: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251637753110528 op_type: NO_OP noop_request { }
I20260925 13:57:18.504288 22416 raft_consensus.cc:2889] T edfa0efda2a24efd89c38ef688c917d5 P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:57:18.505765 22293 tablet_bootstrap.cc:492] T f68ccd8a7f4a4e518c2584e579de0c87 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:18.506145 22293 ts_tablet_manager.cc:1403] T f68ccd8a7f4a4e518c2584e579de0c87 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.238s user 0.004s sys 0.003s
I20260925 13:57:18.508569 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet 456a6d0a631a4c9aa8d343eb014555ce (table default.table_e0eb6c1d49224d1cb9b4f6d06f0f7a29 [id=420cd09cc1184f6e9fb38b2cbc3f6fce]) successfully deleted
I20260925 13:57:18.508863 22415 tablet_replica.cc:333] T edfa0efda2a24efd89c38ef688c917d5 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:18.509500 22415 raft_consensus.cc:2243] T edfa0efda2a24efd89c38ef688c917d5 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260925 13:57:18.509634 21572 catalog_manager.cc:5043] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): delete failed for tablet e07f41572a914b8a86c063c65e73e072 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e07f41572a914b8a86c063c65e73e072 already in progress: creating tablet
I20260925 13:57:18.509925 22415 raft_consensus.cc:2272] T edfa0efda2a24efd89c38ef688c917d5 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:18.512413 22416 raft_consensus.cc:2272] T edfa0efda2a24efd89c38ef688c917d5 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:18.501479 22212 raft_consensus.cc:740] T f3b0e015d5dd4a668b6a04b49f628070 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:18.518941 22214 raft_consensus.cc:2243] T d71049b217a74662b7a9974e776d1448 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:18.519522 22214 raft_consensus.cc:2272] T d71049b217a74662b7a9974e776d1448 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:18.521637 22214 ts_tablet_manager.cc:1916] T d71049b217a74662b7a9974e776d1448 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:18.524883 22212 consensus_queue.cc:260] T f3b0e015d5dd4a668b6a04b49f628070 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.526383 22416 ts_tablet_manager.cc:1916] T edfa0efda2a24efd89c38ef688c917d5 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:18.526978 21958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ede873f5648ccb5216a4c0a8afa2e" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:18.531929 22211 tablet_bootstrap.cc:492] T 761ede873f5648ccb5216a4c0a8afa2e P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:18.538323 22211 tablet_bootstrap.cc:654] T 761ede873f5648ccb5216a4c0a8afa2e P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:18.542239 22212 ts_tablet_manager.cc:1434] T f3b0e015d5dd4a668b6a04b49f628070 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.085s user 0.004s sys 0.002s
I20260925 13:57:18.562403 22212 tablet_bootstrap.cc:492] T 761ede873f5648ccb5216a4c0a8afa2e P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:18.509563 22293 raft_consensus.cc:359] T f68ccd8a7f4a4e518c2584e579de0c87 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.567790 22293 raft_consensus.cc:385] T f68ccd8a7f4a4e518c2584e579de0c87 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:18.568128 22293 raft_consensus.cc:740] T f68ccd8a7f4a4e518c2584e579de0c87 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:18.567844 22415 ts_tablet_manager.cc:1916] T edfa0efda2a24efd89c38ef688c917d5 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:18.573957 22211 tablet_bootstrap.cc:492] T 761ede873f5648ccb5216a4c0a8afa2e P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:18.574332 22211 ts_tablet_manager.cc:1403] T 761ede873f5648ccb5216a4c0a8afa2e P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.043s user 0.008s sys 0.000s
W20260925 13:57:18.575825 21732 leader_election.cc:343] T 761ede873f5648ccb5216a4c0a8afa2e P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:57:18.576570 21732 leader_election.cc:304] T 761ede873f5648ccb5216a4c0a8afa2e P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
W20260925 13:57:18.577416 21732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.194:44291, user_credentials={real_user=slave}} blocked reactor thread for 48541.7us
I20260925 13:57:18.577919 22504 raft_consensus.cc:2749] T 761ede873f5648ccb5216a4c0a8afa2e P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:57:18.588155 22293 consensus_queue.cc:260] T f68ccd8a7f4a4e518c2584e579de0c87 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.594661 22293 ts_tablet_manager.cc:1434] T f68ccd8a7f4a4e518c2584e579de0c87 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.088s user 0.008s sys 0.004s
I20260925 13:57:18.595497 22293 tablet_bootstrap.cc:492] T e07f41572a914b8a86c063c65e73e072 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:18.600060 22212 tablet_bootstrap.cc:654] T 761ede873f5648ccb5216a4c0a8afa2e P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:18.597893 22211 raft_consensus.cc:359] T 761ede873f5648ccb5216a4c0a8afa2e P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.610638 22211 raft_consensus.cc:385] T 761ede873f5648ccb5216a4c0a8afa2e P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:18.610879 22211 raft_consensus.cc:740] T 761ede873f5648ccb5216a4c0a8afa2e P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:18.614799 22211 consensus_queue.cc:260] T 761ede873f5648ccb5216a4c0a8afa2e P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.623057 22416 ts_tablet_manager.cc:1929] T edfa0efda2a24efd89c38ef688c917d5 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:18.625325 21581 master_service.cc:960] ReplaceTablet: received request to replace tablet 7efa9e6cb797487bb11bf47ad3c320ba from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:18.632709 22211 ts_tablet_manager.cc:1434] T 761ede873f5648ccb5216a4c0a8afa2e P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.058s user 0.007s sys 0.000s
I20260925 13:57:18.633503 22211 tablet_bootstrap.cc:492] T f68ccd8a7f4a4e518c2584e579de0c87 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:18.647441 22416 log.cc:1199] T edfa0efda2a24efd89c38ef688c917d5 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/edfa0efda2a24efd89c38ef688c917d5
I20260925 13:57:18.648294 22416 ts_tablet_manager.cc:1950] T edfa0efda2a24efd89c38ef688c917d5 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:18.654510 22293 tablet_bootstrap.cc:654] T e07f41572a914b8a86c063c65e73e072 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:18.659005 22211 tablet_bootstrap.cc:654] T f68ccd8a7f4a4e518c2584e579de0c87 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:18.659144 22214 ts_tablet_manager.cc:1929] T d71049b217a74662b7a9974e776d1448 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:57:18.661064 22214 log.cc:1199] T d71049b217a74662b7a9974e776d1448 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/d71049b217a74662b7a9974e776d1448
I20260925 13:57:18.661124 22415 ts_tablet_manager.cc:1929] T edfa0efda2a24efd89c38ef688c917d5 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:18.661480 22415 log.cc:1199] T edfa0efda2a24efd89c38ef688c917d5 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/edfa0efda2a24efd89c38ef688c917d5
I20260925 13:57:18.662561 22214 ts_tablet_manager.cc:1950] T d71049b217a74662b7a9974e776d1448 P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:18.663612 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet edfa0efda2a24efd89c38ef688c917d5 (table default.table_8b5db7d66475449d9772db568c9ded56 [id=7fb6c448c78344719796980b4be1cda9]) successfully deleted
I20260925 13:57:18.663645 21581 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet 7efa9e6cb797487bb11bf47ad3c320ba
I20260925 13:57:18.665400 21777 tablet_service.cc:1558] Processing DeleteTablet for tablet 7efa9e6cb797487bb11bf47ad3c320ba with delete_type TABLET_DATA_DELETED (replaced by tablet f66354b75eb8446396a27c3d563191cc) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:18.666651 21938 tablet_service.cc:1558] Processing DeleteTablet for tablet 7efa9e6cb797487bb11bf47ad3c320ba with delete_type TABLET_DATA_DELETED (replaced by tablet f66354b75eb8446396a27c3d563191cc) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:18.667743 21581 catalog_manager.cc:6934] ReplaceTablet: tablet 7efa9e6cb797487bb11bf47ad3c320ba deleted and replaced by tablet f66354b75eb8446396a27c3d563191cc
I20260925 13:57:18.667804 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet 7efa9e6cb797487bb11bf47ad3c320ba with delete_type TABLET_DATA_DELETED (replaced by tablet f66354b75eb8446396a27c3d563191cc) from {username='slave'} at 127.0.0.1:50052
W20260925 13:57:18.668592 21575 catalog_manager.cc:5043] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): delete failed for tablet 7efa9e6cb797487bb11bf47ad3c320ba because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7efa9e6cb797487bb11bf47ad3c320ba already in progress: creating tablet
I20260925 13:57:18.670524 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet d71049b217a74662b7a9974e776d1448 (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]) successfully deleted
I20260925 13:57:18.662057 22415 ts_tablet_manager.cc:1950] T edfa0efda2a24efd89c38ef688c917d5 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:18.672976 22214 tablet_replica.cc:333] T 456a6d0a631a4c9aa8d343eb014555ce P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:18.673686 22214 raft_consensus.cc:2243] T 456a6d0a631a4c9aa8d343eb014555ce P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:18.674108 22214 raft_consensus.cc:2272] T 456a6d0a631a4c9aa8d343eb014555ce P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:18.675698 22214 ts_tablet_manager.cc:1916] T 456a6d0a631a4c9aa8d343eb014555ce P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:18.681322 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet edfa0efda2a24efd89c38ef688c917d5 (table default.table_8b5db7d66475449d9772db568c9ded56 [id=7fb6c448c78344719796980b4be1cda9]) successfully deleted
W20260925 13:57:18.685499 21572 catalog_manager.cc:5043] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): delete failed for tablet 7efa9e6cb797487bb11bf47ad3c320ba because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7efa9e6cb797487bb11bf47ad3c320ba already in progress: creating tablet
I20260925 13:57:18.686429 22211 tablet_bootstrap.cc:492] T f68ccd8a7f4a4e518c2584e579de0c87 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:18.688037 22211 ts_tablet_manager.cc:1403] T f68ccd8a7f4a4e518c2584e579de0c87 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.055s user 0.005s sys 0.004s
I20260925 13:57:18.687817 21585 catalog_manager.cc:5938] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291) does not have the latest schema for tablet 761ede873f5648ccb5216a4c0a8afa2e (table default.table_d6100b0f2b294ab099c179a7681cb4ed [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 2 got 0
I20260925 13:57:18.691905 22211 raft_consensus.cc:359] T f68ccd8a7f4a4e518c2584e579de0c87 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.699077 22211 raft_consensus.cc:385] T f68ccd8a7f4a4e518c2584e579de0c87 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:18.699769 22211 raft_consensus.cc:740] T f68ccd8a7f4a4e518c2584e579de0c87 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:18.700976 22211 consensus_queue.cc:260] T f68ccd8a7f4a4e518c2584e579de0c87 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.708930 22083 tablet_service.cc:1511] Processing CreateTablet for tablet f5bd96a2bdb842e9b3bc1f706786d6b1 (DEFAULT_TABLE table=default.table_fe2e32c0275e4744b1163ae7b22eaf51 [id=9e8b522f92404f2ba5505e6fa689ea5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:18.709940 22083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5bd96a2bdb842e9b3bc1f706786d6b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:18.712958 21938 tablet_service.cc:1511] Processing CreateTablet for tablet f5bd96a2bdb842e9b3bc1f706786d6b1 (DEFAULT_TABLE table=default.table_fe2e32c0275e4744b1163ae7b22eaf51 [id=9e8b522f92404f2ba5505e6fa689ea5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:18.713383 21940 tablet_service.cc:1511] Processing CreateTablet for tablet f481b714a5f34fff8f74ba30bdd5dd35 (DEFAULT_TABLE table=default.table_61d48f9061c741fc8aa556e3edd984fb [id=7377fa3ea9664b47a930d9c5f88c60fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:18.714407 21940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f481b714a5f34fff8f74ba30bdd5dd35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:18.717379 21777 tablet_service.cc:1511] Processing CreateTablet for tablet f5bd96a2bdb842e9b3bc1f706786d6b1 (DEFAULT_TABLE table=default.table_fe2e32c0275e4744b1163ae7b22eaf51 [id=9e8b522f92404f2ba5505e6fa689ea5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:18.718294 21777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5bd96a2bdb842e9b3bc1f706786d6b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:18.720059 21776 tablet_service.cc:1511] Processing CreateTablet for tablet f481b714a5f34fff8f74ba30bdd5dd35 (DEFAULT_TABLE table=default.table_61d48f9061c741fc8aa556e3edd984fb [id=7377fa3ea9664b47a930d9c5f88c60fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:18.720995 21776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f481b714a5f34fff8f74ba30bdd5dd35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:18.722595 21938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5bd96a2bdb842e9b3bc1f706786d6b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:18.729948 21581 master_service.cc:960] ReplaceTablet: received request to replace tablet abadb0b2c7d34451b7b23489e6d866c6 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:18.734423 22085 tablet_service.cc:1511] Processing CreateTablet for tablet f481b714a5f34fff8f74ba30bdd5dd35 (DEFAULT_TABLE table=default.table_61d48f9061c741fc8aa556e3edd984fb [id=7377fa3ea9664b47a930d9c5f88c60fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:18.737497 21775 tablet_service.cc:1511] Processing CreateTablet for tablet 60256642160a44a2bab77b97d78c27a8 (DEFAULT_TABLE table=default.table_8b5db7d66475449d9772db568c9ded56 [id=7fb6c448c78344719796980b4be1cda9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:18.738680 21775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60256642160a44a2bab77b97d78c27a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:18.738977 22293 tablet_bootstrap.cc:492] T e07f41572a914b8a86c063c65e73e072 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:18.739342 22293 ts_tablet_manager.cc:1403] T e07f41572a914b8a86c063c65e73e072 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.144s user 0.010s sys 0.007s
I20260925 13:57:18.740883 22085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f481b714a5f34fff8f74ba30bdd5dd35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:18.741911 22211 ts_tablet_manager.cc:1434] T f68ccd8a7f4a4e518c2584e579de0c87 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.053s user 0.005s sys 0.000s
I20260925 13:57:18.744750 21588 catalog_manager.cc:2770] deleted HMS catalog entry for table default.table_8b5db7d66475449d9772db568c9ded56 [id=7fb6c448c78344719796980b4be1cda9]
I20260925 13:57:18.745564 22087 tablet_service.cc:1511] Processing CreateTablet for tablet 60256642160a44a2bab77b97d78c27a8 (DEFAULT_TABLE table=default.table_8b5db7d66475449d9772db568c9ded56 [id=7fb6c448c78344719796980b4be1cda9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:18.746505 22087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60256642160a44a2bab77b97d78c27a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:18.755160 22211 tablet_bootstrap.cc:492] T e07f41572a914b8a86c063c65e73e072 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:18.742496 22293 raft_consensus.cc:359] T e07f41572a914b8a86c063c65e73e072 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.756575 22293 raft_consensus.cc:385] T e07f41572a914b8a86c063c65e73e072 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:18.756794 22293 raft_consensus.cc:740] T e07f41572a914b8a86c063c65e73e072 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:18.760110 22293 consensus_queue.cc:260] T e07f41572a914b8a86c063c65e73e072 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.766494 22293 ts_tablet_manager.cc:1434] T e07f41572a914b8a86c063c65e73e072 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.027s user 0.004s sys 0.000s
I20260925 13:57:18.767196 22293 tablet_bootstrap.cc:492] T 7efa9e6cb797487bb11bf47ad3c320ba P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:18.771999 21774 tablet_service.cc:1558] Processing DeleteTablet for tablet e07f41572a914b8a86c063c65e73e072 with delete_type TABLET_DATA_DELETED (replaced by tablet f481b714a5f34fff8f74ba30bdd5dd35) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:18.772652 22415 tablet_replica.cc:333] T e07f41572a914b8a86c063c65e73e072 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:18.773262 22415 raft_consensus.cc:2243] T e07f41572a914b8a86c063c65e73e072 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:18.773603 22415 raft_consensus.cc:2272] T e07f41572a914b8a86c063c65e73e072 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:18.775843 22415 ts_tablet_manager.cc:1916] T e07f41572a914b8a86c063c65e73e072 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:18.786419 22211 tablet_bootstrap.cc:654] T e07f41572a914b8a86c063c65e73e072 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:18.793728 21643 catalog_manager.cc:2811] Deleting table default.table_8b5db7d66475449d9772db568c9ded56 [id=7fb6c448c78344719796980b4be1cda9] in response to Hive Metastore notification log event 18
I20260925 13:57:18.802752 22212 tablet_bootstrap.cc:492] T 761ede873f5648ccb5216a4c0a8afa2e P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:18.806576 21581 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet abadb0b2c7d34451b7b23489e6d866c6
I20260925 13:57:18.806823 22462 raft_consensus.cc:493] T f68ccd8a7f4a4e518c2584e579de0c87 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:18.807226 22462 raft_consensus.cc:515] T f68ccd8a7f4a4e518c2584e579de0c87 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.803151 22212 ts_tablet_manager.cc:1403] T 761ede873f5648ccb5216a4c0a8afa2e P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.241s user 0.010s sys 0.017s
I20260925 13:57:18.808391 21937 tablet_service.cc:1558] Processing DeleteTablet for tablet abadb0b2c7d34451b7b23489e6d866c6 with delete_type TABLET_DATA_DELETED (replaced by tablet 76d4d7d706824892b075ab1aa4fbaf85) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:18.809134 22416 tablet_replica.cc:333] T abadb0b2c7d34451b7b23489e6d866c6 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:18.809158 22462 leader_election.cc:290] T f68ccd8a7f4a4e518c2584e579de0c87 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:18.809803 22416 raft_consensus.cc:2243] T abadb0b2c7d34451b7b23489e6d866c6 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:18.810243 22416 raft_consensus.cc:2272] T abadb0b2c7d34451b7b23489e6d866c6 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:18.810586 22212 raft_consensus.cc:359] T 761ede873f5648ccb5216a4c0a8afa2e P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.812639 22293 tablet_bootstrap.cc:654] T 7efa9e6cb797487bb11bf47ad3c320ba P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:18.813126 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68ccd8a7f4a4e518c2584e579de0c87" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:18.813730 21960 raft_consensus.cc:2468] T f68ccd8a7f4a4e518c2584e579de0c87 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:18.814663 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68ccd8a7f4a4e518c2584e579de0c87" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
W20260925 13:57:18.817077 21732 leader_election.cc:343] T f68ccd8a7f4a4e518c2584e579de0c87 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:57:18.817678 21732 leader_election.cc:304] T f68ccd8a7f4a4e518c2584e579de0c87 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters: ee02f77594864349982492862d668a36
I20260925 13:57:18.818645 22493 raft_consensus.cc:2804] T f68ccd8a7f4a4e518c2584e579de0c87 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:18.818967 22493 raft_consensus.cc:493] T f68ccd8a7f4a4e518c2584e579de0c87 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:18.819255 22493 raft_consensus.cc:3060] T f68ccd8a7f4a4e518c2584e579de0c87 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:18.825043 22416 ts_tablet_manager.cc:1916] T abadb0b2c7d34451b7b23489e6d866c6 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:18.825059 22415 ts_tablet_manager.cc:1929] T e07f41572a914b8a86c063c65e73e072 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:18.825362 22415 log.cc:1199] T e07f41572a914b8a86c063c65e73e072 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/e07f41572a914b8a86c063c65e73e072
I20260925 13:57:18.825960 22415 ts_tablet_manager.cc:1950] T e07f41572a914b8a86c063c65e73e072 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:18.832283 21938 tablet_service.cc:1511] Processing CreateTablet for tablet 60256642160a44a2bab77b97d78c27a8 (DEFAULT_TABLE table=default.table_8b5db7d66475449d9772db568c9ded56 [id=7fb6c448c78344719796980b4be1cda9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:18.833214 21938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60256642160a44a2bab77b97d78c27a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:18.834901 22211 tablet_bootstrap.cc:492] T e07f41572a914b8a86c063c65e73e072 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:18.835274 22211 ts_tablet_manager.cc:1403] T e07f41572a914b8a86c063c65e73e072 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.080s user 0.004s sys 0.003s
I20260925 13:57:18.837442 22211 raft_consensus.cc:359] T e07f41572a914b8a86c063c65e73e072 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.838055 22211 raft_consensus.cc:385] T e07f41572a914b8a86c063c65e73e072 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:18.838289 22211 raft_consensus.cc:740] T e07f41572a914b8a86c063c65e73e072 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:18.838330 22493 raft_consensus.cc:515] T f68ccd8a7f4a4e518c2584e579de0c87 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.841509 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68ccd8a7f4a4e518c2584e579de0c87" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
I20260925 13:57:18.842016 21960 raft_consensus.cc:3060] T f68ccd8a7f4a4e518c2584e579de0c87 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:18.846683 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet e07f41572a914b8a86c063c65e73e072 (table default.table_61d48f9061c741fc8aa556e3edd984fb [id=7377fa3ea9664b47a930d9c5f88c60fa]) successfully deleted
I20260925 13:57:18.847033 22211 consensus_queue.cc:260] T e07f41572a914b8a86c063c65e73e072 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.811194 22212 raft_consensus.cc:385] T 761ede873f5648ccb5216a4c0a8afa2e P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:18.850298 22212 raft_consensus.cc:740] T 761ede873f5648ccb5216a4c0a8afa2e P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:18.850947 22212 consensus_queue.cc:260] T 761ede873f5648ccb5216a4c0a8afa2e P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.853638 22211 ts_tablet_manager.cc:1434] T e07f41572a914b8a86c063c65e73e072 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.018s user 0.004s sys 0.000s
I20260925 13:57:18.854496 22211 tablet_bootstrap.cc:492] T 7efa9e6cb797487bb11bf47ad3c320ba P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:18.861760 22211 tablet_bootstrap.cc:654] T 7efa9e6cb797487bb11bf47ad3c320ba P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:18.864270 22212 ts_tablet_manager.cc:1434] T 761ede873f5648ccb5216a4c0a8afa2e P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.056s user 0.005s sys 0.000s
I20260925 13:57:18.865147 22212 tablet_bootstrap.cc:492] T e07f41572a914b8a86c063c65e73e072 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:18.876588 21960 raft_consensus.cc:2468] T f68ccd8a7f4a4e518c2584e579de0c87 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:18.878367 21732 leader_election.cc:304] T f68ccd8a7f4a4e518c2584e579de0c87 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:18.879113 22504 raft_consensus.cc:2804] T f68ccd8a7f4a4e518c2584e579de0c87 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:18.879626 22504 raft_consensus.cc:697] T f68ccd8a7f4a4e518c2584e579de0c87 P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Becoming Leader. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Running, Role: LEADER
I20260925 13:57:18.880645 22504 consensus_queue.cc:237] T f68ccd8a7f4a4e518c2584e579de0c87 P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.857331 22214 ts_tablet_manager.cc:1929] T 456a6d0a631a4c9aa8d343eb014555ce P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:18.882261 22214 log.cc:1199] T 456a6d0a631a4c9aa8d343eb014555ce P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/456a6d0a631a4c9aa8d343eb014555ce
I20260925 13:57:18.882908 22214 ts_tablet_manager.cc:1950] T 456a6d0a631a4c9aa8d343eb014555ce P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:18.895327 21775 tablet_service.cc:1558] Processing DeleteTablet for tablet abadb0b2c7d34451b7b23489e6d866c6 with delete_type TABLET_DATA_DELETED (replaced by tablet 76d4d7d706824892b075ab1aa4fbaf85) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:18.896104 22415 tablet_replica.cc:333] T abadb0b2c7d34451b7b23489e6d866c6 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:18.896749 22415 raft_consensus.cc:2243] T abadb0b2c7d34451b7b23489e6d866c6 P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:18.897490 22415 raft_consensus.cc:2272] T abadb0b2c7d34451b7b23489e6d866c6 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:18.899662 22415 ts_tablet_manager.cc:1916] T abadb0b2c7d34451b7b23489e6d866c6 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:18.901065 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet 456a6d0a631a4c9aa8d343eb014555ce (table default.table_e0eb6c1d49224d1cb9b4f6d06f0f7a29 [id=420cd09cc1184f6e9fb38b2cbc3f6fce]) successfully deleted
I20260925 13:57:18.903167 22212 tablet_bootstrap.cc:654] T e07f41572a914b8a86c063c65e73e072 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
W20260925 13:57:18.904753 21575 catalog_manager.cc:5043] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): delete failed for tablet e07f41572a914b8a86c063c65e73e072 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e07f41572a914b8a86c063c65e73e072 already in progress: creating tablet
I20260925 13:57:18.905335 21581 catalog_manager.cc:6934] ReplaceTablet: tablet abadb0b2c7d34451b7b23489e6d866c6 deleted and replaced by tablet 76d4d7d706824892b075ab1aa4fbaf85
I20260925 13:57:18.903724 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68ccd8a7f4a4e518c2584e579de0c87" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
W20260925 13:57:18.910535 21732 leader_election.cc:343] T f68ccd8a7f4a4e518c2584e579de0c87 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:57:18.907101 22214 tablet_replica.cc:333] T edfa0efda2a24efd89c38ef688c917d5 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:18.911648 22493 leader_election.cc:290] T f68ccd8a7f4a4e518c2584e579de0c87 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Requested vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:18.909396 22087 tablet_service.cc:1558] Processing DeleteTablet for tablet abadb0b2c7d34451b7b23489e6d866c6 with delete_type TABLET_DATA_DELETED (replaced by tablet 76d4d7d706824892b075ab1aa4fbaf85) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:18.911911 22214 raft_consensus.cc:2243] T edfa0efda2a24efd89c38ef688c917d5 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:18.915328 22214 raft_consensus.cc:2272] T edfa0efda2a24efd89c38ef688c917d5 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:18.916973 22214 ts_tablet_manager.cc:1916] T edfa0efda2a24efd89c38ef688c917d5 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:18.924182 22212 tablet_bootstrap.cc:492] T e07f41572a914b8a86c063c65e73e072 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:18.924536 22212 ts_tablet_manager.cc:1403] T e07f41572a914b8a86c063c65e73e072 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.060s user 0.006s sys 0.003s
I20260925 13:57:18.927314 22212 raft_consensus.cc:359] T e07f41572a914b8a86c063c65e73e072 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.932343 22212 raft_consensus.cc:385] T e07f41572a914b8a86c063c65e73e072 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:18.932578 22212 raft_consensus.cc:740] T e07f41572a914b8a86c063c65e73e072 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:18.933178 22212 consensus_queue.cc:260] T e07f41572a914b8a86c063c65e73e072 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.934062 22415 ts_tablet_manager.cc:1929] T abadb0b2c7d34451b7b23489e6d866c6 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:18.934396 22415 log.cc:1199] T abadb0b2c7d34451b7b23489e6d866c6 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/abadb0b2c7d34451b7b23489e6d866c6
I20260925 13:57:18.935014 22212 ts_tablet_manager.cc:1434] T e07f41572a914b8a86c063c65e73e072 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.010s user 0.004s sys 0.000s
I20260925 13:57:18.935088 22415 ts_tablet_manager.cc:1950] T abadb0b2c7d34451b7b23489e6d866c6 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:18.943531 22212 tablet_bootstrap.cc:492] T f68ccd8a7f4a4e518c2584e579de0c87 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:18.956200 22293 tablet_bootstrap.cc:492] T 7efa9e6cb797487bb11bf47ad3c320ba P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:18.956741 22293 ts_tablet_manager.cc:1403] T 7efa9e6cb797487bb11bf47ad3c320ba P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.190s user 0.017s sys 0.009s
I20260925 13:57:18.960322 22293 raft_consensus.cc:359] T 7efa9e6cb797487bb11bf47ad3c320ba P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.960922 22293 raft_consensus.cc:385] T 7efa9e6cb797487bb11bf47ad3c320ba P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:18.961189 22293 raft_consensus.cc:740] T 7efa9e6cb797487bb11bf47ad3c320ba P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:18.964704 22212 tablet_bootstrap.cc:654] T f68ccd8a7f4a4e518c2584e579de0c87 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:18.964847 21938 tablet_service.cc:1558] Processing DeleteTablet for tablet e07f41572a914b8a86c063c65e73e072 with delete_type TABLET_DATA_DELETED (replaced by tablet f481b714a5f34fff8f74ba30bdd5dd35) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:18.970153 22368 raft_consensus.cc:493] T 761ede873f5648ccb5216a4c0a8afa2e P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:18.970633 22368 raft_consensus.cc:515] T 761ede873f5648ccb5216a4c0a8afa2e P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.972657 22212 tablet_bootstrap.cc:492] T f68ccd8a7f4a4e518c2584e579de0c87 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:18.972689 22368 leader_election.cc:290] T 761ede873f5648ccb5216a4c0a8afa2e P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:18.973034 22212 ts_tablet_manager.cc:1403] T f68ccd8a7f4a4e518c2584e579de0c87 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.030s user 0.008s sys 0.000s
I20260925 13:57:18.973995 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet abadb0b2c7d34451b7b23489e6d866c6 (table default.table_8008d16a92cf4c0dafb258a1122bffb5 [id=a22f86f8c2404aa69bdeb9883471453c]) successfully deleted
I20260925 13:57:18.975183 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ede873f5648ccb5216a4c0a8afa2e" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2" is_pre_election: true
I20260925 13:57:18.975915 21797 raft_consensus.cc:2468] T 761ede873f5648ccb5216a4c0a8afa2e P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 0.
I20260925 13:57:18.976130 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ede873f5648ccb5216a4c0a8afa2e" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:18.976830 22107 raft_consensus.cc:2468] T 761ede873f5648ccb5216a4c0a8afa2e P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 0.
I20260925 13:57:18.977046 21893 leader_election.cc:304] T 761ede873f5648ccb5216a4c0a8afa2e P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:18.977761 22368 raft_consensus.cc:2804] T 761ede873f5648ccb5216a4c0a8afa2e P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:18.978081 22368 raft_consensus.cc:493] T 761ede873f5648ccb5216a4c0a8afa2e P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:18.978358 22368 raft_consensus.cc:3060] T 761ede873f5648ccb5216a4c0a8afa2e P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:18.975260 22212 raft_consensus.cc:359] T f68ccd8a7f4a4e518c2584e579de0c87 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.983737 22211 tablet_bootstrap.cc:492] T 7efa9e6cb797487bb11bf47ad3c320ba P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:18.984129 22211 ts_tablet_manager.cc:1403] T 7efa9e6cb797487bb11bf47ad3c320ba P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.130s user 0.023s sys 0.024s
I20260925 13:57:18.985860 22368 raft_consensus.cc:515] T 761ede873f5648ccb5216a4c0a8afa2e P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.986516 22211 raft_consensus.cc:359] T 7efa9e6cb797487bb11bf47ad3c320ba P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.987118 22211 raft_consensus.cc:385] T 7efa9e6cb797487bb11bf47ad3c320ba P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:18.987396 22211 raft_consensus.cc:740] T 7efa9e6cb797487bb11bf47ad3c320ba P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:18.988173 22416 ts_tablet_manager.cc:1929] T abadb0b2c7d34451b7b23489e6d866c6 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:18.988447 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ede873f5648ccb5216a4c0a8afa2e" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2"
I20260925 13:57:18.981144 22212 raft_consensus.cc:385] T f68ccd8a7f4a4e518c2584e579de0c87 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:18.989004 21797 raft_consensus.cc:3060] T 761ede873f5648ccb5216a4c0a8afa2e P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:18.989213 22212 raft_consensus.cc:740] T f68ccd8a7f4a4e518c2584e579de0c87 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:18.989949 22416 log.cc:1199] T abadb0b2c7d34451b7b23489e6d866c6 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/abadb0b2c7d34451b7b23489e6d866c6
I20260925 13:57:18.990167 22211 consensus_queue.cc:260] T 7efa9e6cb797487bb11bf47ad3c320ba P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.991602 22212 consensus_queue.cc:260] T f68ccd8a7f4a4e518c2584e579de0c87 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:18.995625 21775 tablet_service.cc:1511] Processing CreateTablet for tablet f66354b75eb8446396a27c3d563191cc (DEFAULT_TABLE table=default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:18.996658 22212 ts_tablet_manager.cc:1434] T f68ccd8a7f4a4e518c2584e579de0c87 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.023s user 0.007s sys 0.000s
I20260925 13:57:18.996703 21775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f66354b75eb8446396a27c3d563191cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:18.997459 22212 tablet_bootstrap.cc:492] T 7efa9e6cb797487bb11bf47ad3c320ba P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:18.998502 21797 raft_consensus.cc:2468] T 761ede873f5648ccb5216a4c0a8afa2e P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 1.
I20260925 13:57:18.999697 21893 leader_election.cc:304] T 761ede873f5648ccb5216a4c0a8afa2e P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:19.003319 21585 catalog_manager.cc:5938] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873) does not have the latest schema for tablet 761ede873f5648ccb5216a4c0a8afa2e (table default.table_d6100b0f2b294ab099c179a7681cb4ed [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 2 got 0
I20260925 13:57:19.007613 22293 consensus_queue.cc:260] T 7efa9e6cb797487bb11bf47ad3c320ba P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.014919 22212 tablet_bootstrap.cc:654] T 7efa9e6cb797487bb11bf47ad3c320ba P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:19.015913 22368 leader_election.cc:290] T 761ede873f5648ccb5216a4c0a8afa2e P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Requested vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:19.020274 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "761ede873f5648ccb5216a4c0a8afa2e" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:19.020812 22107 raft_consensus.cc:3060] T 761ede873f5648ccb5216a4c0a8afa2e P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:19.021039 22293 ts_tablet_manager.cc:1434] T 7efa9e6cb797487bb11bf47ad3c320ba P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.064s user 0.004s sys 0.003s
I20260925 13:57:19.021852 22293 tablet_bootstrap.cc:492] T 72fc427f45934cba91e52ad84275c461 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:19.024405 21643 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet 60256642160a44a2bab77b97d78c27a8
I20260925 13:57:19.027596 22087 tablet_service.cc:1558] Processing DeleteTablet for tablet 60256642160a44a2bab77b97d78c27a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:57:18 UTC) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:19.029942 22368 raft_consensus.cc:2804] T 761ede873f5648ccb5216a4c0a8afa2e P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:19.030371 22368 raft_consensus.cc:697] T 761ede873f5648ccb5216a4c0a8afa2e P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: Becoming Leader. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Running, Role: LEADER
I20260925 13:57:19.033679 21579 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:35994 (request call id 29) took 7948 ms. Trace:
I20260925 13:57:19.034071 21579 rpcz_store.cc:276] 0925 13:57:11.085528 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:11.152685 (+ 67157us) service_pool.cc:224] Handling call
0925 13:57:16.317850 (+5165165us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 13:57:19.031199 (+2713349us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 13:57:19.032744 (+ 1545us) catalog_manager.cc:3764] Apply alter schema
0925 13:57:19.033621 (+ 877us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":1569,"Hive Metastore.run_cpu_time_us":2458,"Hive Metastore.run_wall_time_us":436289}
I20260925 13:57:19.036657 21938 tablet_service.cc:1558] Processing DeleteTablet for tablet 60256642160a44a2bab77b97d78c27a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:57:18 UTC) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:19.038008 21777 tablet_service.cc:1558] Processing DeleteTablet for tablet 60256642160a44a2bab77b97d78c27a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:57:18 UTC) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:19.029486 21581 catalog_manager.cc:5838] T f68ccd8a7f4a4e518c2584e579de0c87 P 5e2731975dbf433aade4a859df9699e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e2731975dbf433aade4a859df9699e2 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "5e2731975dbf433aade4a859df9699e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: UNKNOWN } } }
W20260925 13:57:19.040822 21572 catalog_manager.cc:5043] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): delete failed for tablet 60256642160a44a2bab77b97d78c27a8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 60256642160a44a2bab77b97d78c27a8 already in progress: creating tablet
W20260925 13:57:19.042702 21580 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_fce30b75d46f47089d7a31e04841dcdf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:57:19.045734 22464 raft_consensus.cc:493] T f3b0e015d5dd4a668b6a04b49f628070 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260925 13:57:19.046147 21583 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_2138b0cdfa7c4afda3a919dd0945a2d1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:57:19.046151 22464 raft_consensus.cc:515] T f3b0e015d5dd4a668b6a04b49f628070 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.048102 22464 leader_election.cc:290] T f3b0e015d5dd4a668b6a04b49f628070 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:19.047681 22368 consensus_queue.cc:237] T 761ede873f5648ccb5216a4c0a8afa2e P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.048346 22211 ts_tablet_manager.cc:1434] T 7efa9e6cb797487bb11bf47ad3c320ba P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.064s user 0.011s sys 0.009s
I20260925 13:57:19.049213 22211 tablet_bootstrap.cc:492] T 72fc427f45934cba91e52ad84275c461 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
W20260925 13:57:19.049422 21584 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_dc864d7428e5405da360d2f379eef37b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:57:19.050758 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b0e015d5dd4a668b6a04b49f628070" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2" is_pre_election: true
I20260925 13:57:19.051335 21797 raft_consensus.cc:2468] T f3b0e015d5dd4a668b6a04b49f628070 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee02f77594864349982492862d668a36 in term 0.
I20260925 13:57:19.053370 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b0e015d5dd4a668b6a04b49f628070" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:19.053325 22040 leader_election.cc:304] T f3b0e015d5dd4a668b6a04b49f628070 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:19.053857 21960 raft_consensus.cc:2468] T f3b0e015d5dd4a668b6a04b49f628070 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee02f77594864349982492862d668a36 in term 0.
I20260925 13:57:19.054193 22464 raft_consensus.cc:2804] T f3b0e015d5dd4a668b6a04b49f628070 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:19.054492 22464 raft_consensus.cc:493] T f3b0e015d5dd4a668b6a04b49f628070 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:19.054782 22464 raft_consensus.cc:3060] T f3b0e015d5dd4a668b6a04b49f628070 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
W20260925 13:57:19.055408 21582 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_d8ce7cda4ecf4422a5e74c80d7ebf4da in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:57:19.061872 22293 tablet_bootstrap.cc:654] T 72fc427f45934cba91e52ad84275c461 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:19.062415 22211 tablet_bootstrap.cc:654] T 72fc427f45934cba91e52ad84275c461 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:19.041049 21585 master_service.cc:960] ReplaceTablet: received request to replace tablet 72fc427f45934cba91e52ad84275c461 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:19.043840 22107 raft_consensus.cc:2468] T 761ede873f5648ccb5216a4c0a8afa2e P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 1.
I20260925 13:57:19.071803 22211 tablet_bootstrap.cc:492] T 72fc427f45934cba91e52ad84275c461 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:19.072201 22211 ts_tablet_manager.cc:1403] T 72fc427f45934cba91e52ad84275c461 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.023s user 0.008s sys 0.000s
I20260925 13:57:19.072263 22416 ts_tablet_manager.cc:1950] T abadb0b2c7d34451b7b23489e6d866c6 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:19.074911 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet abadb0b2c7d34451b7b23489e6d866c6 (table default.table_8008d16a92cf4c0dafb258a1122bffb5 [id=a22f86f8c2404aa69bdeb9883471453c]) successfully deleted
I20260925 13:57:19.081275 21579 catalog_manager.cc:5938] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873) does not have the latest schema for tablet 761ede873f5648ccb5216a4c0a8afa2e (table default.table_d6100b0f2b294ab099c179a7681cb4ed [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 2 got 0
I20260925 13:57:19.074324 22211 raft_consensus.cc:359] T 72fc427f45934cba91e52ad84275c461 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.082069 22211 raft_consensus.cc:385] T 72fc427f45934cba91e52ad84275c461 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:19.082320 22211 raft_consensus.cc:740] T 72fc427f45934cba91e52ad84275c461 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:19.082911 22211 consensus_queue.cc:260] T 72fc427f45934cba91e52ad84275c461 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.084729 22211 ts_tablet_manager.cc:1434] T 72fc427f45934cba91e52ad84275c461 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.012s user 0.005s sys 0.000s
I20260925 13:57:19.085534 22211 tablet_bootstrap.cc:492] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:19.085950 21938 tablet_service.cc:1511] Processing CreateTablet for tablet f66354b75eb8446396a27c3d563191cc (DEFAULT_TABLE table=default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:19.086871 21938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f66354b75eb8446396a27c3d563191cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:19.088088 22464 raft_consensus.cc:515] T f3b0e015d5dd4a668b6a04b49f628070 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.090656 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b0e015d5dd4a668b6a04b49f628070" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2"
I20260925 13:57:19.091272 21797 raft_consensus.cc:3060] T f3b0e015d5dd4a668b6a04b49f628070 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:19.099422 22087 tablet_service.cc:1511] Processing CreateTablet for tablet f66354b75eb8446396a27c3d563191cc (DEFAULT_TABLE table=default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:19.100399 22087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f66354b75eb8446396a27c3d563191cc. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:57:19.111259 21573 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.182.253:46177: connect: Connection refused (error 111) [suppressed 10 similar messages]
I20260925 13:57:19.113611 22085 tablet_service.cc:1558] Processing DeleteTablet for tablet e07f41572a914b8a86c063c65e73e072 with delete_type TABLET_DATA_DELETED (replaced by tablet f481b714a5f34fff8f74ba30bdd5dd35) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:19.113859 22211 tablet_bootstrap.cc:654] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:19.116359 22416 tablet_replica.cc:333] T e07f41572a914b8a86c063c65e73e072 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:19.117023 22416 raft_consensus.cc:2243] T e07f41572a914b8a86c063c65e73e072 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:19.117444 22416 raft_consensus.cc:2272] T e07f41572a914b8a86c063c65e73e072 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Raft consensus is shut down!
W20260925 13:57:19.119865 21893 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.193:39881, user_credentials={real_user=slave}} blocked reactor thread for 120218us
I20260925 13:57:19.123493 22518 raft_consensus.cc:493] T 7efa9e6cb797487bb11bf47ad3c320ba P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:19.123909 22518 raft_consensus.cc:515] T 7efa9e6cb797487bb11bf47ad3c320ba P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.126405 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7efa9e6cb797487bb11bf47ad3c320ba" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2" is_pre_election: true
I20260925 13:57:19.126989 21796 raft_consensus.cc:2468] T 7efa9e6cb797487bb11bf47ad3c320ba P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 0.
I20260925 13:57:19.127179 22293 tablet_bootstrap.cc:492] T 72fc427f45934cba91e52ad84275c461 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:19.127777 22293 ts_tablet_manager.cc:1403] T 72fc427f45934cba91e52ad84275c461 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.106s user 0.010s sys 0.000s
I20260925 13:57:19.128777 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b0e015d5dd4a668b6a04b49f628070" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
I20260925 13:57:19.129243 21960 raft_consensus.cc:3060] T f3b0e015d5dd4a668b6a04b49f628070 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:19.130687 21893 leader_election.cc:304] T 7efa9e6cb797487bb11bf47ad3c320ba P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:19.130434 22293 raft_consensus.cc:359] T 72fc427f45934cba91e52ad84275c461 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.130926 22211 tablet_bootstrap.cc:492] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:19.131130 22293 raft_consensus.cc:385] T 72fc427f45934cba91e52ad84275c461 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:19.132297 21585 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet 72fc427f45934cba91e52ad84275c461
I20260925 13:57:19.131778 22368 raft_consensus.cc:2804] T 7efa9e6cb797487bb11bf47ad3c320ba P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:19.133030 22368 raft_consensus.cc:493] T 7efa9e6cb797487bb11bf47ad3c320ba P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:19.133342 22368 raft_consensus.cc:3060] T 7efa9e6cb797487bb11bf47ad3c320ba P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:19.134196 21777 tablet_service.cc:1558] Processing DeleteTablet for tablet 72fc427f45934cba91e52ad84275c461 with delete_type TABLET_DATA_DELETED (replaced by tablet 24dd39b302344f8d8f5166e9d641d13a) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:19.135675 21960 raft_consensus.cc:2468] T f3b0e015d5dd4a668b6a04b49f628070 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee02f77594864349982492862d668a36 in term 1.
W20260925 13:57:19.135882 21572 catalog_manager.cc:5043] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): delete failed for tablet 72fc427f45934cba91e52ad84275c461 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 72fc427f45934cba91e52ad84275c461 already in progress: creating tablet
I20260925 13:57:19.137112 22043 leader_election.cc:304] T f3b0e015d5dd4a668b6a04b49f628070 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:19.138669 21940 tablet_service.cc:1558] Processing DeleteTablet for tablet 72fc427f45934cba91e52ad84275c461 with delete_type TABLET_DATA_DELETED (replaced by tablet 24dd39b302344f8d8f5166e9d641d13a) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:19.139417 22368 raft_consensus.cc:515] T 7efa9e6cb797487bb11bf47ad3c320ba P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.139549 22085 tablet_service.cc:1558] Processing DeleteTablet for tablet 72fc427f45934cba91e52ad84275c461 with delete_type TABLET_DATA_DELETED (replaced by tablet 24dd39b302344f8d8f5166e9d641d13a) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:19.141148 22464 leader_election.cc:290] T f3b0e015d5dd4a668b6a04b49f628070 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 election: Requested vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:19.141368 22368 leader_election.cc:290] T 7efa9e6cb797487bb11bf47ad3c320ba P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Requested vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:19.131326 22211 ts_tablet_manager.cc:1403] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.046s user 0.002s sys 0.006s
I20260925 13:57:19.142109 22293 raft_consensus.cc:740] T 72fc427f45934cba91e52ad84275c461 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:19.142688 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7efa9e6cb797487bb11bf47ad3c320ba" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2"
I20260925 13:57:19.143077 21585 catalog_manager.cc:6934] ReplaceTablet: tablet 72fc427f45934cba91e52ad84275c461 deleted and replaced by tablet 24dd39b302344f8d8f5166e9d641d13a
I20260925 13:57:19.143213 21796 raft_consensus.cc:3060] T 7efa9e6cb797487bb11bf47ad3c320ba P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:19.142905 22293 consensus_queue.cc:260] T 72fc427f45934cba91e52ad84275c461 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.144400 22211 raft_consensus.cc:359] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.144948 22211 raft_consensus.cc:385] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:19.145185 22211 raft_consensus.cc:740] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:19.150017 21796 raft_consensus.cc:2468] T 7efa9e6cb797487bb11bf47ad3c320ba P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 1.
I20260925 13:57:19.151312 21893 leader_election.cc:304] T 7efa9e6cb797487bb11bf47ad3c320ba P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:19.152097 22517 raft_consensus.cc:2804] T 7efa9e6cb797487bb11bf47ad3c320ba P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:19.155113 21797 raft_consensus.cc:2468] T f3b0e015d5dd4a668b6a04b49f628070 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee02f77594864349982492862d668a36 in term 1.
I20260925 13:57:19.155617 22464 raft_consensus.cc:493] T e07f41572a914b8a86c063c65e73e072 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:19.156901 21940 tablet_service.cc:1558] Processing DeleteTablet for tablet 7efa9e6cb797487bb11bf47ad3c320ba with delete_type TABLET_DATA_DELETED (replaced by tablet f66354b75eb8446396a27c3d563191cc) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:19.156679 22211 consensus_queue.cc:260] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.157701 22368 raft_consensus.cc:493] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:19.158192 22368 raft_consensus.cc:515] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.160602 22416 ts_tablet_manager.cc:1916] T e07f41572a914b8a86c063c65e73e072 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:19.159065 22464 raft_consensus.cc:515] T e07f41572a914b8a86c063c65e73e072 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.164484 22520 raft_consensus.cc:2804] T f3b0e015d5dd4a668b6a04b49f628070 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:19.165119 22520 raft_consensus.cc:697] T f3b0e015d5dd4a668b6a04b49f628070 P ee02f77594864349982492862d668a36 [term 1 LEADER]: Becoming Leader. State: Replica: ee02f77594864349982492862d668a36, State: Running, Role: LEADER
W20260925 13:57:19.166273 22040 leader_election.cc:343] T e07f41572a914b8a86c063c65e73e072 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: e07f41572a914b8a86c063c65e73e072
I20260925 13:57:19.165993 22520 consensus_queue.cc:237] T f3b0e015d5dd4a668b6a04b49f628070 P ee02f77594864349982492862d668a36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:19.161377 21573 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 761ede873f5648ccb5216a4c0a8afa2e (table default.table_d6100b0f2b294ab099c179a7681cb4ed [id=74ee67384ecf4220b7d08fdf0637aefb], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 761ede873f5648ccb5216a4c0a8afa2e consensus state has no leader
I20260925 13:57:19.159215 22293 ts_tablet_manager.cc:1434] T 72fc427f45934cba91e52ad84275c461 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.031s user 0.005s sys 0.000s
I20260925 13:57:19.162870 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bd96a2bdb842e9b3bc1f706786d6b1" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2" is_pre_election: true
I20260925 13:57:19.175336 21777 tablet_service.cc:1558] Processing DeleteTablet for tablet 7efa9e6cb797487bb11bf47ad3c320ba with delete_type TABLET_DATA_DELETED (replaced by tablet f66354b75eb8446396a27c3d563191cc) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:19.176857 22415 tablet_replica.cc:333] T 7efa9e6cb797487bb11bf47ad3c320ba P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:19.177521 22415 raft_consensus.cc:2243] T 7efa9e6cb797487bb11bf47ad3c320ba P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:19.177951 22415 raft_consensus.cc:2272] T 7efa9e6cb797487bb11bf47ad3c320ba P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:19.180799 22415 ts_tablet_manager.cc:1916] T 7efa9e6cb797487bb11bf47ad3c320ba P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:19.162077 22368 leader_election.cc:290] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:19.181535 22293 tablet_bootstrap.cc:492] T f481b714a5f34fff8f74ba30bdd5dd35 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
W20260925 13:57:19.175618 21893 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:57:19.180908 22518 leader_election.cc:290] T 7efa9e6cb797487bb11bf47ad3c320ba P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 13:57:19.192451 22043 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.194:44291, user_credentials={real_user=slave}} blocked reactor thread for 55425.1us
I20260925 13:57:19.193946 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7efa9e6cb797487bb11bf47ad3c320ba" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:19.195679 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7efa9e6cb797487bb11bf47ad3c320ba" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:19.195749 22517 raft_consensus.cc:697] T 7efa9e6cb797487bb11bf47ad3c320ba P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: Becoming Leader. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Running, Role: LEADER
I20260925 13:57:19.196693 22106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bd96a2bdb842e9b3bc1f706786d6b1" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:19.196583 22517 consensus_queue.cc:237] T 7efa9e6cb797487bb11bf47ad3c320ba P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:19.197856 21896 leader_election.cc:343] T 7efa9e6cb797487bb11bf47ad3c320ba P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:57:19.199070 21896 leader_election.cc:343] T 7efa9e6cb797487bb11bf47ad3c320ba P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:57:19.200225 21896 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:57:19.200592 21896 leader_election.cc:304] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee; no voters: 5e2731975dbf433aade4a859df9699e2, ee02f77594864349982492862d668a36
I20260925 13:57:19.202093 22527 raft_consensus.cc:2749] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:57:19.203222 22464 leader_election.cc:290] T e07f41572a914b8a86c063c65e73e072 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:19.204847 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e07f41572a914b8a86c063c65e73e072" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
W20260925 13:57:19.206094 22043 leader_election.cc:343] T e07f41572a914b8a86c063c65e73e072 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Illegal state: Tablet not RUNNING: STOPPED
I20260925 13:57:19.206532 22043 leader_election.cc:304] T e07f41572a914b8a86c063c65e73e072 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:19.208467 22526 raft_consensus.cc:2749] T e07f41572a914b8a86c063c65e73e072 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:57:19.209076 22293 tablet_bootstrap.cc:654] T f481b714a5f34fff8f74ba30bdd5dd35 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:19.221621 22211 ts_tablet_manager.cc:1434] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.079s user 0.010s sys 0.013s
I20260925 13:57:19.223109 21777 tablet_service.cc:1558] Processing DeleteTablet for tablet 72fc427f45934cba91e52ad84275c461 with delete_type TABLET_DATA_DELETED (replaced by tablet 24dd39b302344f8d8f5166e9d641d13a) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:19.226447 22212 tablet_bootstrap.cc:492] T 7efa9e6cb797487bb11bf47ad3c320ba P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:19.226841 22212 ts_tablet_manager.cc:1403] T 7efa9e6cb797487bb11bf47ad3c320ba P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.230s user 0.029s sys 0.033s
I20260925 13:57:19.229351 22212 raft_consensus.cc:359] T 7efa9e6cb797487bb11bf47ad3c320ba P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.229914 22212 raft_consensus.cc:385] T 7efa9e6cb797487bb11bf47ad3c320ba P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:19.230196 22212 raft_consensus.cc:740] T 7efa9e6cb797487bb11bf47ad3c320ba P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:19.237833 22211 tablet_bootstrap.cc:492] T f481b714a5f34fff8f74ba30bdd5dd35 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:19.241590 22214 ts_tablet_manager.cc:1929] T edfa0efda2a24efd89c38ef688c917d5 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:19.243796 22211 tablet_bootstrap.cc:654] T f481b714a5f34fff8f74ba30bdd5dd35 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:19.246666 22212 consensus_queue.cc:260] T 7efa9e6cb797487bb11bf47ad3c320ba P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.247275 21581 master_service.cc:960] ReplaceTablet: received request to replace tablet d267def3f0864435a0d2c40fd7052eab from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:19.248836 22212 ts_tablet_manager.cc:1434] T 7efa9e6cb797487bb11bf47ad3c320ba P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.022s user 0.006s sys 0.000s
I20260925 13:57:19.249668 22212 tablet_bootstrap.cc:492] T 72fc427f45934cba91e52ad84275c461 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:19.253036 22293 tablet_bootstrap.cc:492] T f481b714a5f34fff8f74ba30bdd5dd35 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:19.253458 22293 ts_tablet_manager.cc:1403] T f481b714a5f34fff8f74ba30bdd5dd35 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.072s user 0.009s sys 0.000s
I20260925 13:57:19.254988 22212 tablet_bootstrap.cc:654] T 72fc427f45934cba91e52ad84275c461 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
W20260925 13:57:19.260730 21581 catalog_manager.cc:5671] Ignoring report from unknown tablet 7efa9e6cb797487bb11bf47ad3c320ba
I20260925 13:57:19.268822 22212 tablet_bootstrap.cc:492] T 72fc427f45934cba91e52ad84275c461 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:19.269196 22212 ts_tablet_manager.cc:1403] T 72fc427f45934cba91e52ad84275c461 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.004s
I20260925 13:57:19.269577 22416 ts_tablet_manager.cc:1929] T e07f41572a914b8a86c063c65e73e072 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:19.269984 22416 log.cc:1199] T e07f41572a914b8a86c063c65e73e072 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/e07f41572a914b8a86c063c65e73e072
I20260925 13:57:19.241954 22214 log.cc:1199] T edfa0efda2a24efd89c38ef688c917d5 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/edfa0efda2a24efd89c38ef688c917d5
I20260925 13:57:19.271308 22214 ts_tablet_manager.cc:1950] T edfa0efda2a24efd89c38ef688c917d5 P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:19.271970 22293 raft_consensus.cc:359] T f481b714a5f34fff8f74ba30bdd5dd35 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.272565 22293 raft_consensus.cc:385] T f481b714a5f34fff8f74ba30bdd5dd35 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:19.271417 22416 ts_tablet_manager.cc:1950] T e07f41572a914b8a86c063c65e73e072 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:19.272801 22293 raft_consensus.cc:740] T f481b714a5f34fff8f74ba30bdd5dd35 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:19.277078 22211 tablet_bootstrap.cc:492] T f481b714a5f34fff8f74ba30bdd5dd35 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:19.277446 22211 ts_tablet_manager.cc:1403] T f481b714a5f34fff8f74ba30bdd5dd35 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.040s user 0.009s sys 0.001s
I20260925 13:57:19.279421 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet e07f41572a914b8a86c063c65e73e072 (table default.table_61d48f9061c741fc8aa556e3edd984fb [id=7377fa3ea9664b47a930d9c5f88c60fa]) successfully deleted
I20260925 13:57:19.279647 22211 raft_consensus.cc:359] T f481b714a5f34fff8f74ba30bdd5dd35 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.280249 22211 raft_consensus.cc:385] T f481b714a5f34fff8f74ba30bdd5dd35 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:19.280517 22211 raft_consensus.cc:740] T f481b714a5f34fff8f74ba30bdd5dd35 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:19.281137 22211 consensus_queue.cc:260] T f481b714a5f34fff8f74ba30bdd5dd35 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:19.282822 21575 catalog_manager.cc:5043] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): delete failed for tablet 60256642160a44a2bab77b97d78c27a8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 60256642160a44a2bab77b97d78c27a8 already in progress: creating tablet
I20260925 13:57:19.283025 22211 ts_tablet_manager.cc:1434] T f481b714a5f34fff8f74ba30bdd5dd35 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260925 13:57:19.288158 22416 tablet_replica.cc:333] T 72fc427f45934cba91e52ad84275c461 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:19.288407 22293 consensus_queue.cc:260] T f481b714a5f34fff8f74ba30bdd5dd35 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.288851 22416 raft_consensus.cc:2243] T 72fc427f45934cba91e52ad84275c461 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:19.289726 22416 raft_consensus.cc:2272] T 72fc427f45934cba91e52ad84275c461 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:19.291458 22212 raft_consensus.cc:359] T 72fc427f45934cba91e52ad84275c461 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.292073 22212 raft_consensus.cc:385] T 72fc427f45934cba91e52ad84275c461 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:19.292326 22212 raft_consensus.cc:740] T 72fc427f45934cba91e52ad84275c461 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:19.293975 22211 tablet_bootstrap.cc:492] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:19.299017 22416 ts_tablet_manager.cc:1916] T 72fc427f45934cba91e52ad84275c461 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:19.299237 22211 tablet_bootstrap.cc:654] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:19.300782 22293 ts_tablet_manager.cc:1434] T f481b714a5f34fff8f74ba30bdd5dd35 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.047s user 0.007s sys 0.000s
I20260925 13:57:19.301640 22293 tablet_bootstrap.cc:492] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:19.293366 22212 consensus_queue.cc:260] T 72fc427f45934cba91e52ad84275c461 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.304379 22212 ts_tablet_manager.cc:1434] T 72fc427f45934cba91e52ad84275c461 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.035s user 0.006s sys 0.001s
I20260925 13:57:19.305300 22212 tablet_bootstrap.cc:492] T f5bd96a2bdb842e9b3bc1f706786d6b1 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:19.310272 22212 tablet_bootstrap.cc:654] T f5bd96a2bdb842e9b3bc1f706786d6b1 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:19.326762 22214 tablet_replica.cc:333] T 7efa9e6cb797487bb11bf47ad3c320ba P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:19.327476 22214 raft_consensus.cc:2243] T 7efa9e6cb797487bb11bf47ad3c320ba P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:19.327907 22214 raft_consensus.cc:2272] T 7efa9e6cb797487bb11bf47ad3c320ba P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:19.329340 22212 tablet_bootstrap.cc:492] T f5bd96a2bdb842e9b3bc1f706786d6b1 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:19.329730 22212 ts_tablet_manager.cc:1403] T f5bd96a2bdb842e9b3bc1f706786d6b1 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.025s user 0.010s sys 0.001s
I20260925 13:57:19.330863 22415 ts_tablet_manager.cc:1929] T 7efa9e6cb797487bb11bf47ad3c320ba P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:19.331238 22415 log.cc:1199] T 7efa9e6cb797487bb11bf47ad3c320ba P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/7efa9e6cb797487bb11bf47ad3c320ba
I20260925 13:57:19.337030 22415 ts_tablet_manager.cc:1950] T 7efa9e6cb797487bb11bf47ad3c320ba P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:19.338757 22214 ts_tablet_manager.cc:1916] T 7efa9e6cb797487bb11bf47ad3c320ba P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:19.342351 22416 ts_tablet_manager.cc:1929] T 72fc427f45934cba91e52ad84275c461 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:19.343971 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet 7efa9e6cb797487bb11bf47ad3c320ba (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]) successfully deleted
I20260925 13:57:19.344465 22212 raft_consensus.cc:359] T f5bd96a2bdb842e9b3bc1f706786d6b1 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.345116 22212 raft_consensus.cc:385] T f5bd96a2bdb842e9b3bc1f706786d6b1 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:19.345362 22212 raft_consensus.cc:740] T f5bd96a2bdb842e9b3bc1f706786d6b1 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:19.345252 22416 log.cc:1199] T 72fc427f45934cba91e52ad84275c461 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/72fc427f45934cba91e52ad84275c461
I20260925 13:57:19.347121 22293 tablet_bootstrap.cc:654] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:19.347540 22416 ts_tablet_manager.cc:1950] T 72fc427f45934cba91e52ad84275c461 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:19.345963 22212 consensus_queue.cc:260] T f5bd96a2bdb842e9b3bc1f706786d6b1 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.353940 22212 ts_tablet_manager.cc:1434] T f5bd96a2bdb842e9b3bc1f706786d6b1 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.024s user 0.005s sys 0.000s
I20260925 13:57:19.354748 22212 tablet_bootstrap.cc:492] T 60256642160a44a2bab77b97d78c27a8 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:19.358175 22211 tablet_bootstrap.cc:492] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:19.358554 22211 ts_tablet_manager.cc:1403] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.065s user 0.012s sys 0.004s
I20260925 13:57:19.359751 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet 72fc427f45934cba91e52ad84275c461 (table default.table_e0eb6c1d49224d1cb9b4f6d06f0f7a29 [id=420cd09cc1184f6e9fb38b2cbc3f6fce]) successfully deleted
I20260925 13:57:19.360674 22416 tablet_replica.cc:333] T 7efa9e6cb797487bb11bf47ad3c320ba P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:19.361801 22211 raft_consensus.cc:359] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.366773 22211 raft_consensus.cc:385] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:19.367089 22211 raft_consensus.cc:740] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:19.367498 22464 raft_consensus.cc:493] T f5bd96a2bdb842e9b3bc1f706786d6b1 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:19.367915 22464 raft_consensus.cc:515] T f5bd96a2bdb842e9b3bc1f706786d6b1 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.367949 22211 consensus_queue.cc:260] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.369473 22211 ts_tablet_manager.cc:1434] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.011s user 0.005s sys 0.001s
I20260925 13:57:19.369800 22464 leader_election.cc:290] T f5bd96a2bdb842e9b3bc1f706786d6b1 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881)
I20260925 13:57:19.375732 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bd96a2bdb842e9b3bc1f706786d6b1" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2" is_pre_election: true
I20260925 13:57:19.377144 22212 tablet_bootstrap.cc:654] T 60256642160a44a2bab77b97d78c27a8 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
W20260925 13:57:19.377605 22040 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:57:19.373500 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bd96a2bdb842e9b3bc1f706786d6b1" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:19.378259 21960 raft_consensus.cc:2468] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee02f77594864349982492862d668a36 in term 0.
I20260925 13:57:19.374845 22416 raft_consensus.cc:2243] T 7efa9e6cb797487bb11bf47ad3c320ba P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:19.379555 22416 pending_rounds.cc:70] T 7efa9e6cb797487bb11bf47ad3c320ba P 145e520e0a5a445d943a53cf8c27efee: Trying to abort 1 pending ops.
I20260925 13:57:19.379590 22043 leader_election.cc:304] T f5bd96a2bdb842e9b3bc1f706786d6b1 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36; no voters: 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:19.379837 22416 pending_rounds.cc:77] T 7efa9e6cb797487bb11bf47ad3c320ba P 145e520e0a5a445d943a53cf8c27efee: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251642158153728 op_type: NO_OP noop_request { }
I20260925 13:57:19.380263 22416 raft_consensus.cc:2889] T 7efa9e6cb797487bb11bf47ad3c320ba P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:57:19.380256 22464 raft_consensus.cc:2804] T f5bd96a2bdb842e9b3bc1f706786d6b1 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:19.380556 22464 raft_consensus.cc:493] T f5bd96a2bdb842e9b3bc1f706786d6b1 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:19.380823 22464 raft_consensus.cc:3060] T f5bd96a2bdb842e9b3bc1f706786d6b1 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:19.381265 22416 raft_consensus.cc:2272] T 7efa9e6cb797487bb11bf47ad3c320ba P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:19.383453 21586 catalog_manager.cc:2770] deleted HMS catalog entry for table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]
I20260925 13:57:19.386567 22464 raft_consensus.cc:515] T f5bd96a2bdb842e9b3bc1f706786d6b1 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.388197 22293 tablet_bootstrap.cc:492] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:19.388645 22293 ts_tablet_manager.cc:1403] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.087s user 0.006s sys 0.004s
I20260925 13:57:19.389248 21581 catalog_manager.cc:5838] T 761ede873f5648ccb5216a4c0a8afa2e P 145e520e0a5a445d943a53cf8c27efee reported cstate change: term changed from 0 to 1, leader changed from <none> to 145e520e0a5a445d943a53cf8c27efee (127.18.182.194). New cstate: current_term: 1 leader_uuid: "145e520e0a5a445d943a53cf8c27efee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: UNKNOWN } } }
I20260925 13:57:19.390498 21581 catalog_manager.cc:5938] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291) does not have the latest schema for tablet 761ede873f5648ccb5216a4c0a8afa2e (table default.table_d6100b0f2b294ab099c179a7681cb4ed [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 2 got 0
I20260925 13:57:19.388938 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bd96a2bdb842e9b3bc1f706786d6b1" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
I20260925 13:57:19.391093 21960 raft_consensus.cc:3060] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:19.391144 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bd96a2bdb842e9b3bc1f706786d6b1" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2"
W20260925 13:57:19.392336 22040 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:57:19.393723 22464 leader_election.cc:290] T f5bd96a2bdb842e9b3bc1f706786d6b1 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 election: Requested vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881)
I20260925 13:57:19.397017 21960 raft_consensus.cc:2468] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee02f77594864349982492862d668a36 in term 1.
I20260925 13:57:19.398002 22043 leader_election.cc:304] T f5bd96a2bdb842e9b3bc1f706786d6b1 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36; no voters: 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:19.398725 22464 raft_consensus.cc:2804] T f5bd96a2bdb842e9b3bc1f706786d6b1 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:19.399122 22464 raft_consensus.cc:697] T f5bd96a2bdb842e9b3bc1f706786d6b1 P ee02f77594864349982492862d668a36 [term 1 LEADER]: Becoming Leader. State: Replica: ee02f77594864349982492862d668a36, State: Running, Role: LEADER
I20260925 13:57:19.380895 22211 tablet_bootstrap.cc:492] T f66354b75eb8446396a27c3d563191cc P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:19.399838 22464 consensus_queue.cc:237] T f5bd96a2bdb842e9b3bc1f706786d6b1 P ee02f77594864349982492862d668a36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.402113 22293 raft_consensus.cc:359] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.404191 22293 raft_consensus.cc:385] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:19.404564 22293 raft_consensus.cc:740] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:19.405293 22293 consensus_queue.cc:260] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.411125 22293 ts_tablet_manager.cc:1434] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.022s user 0.004s sys 0.001s
I20260925 13:57:19.411649 22211 tablet_bootstrap.cc:654] T f66354b75eb8446396a27c3d563191cc P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:19.413594 21938 tablet_service.cc:1558] Processing DeleteTablet for tablet 72fc427f45934cba91e52ad84275c461 with delete_type TABLET_DATA_DELETED (replaced by tablet 24dd39b302344f8d8f5166e9d641d13a) from {username='slave'} at 127.0.0.1:55588
W20260925 13:57:19.419970 21581 catalog_manager.cc:5671] Ignoring report from unknown tablet 60256642160a44a2bab77b97d78c27a8
W20260925 13:57:19.420382 21581 catalog_manager.cc:5671] Ignoring report from unknown tablet 7efa9e6cb797487bb11bf47ad3c320ba
I20260925 13:57:19.425087 22416 ts_tablet_manager.cc:1916] T 7efa9e6cb797487bb11bf47ad3c320ba P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:19.428500 21579 catalog_manager.cc:5838] T f3b0e015d5dd4a668b6a04b49f628070 P ee02f77594864349982492862d668a36 reported cstate change: term changed from 0 to 1, leader changed from <none> to ee02f77594864349982492862d668a36 (127.18.182.195). New cstate: current_term: 1 leader_uuid: "ee02f77594864349982492862d668a36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: HEALTHY } } }
I20260925 13:57:19.429638 22212 tablet_bootstrap.cc:492] T 60256642160a44a2bab77b97d78c27a8 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:19.430064 22212 ts_tablet_manager.cc:1403] T 60256642160a44a2bab77b97d78c27a8 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.075s user 0.004s sys 0.004s
I20260925 13:57:19.431797 22214 ts_tablet_manager.cc:1929] T 7efa9e6cb797487bb11bf47ad3c320ba P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:19.432154 22214 log.cc:1199] T 7efa9e6cb797487bb11bf47ad3c320ba P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/7efa9e6cb797487bb11bf47ad3c320ba
I20260925 13:57:19.432766 22214 ts_tablet_manager.cc:1950] T 7efa9e6cb797487bb11bf47ad3c320ba P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:19.434893 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet 7efa9e6cb797487bb11bf47ad3c320ba (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]) successfully deleted
I20260925 13:57:19.439662 21585 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35994:
table { table_name: "default.Table_61d48f9061c741fc8aa556e3edd984fb" } new_table_name: "default.Table_683c0db2850941be84798e9dca3fe490" modify_external_catalogs: true
I20260925 13:57:19.443343 22416 ts_tablet_manager.cc:1929] T 7efa9e6cb797487bb11bf47ad3c320ba P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:19.443663 22416 log.cc:1199] T 7efa9e6cb797487bb11bf47ad3c320ba P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/7efa9e6cb797487bb11bf47ad3c320ba
I20260925 13:57:19.444295 22416 ts_tablet_manager.cc:1950] T 7efa9e6cb797487bb11bf47ad3c320ba P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:19.446545 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet 7efa9e6cb797487bb11bf47ad3c320ba (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]) successfully deleted
I20260925 13:57:19.450814 22464 raft_consensus.cc:493] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:19.451207 22464 raft_consensus.cc:515] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:19.454692 22040 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
I20260925 13:57:19.455344 22464 leader_election.cc:290] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:19.458333 22212 raft_consensus.cc:359] T 60256642160a44a2bab77b97d78c27a8 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:19.458928 22043 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
I20260925 13:57:19.459399 22043 leader_election.cc:304] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:19.460556 22464 raft_consensus.cc:2749] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:57:19.461650 21575 catalog_manager.cc:5030] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): delete failed for tablet 72fc427f45934cba91e52ad84275c461 because the tablet was not found. No further retry: Not found: Tablet not found: 72fc427f45934cba91e52ad84275c461
I20260925 13:57:19.352681 22415 tablet_replica.cc:333] T 72fc427f45934cba91e52ad84275c461 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:19.461126 21579 catalog_manager.cc:5838] T f5bd96a2bdb842e9b3bc1f706786d6b1 P ee02f77594864349982492862d668a36 reported cstate change: term changed from 0 to 1, leader changed from <none> to ee02f77594864349982492862d668a36 (127.18.182.195). New cstate: current_term: 1 leader_uuid: "ee02f77594864349982492862d668a36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: HEALTHY } } }
I20260925 13:57:19.463863 22415 raft_consensus.cc:2243] T 72fc427f45934cba91e52ad84275c461 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:19.464285 22415 raft_consensus.cc:2272] T 72fc427f45934cba91e52ad84275c461 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:19.458956 22212 raft_consensus.cc:385] T 60256642160a44a2bab77b97d78c27a8 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:19.467664 22212 raft_consensus.cc:740] T 60256642160a44a2bab77b97d78c27a8 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:19.467743 21775 tablet_service.cc:1511] Processing CreateTablet for tablet 76d4d7d706824892b075ab1aa4fbaf85 (DEFAULT_TABLE table=default.table_8008d16a92cf4c0dafb258a1122bffb5 [id=a22f86f8c2404aa69bdeb9883471453c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:19.469062 22293 tablet_bootstrap.cc:492] T 60256642160a44a2bab77b97d78c27a8 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:19.472854 21775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76d4d7d706824892b075ab1aa4fbaf85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:19.475250 21938 tablet_service.cc:1511] Processing CreateTablet for tablet 76d4d7d706824892b075ab1aa4fbaf85 (DEFAULT_TABLE table=default.table_8008d16a92cf4c0dafb258a1122bffb5 [id=a22f86f8c2404aa69bdeb9883471453c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:19.476109 21938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76d4d7d706824892b075ab1aa4fbaf85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:19.479404 22211 tablet_bootstrap.cc:492] T f66354b75eb8446396a27c3d563191cc P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:19.479789 22211 ts_tablet_manager.cc:1403] T f66354b75eb8446396a27c3d563191cc P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.103s user 0.013s sys 0.005s
I20260925 13:57:19.480437 22212 consensus_queue.cc:260] T 60256642160a44a2bab77b97d78c27a8 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.479727 22214 tablet_replica.cc:333] T abadb0b2c7d34451b7b23489e6d866c6 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:19.481671 22214 raft_consensus.cc:2243] T abadb0b2c7d34451b7b23489e6d866c6 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:19.481997 22211 raft_consensus.cc:359] T f66354b75eb8446396a27c3d563191cc P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.482578 22211 raft_consensus.cc:385] T f66354b75eb8446396a27c3d563191cc P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:19.482815 22211 raft_consensus.cc:740] T f66354b75eb8446396a27c3d563191cc P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:19.483390 22211 consensus_queue.cc:260] T f66354b75eb8446396a27c3d563191cc P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.486639 22212 ts_tablet_manager.cc:1434] T 60256642160a44a2bab77b97d78c27a8 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.056s user 0.006s sys 0.001s
I20260925 13:57:19.488279 22214 raft_consensus.cc:2272] T abadb0b2c7d34451b7b23489e6d866c6 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:19.490228 22214 ts_tablet_manager.cc:1916] T abadb0b2c7d34451b7b23489e6d866c6 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:19.473862 22293 tablet_bootstrap.cc:654] T 60256642160a44a2bab77b97d78c27a8 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:19.493993 22212 tablet_bootstrap.cc:492] T f481b714a5f34fff8f74ba30bdd5dd35 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:19.495826 22211 ts_tablet_manager.cc:1434] T f66354b75eb8446396a27c3d563191cc P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.016s user 0.006s sys 0.000s
I20260925 13:57:19.512074 22293 tablet_bootstrap.cc:492] T 60256642160a44a2bab77b97d78c27a8 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:19.512456 22293 ts_tablet_manager.cc:1403] T 60256642160a44a2bab77b97d78c27a8 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.044s user 0.008s sys 0.003s
I20260925 13:57:19.517263 22415 ts_tablet_manager.cc:1916] T 72fc427f45934cba91e52ad84275c461 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:19.520388 22212 tablet_bootstrap.cc:654] T f481b714a5f34fff8f74ba30bdd5dd35 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:19.522926 21777 tablet_service.cc:1511] Processing CreateTablet for tablet 24dd39b302344f8d8f5166e9d641d13a (DEFAULT_TABLE table=default.table_e0eb6c1d49224d1cb9b4f6d06f0f7a29 [id=420cd09cc1184f6e9fb38b2cbc3f6fce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:19.520159 22293 raft_consensus.cc:359] T 60256642160a44a2bab77b97d78c27a8 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.523938 21777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24dd39b302344f8d8f5166e9d641d13a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:19.522141 22087 tablet_service.cc:1511] Processing CreateTablet for tablet 76d4d7d706824892b075ab1aa4fbaf85 (DEFAULT_TABLE table=default.table_8008d16a92cf4c0dafb258a1122bffb5 [id=a22f86f8c2404aa69bdeb9883471453c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:19.524952 22087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76d4d7d706824892b075ab1aa4fbaf85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:19.526535 21940 tablet_service.cc:1511] Processing CreateTablet for tablet 24dd39b302344f8d8f5166e9d641d13a (DEFAULT_TABLE table=default.table_e0eb6c1d49224d1cb9b4f6d06f0f7a29 [id=420cd09cc1184f6e9fb38b2cbc3f6fce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:19.528584 22085 tablet_service.cc:1511] Processing CreateTablet for tablet 24dd39b302344f8d8f5166e9d641d13a (DEFAULT_TABLE table=default.table_e0eb6c1d49224d1cb9b4f6d06f0f7a29 [id=420cd09cc1184f6e9fb38b2cbc3f6fce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:19.530149 21579 catalog_manager.cc:5938] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291) does not have the latest schema for tablet f66354b75eb8446396a27c3d563191cc (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]). Expected version 1 got 0
I20260925 13:57:19.530519 22085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24dd39b302344f8d8f5166e9d641d13a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:19.523968 22293 raft_consensus.cc:385] T 60256642160a44a2bab77b97d78c27a8 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:19.531561 22293 raft_consensus.cc:740] T 60256642160a44a2bab77b97d78c27a8 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:19.532817 22293 consensus_queue.cc:260] T 60256642160a44a2bab77b97d78c27a8 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.541258 22293 ts_tablet_manager.cc:1434] T 60256642160a44a2bab77b97d78c27a8 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.028s user 0.003s sys 0.001s
I20260925 13:57:19.546669 22293 tablet_bootstrap.cc:492] T f66354b75eb8446396a27c3d563191cc P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
W20260925 13:57:19.546954 21579 catalog_manager.cc:6038] Failed to send AlterTable RPC for tablet f66354b75eb8446396a27c3d563191cc (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet f66354b75eb8446396a27c3d563191cc consensus state has no leader
I20260925 13:57:19.550614 22464 raft_consensus.cc:493] T f68ccd8a7f4a4e518c2584e579de0c87 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:19.551074 22464 raft_consensus.cc:515] T f68ccd8a7f4a4e518c2584e579de0c87 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:19.552891 21581 catalog_manager.cc:5671] Ignoring report from unknown tablet 60256642160a44a2bab77b97d78c27a8
I20260925 13:57:19.553041 22464 leader_election.cc:290] T f68ccd8a7f4a4e518c2584e579de0c87 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881)
I20260925 13:57:19.554561 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68ccd8a7f4a4e518c2584e579de0c87" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2" is_pre_election: true
I20260925 13:57:19.560968 22212 tablet_bootstrap.cc:492] T f481b714a5f34fff8f74ba30bdd5dd35 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:19.561375 22212 ts_tablet_manager.cc:1403] T f481b714a5f34fff8f74ba30bdd5dd35 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.074s user 0.000s sys 0.008s
I20260925 13:57:19.561707 22293 tablet_bootstrap.cc:654] T f66354b75eb8446396a27c3d563191cc P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
W20260925 13:57:19.565045 21579 catalog_manager.cc:5671] Ignoring report from unknown tablet 60256642160a44a2bab77b97d78c27a8
I20260925 13:57:19.568594 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68ccd8a7f4a4e518c2584e579de0c87" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:19.569494 21960 raft_consensus.cc:2393] T f68ccd8a7f4a4e518c2584e579de0c87 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ee02f77594864349982492862d668a36 in current term 1: Already voted for candidate 5e2731975dbf433aade4a859df9699e2 in this term.
I20260925 13:57:19.570778 22043 leader_election.cc:304] T f68ccd8a7f4a4e518c2584e579de0c87 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:19.571082 22214 ts_tablet_manager.cc:1929] T abadb0b2c7d34451b7b23489e6d866c6 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:19.571676 22212 raft_consensus.cc:359] T f481b714a5f34fff8f74ba30bdd5dd35 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.572284 22212 raft_consensus.cc:385] T f481b714a5f34fff8f74ba30bdd5dd35 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:19.572521 22212 raft_consensus.cc:740] T f481b714a5f34fff8f74ba30bdd5dd35 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:19.573216 21940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24dd39b302344f8d8f5166e9d641d13a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:19.575104 21581 catalog_manager.cc:5938] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291) does not have the latest schema for tablet f66354b75eb8446396a27c3d563191cc (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]). Expected version 1 got 0
I20260925 13:57:19.575829 22464 raft_consensus.cc:3060] T f68ccd8a7f4a4e518c2584e579de0c87 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:19.579612 22212 consensus_queue.cc:260] T f481b714a5f34fff8f74ba30bdd5dd35 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.582147 21579 master_service.cc:960] ReplaceTablet: received request to replace tablet 3a8c53e379a9407c920d91b05ebd2afc from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:19.584589 22212 ts_tablet_manager.cc:1434] T f481b714a5f34fff8f74ba30bdd5dd35 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.023s user 0.006s sys 0.001s
I20260925 13:57:19.585000 22214 log.cc:1199] T abadb0b2c7d34451b7b23489e6d866c6 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/abadb0b2c7d34451b7b23489e6d866c6
I20260925 13:57:19.586139 22214 ts_tablet_manager.cc:1950] T abadb0b2c7d34451b7b23489e6d866c6 P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:19.588452 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet abadb0b2c7d34451b7b23489e6d866c6 (table default.table_8008d16a92cf4c0dafb258a1122bffb5 [id=a22f86f8c2404aa69bdeb9883471453c]) successfully deleted
I20260925 13:57:19.590252 22464 raft_consensus.cc:2749] T f68ccd8a7f4a4e518c2584e579de0c87 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:57:19.594172 22212 tablet_bootstrap.cc:492] T f66354b75eb8446396a27c3d563191cc P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:19.594767 22415 ts_tablet_manager.cc:1929] T 72fc427f45934cba91e52ad84275c461 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:19.595098 22415 log.cc:1199] T 72fc427f45934cba91e52ad84275c461 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/72fc427f45934cba91e52ad84275c461
I20260925 13:57:19.599023 22211 tablet_bootstrap.cc:492] T 24dd39b302344f8d8f5166e9d641d13a P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:19.604797 22211 tablet_bootstrap.cc:654] T 24dd39b302344f8d8f5166e9d641d13a P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:19.615329 22214 tablet_replica.cc:333] T 60256642160a44a2bab77b97d78c27a8 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:19.616074 22214 raft_consensus.cc:2243] T 60256642160a44a2bab77b97d78c27a8 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:19.616492 22214 raft_consensus.cc:2272] T 60256642160a44a2bab77b97d78c27a8 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:19.617439 22415 ts_tablet_manager.cc:1950] T 72fc427f45934cba91e52ad84275c461 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:19.580294 22293 tablet_bootstrap.cc:492] T f66354b75eb8446396a27c3d563191cc P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:19.619980 22293 ts_tablet_manager.cc:1403] T f66354b75eb8446396a27c3d563191cc P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.073s user 0.006s sys 0.005s
I20260925 13:57:19.621268 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet 72fc427f45934cba91e52ad84275c461 (table default.table_e0eb6c1d49224d1cb9b4f6d06f0f7a29 [id=420cd09cc1184f6e9fb38b2cbc3f6fce]) successfully deleted
I20260925 13:57:19.622391 22293 raft_consensus.cc:359] T f66354b75eb8446396a27c3d563191cc P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.622980 22293 raft_consensus.cc:385] T f66354b75eb8446396a27c3d563191cc P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:19.626152 22211 tablet_bootstrap.cc:492] T 24dd39b302344f8d8f5166e9d641d13a P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:19.626595 22211 ts_tablet_manager.cc:1403] T 24dd39b302344f8d8f5166e9d641d13a P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.028s user 0.009s sys 0.004s
I20260925 13:57:19.630827 21579 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet 3a8c53e379a9407c920d91b05ebd2afc
I20260925 13:57:19.632476 21579 catalog_manager.cc:6934] ReplaceTablet: tablet 3a8c53e379a9407c920d91b05ebd2afc deleted and replaced by tablet 56a39663479d40be8bbd1dfd4c91040c
I20260925 13:57:19.623214 22293 raft_consensus.cc:740] T f66354b75eb8446396a27c3d563191cc P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:19.637086 21579 master_service.cc:960] ReplaceTablet: received request to replace tablet f66354b75eb8446396a27c3d563191cc from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:19.637790 21940 tablet_service.cc:1558] Processing DeleteTablet for tablet 3a8c53e379a9407c920d91b05ebd2afc with delete_type TABLET_DATA_DELETED (replaced by tablet 56a39663479d40be8bbd1dfd4c91040c) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:19.638732 22416 tablet_replica.cc:333] T 3a8c53e379a9407c920d91b05ebd2afc P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:19.639647 22416 raft_consensus.cc:2243] T 3a8c53e379a9407c920d91b05ebd2afc P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:19.640216 22416 raft_consensus.cc:2272] T 3a8c53e379a9407c920d91b05ebd2afc P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:19.642419 22416 ts_tablet_manager.cc:1916] T 3a8c53e379a9407c920d91b05ebd2afc P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:19.656010 22211 raft_consensus.cc:359] T 24dd39b302344f8d8f5166e9d641d13a P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.656760 22211 raft_consensus.cc:385] T 24dd39b302344f8d8f5166e9d641d13a P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:19.636278 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet 3a8c53e379a9407c920d91b05ebd2afc with delete_type TABLET_DATA_DELETED (replaced by tablet 56a39663479d40be8bbd1dfd4c91040c) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:19.657106 22211 raft_consensus.cc:740] T 24dd39b302344f8d8f5166e9d641d13a P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:19.651671 22293 consensus_queue.cc:260] T f66354b75eb8446396a27c3d563191cc P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.657938 22211 consensus_queue.cc:260] T 24dd39b302344f8d8f5166e9d641d13a P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.658419 22214 ts_tablet_manager.cc:1916] T 60256642160a44a2bab77b97d78c27a8 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:19.658823 21777 tablet_service.cc:1558] Processing DeleteTablet for tablet 3a8c53e379a9407c920d91b05ebd2afc with delete_type TABLET_DATA_DELETED (replaced by tablet 56a39663479d40be8bbd1dfd4c91040c) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:19.656020 22212 tablet_bootstrap.cc:654] T f66354b75eb8446396a27c3d563191cc P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:19.665643 22211 ts_tablet_manager.cc:1434] T 24dd39b302344f8d8f5166e9d641d13a P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.039s user 0.006s sys 0.000s
I20260925 13:57:19.667122 22211 tablet_bootstrap.cc:492] T 76d4d7d706824892b075ab1aa4fbaf85 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:19.671996 22415 tablet_replica.cc:333] T 3a8c53e379a9407c920d91b05ebd2afc P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:19.672667 22415 raft_consensus.cc:2243] T 3a8c53e379a9407c920d91b05ebd2afc P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:19.675918 22415 raft_consensus.cc:2272] T 3a8c53e379a9407c920d91b05ebd2afc P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:19.675999 22293 ts_tablet_manager.cc:1434] T f66354b75eb8446396a27c3d563191cc P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.056s user 0.006s sys 0.000s
I20260925 13:57:19.677583 22415 ts_tablet_manager.cc:1916] T 3a8c53e379a9407c920d91b05ebd2afc P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:19.680998 22211 tablet_bootstrap.cc:654] T 76d4d7d706824892b075ab1aa4fbaf85 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:19.682333 22517 raft_consensus.cc:493] T f481b714a5f34fff8f74ba30bdd5dd35 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:19.682808 22517 raft_consensus.cc:515] T f481b714a5f34fff8f74ba30bdd5dd35 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.684862 22517 leader_election.cc:290] T f481b714a5f34fff8f74ba30bdd5dd35 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:19.686362 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f481b714a5f34fff8f74ba30bdd5dd35" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2" is_pre_election: true
I20260925 13:57:19.686908 21797 raft_consensus.cc:2468] T f481b714a5f34fff8f74ba30bdd5dd35 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 0.
I20260925 13:57:19.687507 22212 tablet_bootstrap.cc:492] T f66354b75eb8446396a27c3d563191cc P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:19.687856 22212 ts_tablet_manager.cc:1403] T f66354b75eb8446396a27c3d563191cc P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.094s user 0.007s sys 0.003s
I20260925 13:57:19.688043 21893 leader_election.cc:304] T f481b714a5f34fff8f74ba30bdd5dd35 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:19.688802 22517 raft_consensus.cc:2804] T f481b714a5f34fff8f74ba30bdd5dd35 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:19.689109 22517 raft_consensus.cc:493] T f481b714a5f34fff8f74ba30bdd5dd35 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:19.682356 22416 ts_tablet_manager.cc:1929] T 3a8c53e379a9407c920d91b05ebd2afc P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:19.689411 22517 raft_consensus.cc:3060] T f481b714a5f34fff8f74ba30bdd5dd35 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:19.689313 22106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f481b714a5f34fff8f74ba30bdd5dd35" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:19.689618 22416 log.cc:1199] T 3a8c53e379a9407c920d91b05ebd2afc P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/3a8c53e379a9407c920d91b05ebd2afc
I20260925 13:57:19.689900 22106 raft_consensus.cc:2468] T f481b714a5f34fff8f74ba30bdd5dd35 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 0.
I20260925 13:57:19.690526 22416 ts_tablet_manager.cc:1950] T 3a8c53e379a9407c920d91b05ebd2afc P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:19.692704 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet 3a8c53e379a9407c920d91b05ebd2afc (table default.table_b17685eaeada404790103bd89711435e [id=df6afcfb1e2c463493b14228e19ade26]) successfully deleted
I20260925 13:57:19.695899 22517 raft_consensus.cc:515] T f481b714a5f34fff8f74ba30bdd5dd35 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.696318 21579 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet f66354b75eb8446396a27c3d563191cc
I20260925 13:57:19.698223 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f481b714a5f34fff8f74ba30bdd5dd35" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2"
I20260925 13:57:19.698673 21940 tablet_service.cc:1558] Processing DeleteTablet for tablet f66354b75eb8446396a27c3d563191cc with delete_type TABLET_DATA_DELETED (replaced by tablet 9745537e91d645fe9874373fe6237a9b) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:19.698736 21797 raft_consensus.cc:3060] T f481b714a5f34fff8f74ba30bdd5dd35 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:19.700299 21777 tablet_service.cc:1558] Processing DeleteTablet for tablet f66354b75eb8446396a27c3d563191cc with delete_type TABLET_DATA_DELETED (replaced by tablet 9745537e91d645fe9874373fe6237a9b) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:19.701858 22416 tablet_replica.cc:333] T f66354b75eb8446396a27c3d563191cc P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:19.702612 22416 raft_consensus.cc:2243] T f66354b75eb8446396a27c3d563191cc P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:19.703505 22416 raft_consensus.cc:2272] T f66354b75eb8446396a27c3d563191cc P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:19.706765 21797 raft_consensus.cc:2468] T f481b714a5f34fff8f74ba30bdd5dd35 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 1.
I20260925 13:57:19.708226 21893 leader_election.cc:304] T f481b714a5f34fff8f74ba30bdd5dd35 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:19.709111 22528 raft_consensus.cc:2804] T f481b714a5f34fff8f74ba30bdd5dd35 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:19.709502 21579 catalog_manager.cc:6934] ReplaceTablet: tablet f66354b75eb8446396a27c3d563191cc deleted and replaced by tablet 9745537e91d645fe9874373fe6237a9b
I20260925 13:57:19.690104 22212 raft_consensus.cc:359] T f66354b75eb8446396a27c3d563191cc P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.720216 22212 raft_consensus.cc:385] T f66354b75eb8446396a27c3d563191cc P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:19.720494 22212 raft_consensus.cc:740] T f66354b75eb8446396a27c3d563191cc P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:19.720975 22528 raft_consensus.cc:697] T f481b714a5f34fff8f74ba30bdd5dd35 P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: Becoming Leader. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Running, Role: LEADER
I20260925 13:57:19.721729 22528 consensus_queue.cc:237] T f481b714a5f34fff8f74ba30bdd5dd35 P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.725222 22293 tablet_bootstrap.cc:492] T 76d4d7d706824892b075ab1aa4fbaf85 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:19.719853 22087 tablet_service.cc:1558] Processing DeleteTablet for tablet f66354b75eb8446396a27c3d563191cc with delete_type TABLET_DATA_DELETED (replaced by tablet 9745537e91d645fe9874373fe6237a9b) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:19.732684 22106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f481b714a5f34fff8f74ba30bdd5dd35" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:19.733203 22106 raft_consensus.cc:3060] T f481b714a5f34fff8f74ba30bdd5dd35 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:19.721158 22212 consensus_queue.cc:260] T f66354b75eb8446396a27c3d563191cc P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.736583 22464 raft_consensus.cc:493] T f66354b75eb8446396a27c3d563191cc P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:19.737030 22416 ts_tablet_manager.cc:1916] T f66354b75eb8446396a27c3d563191cc P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:19.737038 22464 raft_consensus.cc:515] T f66354b75eb8446396a27c3d563191cc P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.738718 21777 tablet_service.cc:1511] Processing CreateTablet for tablet 56a39663479d40be8bbd1dfd4c91040c (DEFAULT_TABLE table=default.table_b17685eaeada404790103bd89711435e [id=df6afcfb1e2c463493b14228e19ade26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:19.739691 21777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56a39663479d40be8bbd1dfd4c91040c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:19.739881 22106 raft_consensus.cc:2468] T f481b714a5f34fff8f74ba30bdd5dd35 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 1.
I20260925 13:57:19.741396 22464 leader_election.cc:290] T f66354b75eb8446396a27c3d563191cc P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:19.741823 22517 leader_election.cc:290] T f481b714a5f34fff8f74ba30bdd5dd35 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Requested vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:19.742328 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66354b75eb8446396a27c3d563191cc" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
W20260925 13:57:19.743479 22043 leader_election.cc:343] T f66354b75eb8446396a27c3d563191cc P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Illegal state: Tablet not RUNNING: STOPPED
I20260925 13:57:19.746670 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66354b75eb8446396a27c3d563191cc" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2" is_pre_election: true
I20260925 13:57:19.751575 21797 raft_consensus.cc:2468] T f66354b75eb8446396a27c3d563191cc P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee02f77594864349982492862d668a36 in term 0.
I20260925 13:57:19.752584 22040 leader_election.cc:304] T f66354b75eb8446396a27c3d563191cc P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2, ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee
I20260925 13:57:19.755801 22464 raft_consensus.cc:2804] T f66354b75eb8446396a27c3d563191cc P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:19.756089 22464 raft_consensus.cc:493] T f66354b75eb8446396a27c3d563191cc P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:19.756345 22464 raft_consensus.cc:3060] T f66354b75eb8446396a27c3d563191cc P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:19.756428 21938 tablet_service.cc:1511] Processing CreateTablet for tablet 56a39663479d40be8bbd1dfd4c91040c (DEFAULT_TABLE table=default.table_b17685eaeada404790103bd89711435e [id=df6afcfb1e2c463493b14228e19ade26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:19.757403 21938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56a39663479d40be8bbd1dfd4c91040c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:19.769380 22087 tablet_service.cc:1511] Processing CreateTablet for tablet 56a39663479d40be8bbd1dfd4c91040c (DEFAULT_TABLE table=default.table_b17685eaeada404790103bd89711435e [id=df6afcfb1e2c463493b14228e19ade26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:19.770331 22087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56a39663479d40be8bbd1dfd4c91040c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:19.770157 21579 catalog_manager.cc:5838] T f481b714a5f34fff8f74ba30bdd5dd35 P 145e520e0a5a445d943a53cf8c27efee reported cstate change: term changed from 0 to 1, leader changed from <none> to 145e520e0a5a445d943a53cf8c27efee (127.18.182.194). New cstate: current_term: 1 leader_uuid: "145e520e0a5a445d943a53cf8c27efee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: UNKNOWN } } }
I20260925 13:57:19.773169 22211 tablet_bootstrap.cc:492] T 76d4d7d706824892b075ab1aa4fbaf85 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:19.773653 22211 ts_tablet_manager.cc:1403] T 76d4d7d706824892b075ab1aa4fbaf85 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.107s user 0.009s sys 0.010s
I20260925 13:57:19.773545 22464 raft_consensus.cc:515] T f66354b75eb8446396a27c3d563191cc P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.776059 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66354b75eb8446396a27c3d563191cc" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2"
I20260925 13:57:19.776095 22211 raft_consensus.cc:359] T 76d4d7d706824892b075ab1aa4fbaf85 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.776571 21797 raft_consensus.cc:3060] T f66354b75eb8446396a27c3d563191cc P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:19.776787 22211 raft_consensus.cc:385] T 76d4d7d706824892b075ab1aa4fbaf85 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:19.777122 22211 raft_consensus.cc:740] T 76d4d7d706824892b075ab1aa4fbaf85 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:19.778479 22211 consensus_queue.cc:260] T 76d4d7d706824892b075ab1aa4fbaf85 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.780351 22211 ts_tablet_manager.cc:1434] T 76d4d7d706824892b075ab1aa4fbaf85 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260925 13:57:19.782771 21797 raft_consensus.cc:2468] T f66354b75eb8446396a27c3d563191cc P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee02f77594864349982492862d668a36 in term 1.
I20260925 13:57:19.784303 22040 leader_election.cc:304] T f66354b75eb8446396a27c3d563191cc P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:19.785363 22520 raft_consensus.cc:2804] T f66354b75eb8446396a27c3d563191cc P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:19.786281 22212 ts_tablet_manager.cc:1434] T f66354b75eb8446396a27c3d563191cc P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.098s user 0.004s sys 0.001s
I20260925 13:57:19.789574 22520 raft_consensus.cc:697] T f66354b75eb8446396a27c3d563191cc P ee02f77594864349982492862d668a36 [term 1 LEADER]: Becoming Leader. State: Replica: ee02f77594864349982492862d668a36, State: Running, Role: LEADER
I20260925 13:57:19.790345 22520 consensus_queue.cc:237] T f66354b75eb8446396a27c3d563191cc P ee02f77594864349982492862d668a36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.795198 22212 tablet_bootstrap.cc:492] T 24dd39b302344f8d8f5166e9d641d13a P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:19.797209 22415 ts_tablet_manager.cc:1929] T 3a8c53e379a9407c920d91b05ebd2afc P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:19.797595 22415 log.cc:1199] T 3a8c53e379a9407c920d91b05ebd2afc P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/3a8c53e379a9407c920d91b05ebd2afc
I20260925 13:57:19.798424 22415 ts_tablet_manager.cc:1950] T 3a8c53e379a9407c920d91b05ebd2afc P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:19.802222 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66354b75eb8446396a27c3d563191cc" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
W20260925 13:57:19.803258 22043 leader_election.cc:343] T f66354b75eb8446396a27c3d563191cc P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Illegal state: Tablet not RUNNING: STOPPED
I20260925 13:57:19.803671 22415 tablet_replica.cc:333] T f66354b75eb8446396a27c3d563191cc P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:19.804236 22415 raft_consensus.cc:2243] T f66354b75eb8446396a27c3d563191cc P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:19.801749 22464 leader_election.cc:290] T f66354b75eb8446396a27c3d563191cc P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 election: Requested vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:19.804613 22415 raft_consensus.cc:2272] T f66354b75eb8446396a27c3d563191cc P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:19.807247 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet 3a8c53e379a9407c920d91b05ebd2afc (table default.table_b17685eaeada404790103bd89711435e [id=df6afcfb1e2c463493b14228e19ade26]) successfully deleted
I20260925 13:57:19.809836 21797 raft_consensus.cc:1275] T 761ede873f5648ccb5216a4c0a8afa2e P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Refusing update from remote peer 145e520e0a5a445d943a53cf8c27efee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:57:19.811061 22106 raft_consensus.cc:1275] T 761ede873f5648ccb5216a4c0a8afa2e P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Refusing update from remote peer 145e520e0a5a445d943a53cf8c27efee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:57:19.811239 22547 consensus_queue.cc:1048] T 761ede873f5648ccb5216a4c0a8afa2e P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:19.818805 22517 consensus_queue.cc:1048] T 761ede873f5648ccb5216a4c0a8afa2e P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:19.821710 22214 ts_tablet_manager.cc:1929] T 60256642160a44a2bab77b97d78c27a8 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:19.822067 22214 log.cc:1199] T 60256642160a44a2bab77b97d78c27a8 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/60256642160a44a2bab77b97d78c27a8
I20260925 13:57:19.822638 22214 ts_tablet_manager.cc:1950] T 60256642160a44a2bab77b97d78c27a8 P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:19.824206 22293 tablet_bootstrap.cc:654] T 76d4d7d706824892b075ab1aa4fbaf85 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:19.841601 21777 tablet_service.cc:1511] Processing CreateTablet for tablet 9745537e91d645fe9874373fe6237a9b (DEFAULT_TABLE table=default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:19.842437 22464 raft_consensus.cc:493] T 72fc427f45934cba91e52ad84275c461 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:19.843021 22464 raft_consensus.cc:515] T 72fc427f45934cba91e52ad84275c461 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.844219 21777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9745537e91d645fe9874373fe6237a9b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:57:19.847450 22040 leader_election.cc:343] T 72fc427f45934cba91e52ad84275c461 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 72fc427f45934cba91e52ad84275c461
I20260925 13:57:19.848614 22293 tablet_bootstrap.cc:492] T 76d4d7d706824892b075ab1aa4fbaf85 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:19.849623 22504 consensus_queue.cc:1048] T f68ccd8a7f4a4e518c2584e579de0c87 P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:19.849704 22293 ts_tablet_manager.cc:1403] T 76d4d7d706824892b075ab1aa4fbaf85 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.125s user 0.006s sys 0.003s
I20260925 13:57:19.850819 22085 tablet_service.cc:1511] Processing CreateTablet for tablet 9745537e91d645fe9874373fe6237a9b (DEFAULT_TABLE table=default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:19.851881 22085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9745537e91d645fe9874373fe6237a9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:19.854296 22293 raft_consensus.cc:359] T 76d4d7d706824892b075ab1aa4fbaf85 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.863431 22293 raft_consensus.cc:385] T 76d4d7d706824892b075ab1aa4fbaf85 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:19.864117 22293 raft_consensus.cc:740] T 76d4d7d706824892b075ab1aa4fbaf85 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:19.867497 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet 60256642160a44a2bab77b97d78c27a8 (table default.table_8b5db7d66475449d9772db568c9ded56 [id=7fb6c448c78344719796980b4be1cda9]) successfully deleted
I20260925 13:57:19.868110 22293 consensus_queue.cc:260] T 76d4d7d706824892b075ab1aa4fbaf85 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.868597 22214 tablet_replica.cc:333] T e07f41572a914b8a86c063c65e73e072 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:19.869314 22214 raft_consensus.cc:2243] T e07f41572a914b8a86c063c65e73e072 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:19.869737 22214 raft_consensus.cc:2272] T e07f41572a914b8a86c063c65e73e072 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:19.872396 22212 tablet_bootstrap.cc:654] T 24dd39b302344f8d8f5166e9d641d13a P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:19.873579 22415 ts_tablet_manager.cc:1916] T f66354b75eb8446396a27c3d563191cc P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:19.876835 22293 ts_tablet_manager.cc:1434] T 76d4d7d706824892b075ab1aa4fbaf85 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.026s user 0.005s sys 0.000s
I20260925 13:57:19.882287 22293 tablet_bootstrap.cc:492] T 24dd39b302344f8d8f5166e9d641d13a P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
W20260925 13:57:19.885602 22043 leader_election.cc:343] T 72fc427f45934cba91e52ad84275c461 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 72fc427f45934cba91e52ad84275c461
I20260925 13:57:19.885996 22043 leader_election.cc:304] T 72fc427f45934cba91e52ad84275c461 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:19.886677 22520 raft_consensus.cc:2749] T 72fc427f45934cba91e52ad84275c461 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:57:19.887207 22464 leader_election.cc:290] T 72fc427f45934cba91e52ad84275c461 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:19.898797 22293 tablet_bootstrap.cc:654] T 24dd39b302344f8d8f5166e9d641d13a P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:19.823335 22416 ts_tablet_manager.cc:1929] T f66354b75eb8446396a27c3d563191cc P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:19.900103 22416 log.cc:1199] T f66354b75eb8446396a27c3d563191cc P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/f66354b75eb8446396a27c3d563191cc
I20260925 13:57:19.829329 21940 tablet_service.cc:1511] Processing CreateTablet for tablet 9745537e91d645fe9874373fe6237a9b (DEFAULT_TABLE table=default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:19.901891 21940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9745537e91d645fe9874373fe6237a9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:19.833395 22211 tablet_bootstrap.cc:492] T 56a39663479d40be8bbd1dfd4c91040c P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:19.908610 22211 tablet_bootstrap.cc:654] T 56a39663479d40be8bbd1dfd4c91040c P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:19.934779 22212 tablet_bootstrap.cc:492] T 24dd39b302344f8d8f5166e9d641d13a P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:19.935195 22212 ts_tablet_manager.cc:1403] T 24dd39b302344f8d8f5166e9d641d13a P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.140s user 0.005s sys 0.004s
I20260925 13:57:19.935791 22214 ts_tablet_manager.cc:1916] T e07f41572a914b8a86c063c65e73e072 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:19.944805 22528 raft_consensus.cc:493] T 76d4d7d706824892b075ab1aa4fbaf85 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:19.948287 22211 tablet_bootstrap.cc:492] T 56a39663479d40be8bbd1dfd4c91040c P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:19.948690 22211 ts_tablet_manager.cc:1403] T 56a39663479d40be8bbd1dfd4c91040c P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.115s user 0.009s sys 0.001s
I20260925 13:57:19.954952 21579 catalog_manager.cc:5938] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873) does not have the latest schema for tablet 761ede873f5648ccb5216a4c0a8afa2e (table default.table_d6100b0f2b294ab099c179a7681cb4ed [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 2 got 0
I20260925 13:57:19.900764 22416 ts_tablet_manager.cc:1950] T f66354b75eb8446396a27c3d563191cc P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:19.964413 22212 raft_consensus.cc:359] T 24dd39b302344f8d8f5166e9d641d13a P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.964993 22212 raft_consensus.cc:385] T 24dd39b302344f8d8f5166e9d641d13a P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:19.965216 22212 raft_consensus.cc:740] T 24dd39b302344f8d8f5166e9d641d13a P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:19.971436 22528 raft_consensus.cc:515] T 76d4d7d706824892b075ab1aa4fbaf85 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.974351 22528 leader_election.cc:290] T 76d4d7d706824892b075ab1aa4fbaf85 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:19.976065 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet f66354b75eb8446396a27c3d563191cc (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]) successfully deleted
I20260925 13:57:19.976532 22105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d4d7d706824892b075ab1aa4fbaf85" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:19.977075 22212 consensus_queue.cc:260] T 24dd39b302344f8d8f5166e9d641d13a P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:19.978336 21896 leader_election.cc:343] T 76d4d7d706824892b075ab1aa4fbaf85 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:57:19.984086 21587 catalog_manager.cc:3541] renamed table default.table_b17685eaeada404790103bd89711435e [id=df6afcfb1e2c463493b14228e19ade26] in HMS: new name default.table_05ac73cb4d774c0fbd89f99093663e76
I20260925 13:57:19.987993 22212 ts_tablet_manager.cc:1434] T 24dd39b302344f8d8f5166e9d641d13a P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.051s user 0.002s sys 0.003s
I20260925 13:57:19.990077 22211 raft_consensus.cc:359] T 56a39663479d40be8bbd1dfd4c91040c P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.990684 22211 raft_consensus.cc:385] T 56a39663479d40be8bbd1dfd4c91040c P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:19.990990 22211 raft_consensus.cc:740] T 56a39663479d40be8bbd1dfd4c91040c P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:19.991796 22211 consensus_queue.cc:260] T 56a39663479d40be8bbd1dfd4c91040c P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:19.994028 22212 tablet_bootstrap.cc:492] T 76d4d7d706824892b075ab1aa4fbaf85 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:19.996011 22211 ts_tablet_manager.cc:1434] T 56a39663479d40be8bbd1dfd4c91040c P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.046s user 0.003s sys 0.005s
I20260925 13:57:19.996768 22211 tablet_bootstrap.cc:492] T 9745537e91d645fe9874373fe6237a9b P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:20.011466 22528 raft_consensus.cc:493] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:20.011946 22528 raft_consensus.cc:515] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.014020 22528 leader_election.cc:290] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:20.019774 22211 tablet_bootstrap.cc:654] T 9745537e91d645fe9874373fe6237a9b P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
W20260925 13:57:20.021760 21896 leader_election.cc:343] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Not found: Tablet not found: 60256642160a44a2bab77b97d78c27a8
I20260925 13:57:20.023845 22212 tablet_bootstrap.cc:654] T 76d4d7d706824892b075ab1aa4fbaf85 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:20.027808 22561 tablet.cc:1751] T 761ede873f5648ccb5216a4c0a8afa2e P 145e520e0a5a445d943a53cf8c27efee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 13:57:20.029618 22562 tablet.cc:1751] T 761ede873f5648ccb5216a4c0a8afa2e P ee02f77594864349982492862d668a36: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260925 13:57:12.530228 22283 debug-util.cc:398] Leaking SignalData structure 0x7b08000292a0 after lost signal to thread 22221
W20260925 13:57:20.028990 22221 thread.cc:641] GCE (cloud detector) Time spent creating pthread: real 8.798s user 0.718s sys 1.359s
W20260925 13:57:20.035146 22221 thread.cc:608] GCE (cloud detector) Time spent starting thread: real 8.805s user 0.718s sys 1.359s
W20260925 13:57:11.234963 22296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:57:20.045662 21579 catalog_manager.cc:5938] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873) does not have the latest schema for tablet 761ede873f5648ccb5216a4c0a8afa2e (table default.table_d6100b0f2b294ab099c179a7681cb4ed [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 2 got 0
I20260925 13:57:20.049124 21581 catalog_manager.cc:5938] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291) does not have the latest schema for tablet 761ede873f5648ccb5216a4c0a8afa2e (table default.table_d6100b0f2b294ab099c179a7681cb4ed [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 2 got 0
I20260925 13:57:20.083195 22221 server_base.cc:1061] running on GCE node
I20260925 13:57:20.059237 22214 ts_tablet_manager.cc:1929] T e07f41572a914b8a86c063c65e73e072 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:20.083178 22211 tablet_bootstrap.cc:492] T 9745537e91d645fe9874373fe6237a9b P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:20.085214 22211 ts_tablet_manager.cc:1403] T 9745537e91d645fe9874373fe6237a9b P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.089s user 0.009s sys 0.000s
I20260925 13:57:20.084599 22214 log.cc:1199] T e07f41572a914b8a86c063c65e73e072 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/e07f41572a914b8a86c063c65e73e072
I20260925 13:57:20.086215 22214 ts_tablet_manager.cc:1950] T e07f41572a914b8a86c063c65e73e072 P ee02f77594864349982492862d668a36: Deleting consensus metadata
W20260925 13:57:20.087905 22565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:57:20.089076 22212 tablet_bootstrap.cc:492] T 76d4d7d706824892b075ab1aa4fbaf85 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:20.089450 22212 ts_tablet_manager.cc:1403] T 76d4d7d706824892b075ab1aa4fbaf85 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.096s user 0.012s sys 0.000s
I20260925 13:57:20.091686 22221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:57:20.095335 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet e07f41572a914b8a86c063c65e73e072 (table default.table_61d48f9061c741fc8aa556e3edd984fb [id=7377fa3ea9664b47a930d9c5f88c60fa]) successfully deleted
I20260925 13:57:20.098853 22212 raft_consensus.cc:359] T 76d4d7d706824892b075ab1aa4fbaf85 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.099484 22212 raft_consensus.cc:385] T 76d4d7d706824892b075ab1aa4fbaf85 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:20.099711 22212 raft_consensus.cc:740] T 76d4d7d706824892b075ab1aa4fbaf85 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:20.100308 22212 consensus_queue.cc:260] T 76d4d7d706824892b075ab1aa4fbaf85 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.100627 22214 tablet_replica.cc:333] T 72fc427f45934cba91e52ad84275c461 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:20.101472 22211 raft_consensus.cc:359] T 9745537e91d645fe9874373fe6237a9b P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.102080 22211 raft_consensus.cc:385] T 9745537e91d645fe9874373fe6237a9b P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:20.102310 22211 raft_consensus.cc:740] T 9745537e91d645fe9874373fe6237a9b P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:20.102932 22214 raft_consensus.cc:2243] T 72fc427f45934cba91e52ad84275c461 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:20.102943 22211 consensus_queue.cc:260] T 9745537e91d645fe9874373fe6237a9b P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.121526 22293 tablet_bootstrap.cc:492] T 24dd39b302344f8d8f5166e9d641d13a P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:20.122686 22293 ts_tablet_manager.cc:1403] T 24dd39b302344f8d8f5166e9d641d13a P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.241s user 0.006s sys 0.013s
I20260925 13:57:20.123941 21579 catalog_manager.cc:5938] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291) does not have the latest schema for tablet 9745537e91d645fe9874373fe6237a9b (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]). Expected version 1 got 0
I20260925 13:57:20.132512 22504 raft_consensus.cc:493] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:20.133353 22504 raft_consensus.cc:515] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.136559 21581 master_service.cc:960] ReplaceTablet: received request to replace tablet f3b0e015d5dd4a668b6a04b49f628070 from {username='slave'} at 127.0.0.1:35994
W20260925 13:57:20.137339 21579 catalog_manager.cc:6038] Failed to send AlterTable RPC for tablet 9745537e91d645fe9874373fe6237a9b (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 9745537e91d645fe9874373fe6237a9b consensus state has no leader
I20260925 13:57:20.139133 21579 master_service.cc:960] ReplaceTablet: received request to replace tablet f3b0e015d5dd4a668b6a04b49f628070 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:20.140213 22504 leader_election.cc:290] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:20.140702 22211 ts_tablet_manager.cc:1434] T 9745537e91d645fe9874373fe6237a9b P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.055s user 0.005s sys 0.000s
I20260925 13:57:20.142716 22214 raft_consensus.cc:2272] T 72fc427f45934cba91e52ad84275c461 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:20.144371 22214 ts_tablet_manager.cc:1916] T 72fc427f45934cba91e52ad84275c461 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:20.151091 22212 ts_tablet_manager.cc:1434] T 76d4d7d706824892b075ab1aa4fbaf85 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.061s user 0.006s sys 0.000s
I20260925 13:57:20.147212 22415 ts_tablet_manager.cc:1929] T f66354b75eb8446396a27c3d563191cc P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:20.152508 22212 tablet_bootstrap.cc:492] T 56a39663479d40be8bbd1dfd4c91040c P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:20.151681 22415 log.cc:1199] T f66354b75eb8446396a27c3d563191cc P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/f66354b75eb8446396a27c3d563191cc
I20260925 13:57:20.153632 22415 ts_tablet_manager.cc:1950] T f66354b75eb8446396a27c3d563191cc P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:20.157945 22212 tablet_bootstrap.cc:654] T 56a39663479d40be8bbd1dfd4c91040c P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:20.129892 22293 raft_consensus.cc:359] T 24dd39b302344f8d8f5166e9d641d13a P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.158588 22293 raft_consensus.cc:385] T 24dd39b302344f8d8f5166e9d641d13a P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:20.158885 22293 raft_consensus.cc:740] T 24dd39b302344f8d8f5166e9d641d13a P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:20.159533 22293 consensus_queue.cc:260] T 24dd39b302344f8d8f5166e9d641d13a P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:20.171620 21572 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 9745537e91d645fe9874373fe6237a9b (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 9745537e91d645fe9874373fe6237a9b consensus state has no leader
I20260925 13:57:20.172112 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet f66354b75eb8446396a27c3d563191cc (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]) successfully deleted
I20260925 13:57:20.180887 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bd96a2bdb842e9b3bc1f706786d6b1" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:20.181586 21960 raft_consensus.cc:2393] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e2731975dbf433aade4a859df9699e2 in current term 1: Already voted for candidate ee02f77594864349982492862d668a36 in this term.
I20260925 13:57:20.184134 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bd96a2bdb842e9b3bc1f706786d6b1" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:20.185454 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d4d7d706824892b075ab1aa4fbaf85" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2" is_pre_election: true
I20260925 13:57:20.186170 21797 raft_consensus.cc:2468] T 76d4d7d706824892b075ab1aa4fbaf85 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 0.
I20260925 13:57:20.187585 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60256642160a44a2bab77b97d78c27a8" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2" is_pre_election: true
I20260925 13:57:20.188104 21797 raft_consensus.cc:2468] T 60256642160a44a2bab77b97d78c27a8 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 0.
I20260925 13:57:20.212930 21893 leader_election.cc:304] T 76d4d7d706824892b075ab1aa4fbaf85 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters: ee02f77594864349982492862d668a36
I20260925 13:57:20.214046 22517 raft_consensus.cc:2804] T 76d4d7d706824892b075ab1aa4fbaf85 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:20.214371 22517 raft_consensus.cc:493] T 76d4d7d706824892b075ab1aa4fbaf85 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:20.214666 22517 raft_consensus.cc:3060] T 76d4d7d706824892b075ab1aa4fbaf85 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:20.215566 21581 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet f3b0e015d5dd4a668b6a04b49f628070
I20260925 13:57:20.217566 21940 tablet_service.cc:1558] Processing DeleteTablet for tablet f3b0e015d5dd4a668b6a04b49f628070 with delete_type TABLET_DATA_DELETED (replaced by tablet 195c48349fb5410495299c9e71a72ad8) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:20.218235 22416 tablet_replica.cc:333] T f3b0e015d5dd4a668b6a04b49f628070 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:20.218922 22416 raft_consensus.cc:2243] T f3b0e015d5dd4a668b6a04b49f628070 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:20.219329 22416 raft_consensus.cc:2272] T f3b0e015d5dd4a668b6a04b49f628070 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:20.220952 22416 ts_tablet_manager.cc:1916] T f3b0e015d5dd4a668b6a04b49f628070 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:20.221539 22517 raft_consensus.cc:515] T 76d4d7d706824892b075ab1aa4fbaf85 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.222725 22464 consensus_queue.cc:1048] T f3b0e015d5dd4a668b6a04b49f628070 P ee02f77594864349982492862d668a36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:20.224804 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d4d7d706824892b075ab1aa4fbaf85" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:20.225296 22212 tablet_bootstrap.cc:492] T 56a39663479d40be8bbd1dfd4c91040c P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:20.225338 22107 raft_consensus.cc:3060] T 76d4d7d706824892b075ab1aa4fbaf85 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:20.225742 22212 ts_tablet_manager.cc:1403] T 56a39663479d40be8bbd1dfd4c91040c P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.073s user 0.023s sys 0.032s
I20260925 13:57:20.227658 21579 catalog_manager.cc:5938] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291) does not have the latest schema for tablet 9745537e91d645fe9874373fe6237a9b (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]). Expected version 1 got 0
I20260925 13:57:20.231400 21580 catalog_manager.cc:2213] created HMS catalog entry for table [id=ec768fd401eb409aaa08df2a6cdfa2f3]
I20260925 13:57:20.232211 22107 raft_consensus.cc:2468] T 76d4d7d706824892b075ab1aa4fbaf85 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 1.
I20260925 13:57:20.232275 22085 tablet_service.cc:1558] Processing DeleteTablet for tablet f3b0e015d5dd4a668b6a04b49f628070 with delete_type TABLET_DATA_DELETED (replaced by tablet 195c48349fb5410495299c9e71a72ad8) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:20.233171 22517 leader_election.cc:290] T 76d4d7d706824892b075ab1aa4fbaf85 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Requested vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:20.233341 21896 leader_election.cc:304] T 76d4d7d706824892b075ab1aa4fbaf85 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:20.239850 22212 raft_consensus.cc:359] T 56a39663479d40be8bbd1dfd4c91040c P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.240059 21893 leader_election.cc:304] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters: ee02f77594864349982492862d668a36
I20260925 13:57:20.240594 22212 raft_consensus.cc:385] T 56a39663479d40be8bbd1dfd4c91040c P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:20.240869 22212 raft_consensus.cc:740] T 56a39663479d40be8bbd1dfd4c91040c P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:20.241112 22528 raft_consensus.cc:2804] T 76d4d7d706824892b075ab1aa4fbaf85 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:20.241532 22528 raft_consensus.cc:697] T 76d4d7d706824892b075ab1aa4fbaf85 P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: Becoming Leader. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Running, Role: LEADER
I20260925 13:57:20.241659 22212 consensus_queue.cc:260] T 56a39663479d40be8bbd1dfd4c91040c P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.242280 22528 consensus_queue.cc:237] T 76d4d7d706824892b075ab1aa4fbaf85 P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.243501 22547 raft_consensus.cc:2804] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:20.243712 21580 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35994 (request call id 76) took 3036 ms. Trace:
I20260925 13:57:20.243816 22547 raft_consensus.cc:493] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:20.244251 22547 raft_consensus.cc:3060] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:20.244114 21580 rpcz_store.cc:276] 0925 13:57:17.207428 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:17.207545 (+ 117us) service_pool.cc:224] Handling call
0925 13:57:17.218889 (+ 11344us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b54000203d4. stack: 00007fd96d084323 00007fd96d084220 00007fd96cc28574 0000000000516964 00000000005168f9 00007fd9741825c0 00007fd9742216fa 00007fd975f92d8c 00007fd976092566 00007fd973c3d034 00007fd973c3cfc2 00007fd973c3cef1 00007fd973c3ce6c 00007fd973c3c122 00007fd971ee709c 00007fd971ee64d6
0925 13:57:19.043127 (+1824238us) catalog_manager.cc:2140] Acquired catalog manager lock
0925 13:57:19.044670 (+ 1543us) catalog_manager.cc:2186] Created new table and tablet info
0925 13:57:20.231328 (+1186658us) catalog_manager.cc:2212] Created new table in HMS catalog
0925 13:57:20.243058 (+ 11730us) catalog_manager.cc:2246] Wrote table and tablets to system table
0925 13:57:20.243320 (+ 262us) catalog_manager.cc:2280] Inserted table and tablets into CatalogManager maps
0925 13:57:20.243651 (+ 331us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:57:20.232896 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:57:20.233292 (+ 396us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:57:20.233316 (+ 24us) write_op.cc:435] Acquired schema lock
0925 13:57:20.233330 (+ 14us) tablet.cc:662] Decoding operations
0925 13:57:20.233615 (+ 285us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:57:20.233670 (+ 55us) write_op.cc:641] Partition lock acquired for write op
0925 13:57:20.233701 (+ 31us) tablet.cc:685] Acquiring locks for 2 operations
0925 13:57:20.233948 (+ 247us) tablet.cc:701] Row locks acquired
0925 13:57:20.233966 (+ 18us) write_op.cc:260] PREPARE: finished
0925 13:57:20.234096 (+ 130us) write_op.cc:270] Start()
0925 13:57:20.234194 (+ 98us) write_op.cc:276] Timestamp: P: 1790344640233549 usec, L: 0
0925 13:57:20.234215 (+ 21us) op_driver.cc:348] REPLICATION: starting
0925 13:57:20.234788 (+ 573us) log.cc:844] Serialized 433 byte log entry
0925 13:57:20.241062 (+ 6274us) op_driver.cc:464] REPLICATION: finished
0925 13:57:20.241486 (+ 424us) write_op.cc:301] APPLY: starting
0925 13:57:20.241537 (+ 51us) tablet.cc:1394] starting BulkCheckPresence
0925 13:57:20.241630 (+ 93us) tablet.cc:1397] finished BulkCheckPresence
0925 13:57:20.241719 (+ 89us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:57:20.242086 (+ 367us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:57:20.242202 (+ 116us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 13:57:20.242240 (+ 38us) write_op.cc:312] APPLY: finished
0925 13:57:20.242398 (+ 158us) log.cc:844] Serialized 29 byte log entry
0925 13:57:20.242532 (+ 134us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:20.242590 (+ 58us) write_op.cc:454] Released schema lock
0925 13:57:20.242617 (+ 27us) write_op.cc:341] FINISH: Updating metrics
0925 13:57:20.242806 (+ 189us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:20.242828 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":938896,"Hive Metastore.run_cpu_time_us":609,"Hive Metastore.run_wall_time_us":246577,"spinlock_wait_cycles":24116608,"child_traces":[["op",{"apply.queue_time_us":268,"apply.run_cpu_time_us":1293,"apply.run_wall_time_us":1293,"num_ops":2,"prepare.queue_time_us":105,"prepare.run_cpu_time_us":2233,"prepare.run_wall_time_us":3078,"raft.queue_time_us":140,"raft.run_cpu_time_us":619,"raft.run_wall_time_us":619,"replication_time_us":6740}]]}
I20260925 13:57:20.252106 21581 catalog_manager.cc:6934] ReplaceTablet: tablet f3b0e015d5dd4a668b6a04b49f628070 deleted and replaced by tablet 195c48349fb5410495299c9e71a72ad8
I20260925 13:57:20.253144 22212 ts_tablet_manager.cc:1434] T 56a39663479d40be8bbd1dfd4c91040c P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.027s user 0.006s sys 0.000s
I20260925 13:57:20.253718 22547 raft_consensus.cc:515] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.261188 22547 leader_election.cc:290] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Requested vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:20.261827 21580 catalog_manager.cc:5938] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881) does not have the latest schema for tablet 761ede873f5648ccb5216a4c0a8afa2e (table default.table_d6100b0f2b294ab099c179a7681cb4ed [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 2 got 0
W20260925 13:57:20.263454 21896 leader_election.cc:343] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Not found: Tablet not found: 60256642160a44a2bab77b97d78c27a8
I20260925 13:57:20.262284 22212 tablet_bootstrap.cc:492] T 9745537e91d645fe9874373fe6237a9b P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:20.276546 22547 raft_consensus.cc:493] T 56a39663479d40be8bbd1dfd4c91040c P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:20.276983 22547 raft_consensus.cc:515] T 56a39663479d40be8bbd1dfd4c91040c P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:20.277387 22043 consensus_peers.cc:597] T f3b0e015d5dd4a668b6a04b49f628070 P ee02f77594864349982492862d668a36 -> Peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Couldn't send request to peer 145e520e0a5a445d943a53cf8c27efee. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:57:20.225059 21777 tablet_service.cc:1558] Processing DeleteTablet for tablet f3b0e015d5dd4a668b6a04b49f628070 with delete_type TABLET_DATA_DELETED (replaced by tablet 195c48349fb5410495299c9e71a72ad8) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:20.197870 22293 ts_tablet_manager.cc:1434] T 24dd39b302344f8d8f5166e9d641d13a P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.075s user 0.012s sys 0.000s
I20260925 13:57:20.287407 22293 tablet_bootstrap.cc:492] T 56a39663479d40be8bbd1dfd4c91040c P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:20.288861 22415 tablet_replica.cc:333] T f3b0e015d5dd4a668b6a04b49f628070 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:20.289413 22212 tablet_bootstrap.cc:654] T 9745537e91d645fe9874373fe6237a9b P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:20.289536 22415 raft_consensus.cc:2243] T f3b0e015d5dd4a668b6a04b49f628070 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:20.290202 22415 raft_consensus.cc:2272] T f3b0e015d5dd4a668b6a04b49f628070 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:20.293123 22221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:57:20.306427 22221 hybrid_clock.cc:648] HybridClock initialized: now 1790344640306354 us; error 54 us; skew 500 ppm
I20260925 13:57:20.307683 22221 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 13:57:20.312920 22547 leader_election.cc:290] T 56a39663479d40be8bbd1dfd4c91040c P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:20.315797 21796 raft_consensus.cc:3060] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:20.266595 21732 leader_election.cc:304] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
I20260925 13:57:20.314957 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56a39663479d40be8bbd1dfd4c91040c" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:20.328369 22572 tablet.cc:1751] T 761ede873f5648ccb5216a4c0a8afa2e P 5e2731975dbf433aade4a859df9699e2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 13:57:20.336261 22415 ts_tablet_manager.cc:1916] T f3b0e015d5dd4a668b6a04b49f628070 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:20.338848 21580 catalog_manager.cc:6391] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: default.table_d6100b0f2b294ab099c179a7681cb4ed [id=74ee67384ecf4220b7d08fdf0637aefb] alter complete (version 2)
W20260925 13:57:20.318315 21732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.194:44291, user_credentials={real_user=slave}} blocked reactor thread for 51894.5us
I20260925 13:57:20.349695 22107 raft_consensus.cc:2468] T 56a39663479d40be8bbd1dfd4c91040c P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 0.
I20260925 13:57:20.350538 22570 raft_consensus.cc:2749] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:57:20.353848 21583 catalog_manager.cc:2213] created HMS catalog entry for table [id=012dc51aeefc492ca9aade1c75b97020]
I20260925 13:57:20.355250 21896 leader_election.cc:304] T 56a39663479d40be8bbd1dfd4c91040c P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:20.352519 22570 consensus_queue.cc:1048] T f68ccd8a7f4a4e518c2584e579de0c87 P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260925 13:57:20.366885 22549 consensus_queue.cc:1048] T f5bd96a2bdb842e9b3bc1f706786d6b1 P ee02f77594864349982492862d668a36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 13:57:20.373934 21583 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35994 (request call id 80) took 2791 ms. Trace:
I20260925 13:57:20.374256 21583 rpcz_store.cc:276] 0925 13:57:17.582986 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:17.633659 (+ 50673us) service_pool.cc:224] Handling call
0925 13:57:19.046506 (+1412847us) catalog_manager.cc:2140] Acquired catalog manager lock
0925 13:57:19.048031 (+ 1525us) catalog_manager.cc:2186] Created new table and tablet info
0925 13:57:20.353808 (+1305777us) catalog_manager.cc:2212] Created new table in HMS catalog
0925 13:57:20.373339 (+ 19531us) catalog_manager.cc:2246] Wrote table and tablets to system table
0925 13:57:20.373616 (+ 277us) catalog_manager.cc:2280] Inserted table and tablets into CatalogManager maps
0925 13:57:20.373877 (+ 261us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:57:20.356249 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:57:20.356642 (+ 393us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:57:20.356677 (+ 35us) write_op.cc:435] Acquired schema lock
0925 13:57:20.356689 (+ 12us) tablet.cc:662] Decoding operations
0925 13:57:20.356931 (+ 242us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:57:20.356966 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0925 13:57:20.356985 (+ 19us) tablet.cc:685] Acquiring locks for 2 operations
0925 13:57:20.357163 (+ 178us) tablet.cc:701] Row locks acquired
0925 13:57:20.357176 (+ 13us) write_op.cc:260] PREPARE: finished
0925 13:57:20.357272 (+ 96us) write_op.cc:270] Start()
0925 13:57:20.357345 (+ 73us) write_op.cc:276] Timestamp: P: 1790344640356724 usec, L: 0
0925 13:57:20.357360 (+ 15us) op_driver.cc:348] REPLICATION: starting
0925 13:57:20.357836 (+ 476us) log.cc:844] Serialized 433 byte log entry
0925 13:57:20.370946 (+ 13110us) op_driver.cc:464] REPLICATION: finished
0925 13:57:20.371295 (+ 349us) write_op.cc:301] APPLY: starting
0925 13:57:20.371344 (+ 49us) tablet.cc:1394] starting BulkCheckPresence
0925 13:57:20.371451 (+ 107us) tablet.cc:1397] finished BulkCheckPresence
0925 13:57:20.371548 (+ 97us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:57:20.371919 (+ 371us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:57:20.372041 (+ 122us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 13:57:20.372077 (+ 36us) write_op.cc:312] APPLY: finished
0925 13:57:20.372228 (+ 151us) log.cc:844] Serialized 29 byte log entry
0925 13:57:20.372360 (+ 132us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:20.372420 (+ 60us) write_op.cc:454] Released schema lock
0925 13:57:20.372446 (+ 26us) write_op.cc:341] FINISH: Updating metrics
0925 13:57:20.372649 (+ 203us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:20.372668 (+ 19us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":1182351,"Hive Metastore.run_cpu_time_us":567,"Hive Metastore.run_wall_time_us":122373,"child_traces":[["op",{"apply.queue_time_us":219,"apply.run_cpu_time_us":1328,"apply.run_wall_time_us":1327,"num_ops":2,"prepare.queue_time_us":1011,"prepare.run_cpu_time_us":1911,"prepare.run_wall_time_us":3207,"raft.queue_time_us":2251,"raft.run_cpu_time_us":628,"raft.run_wall_time_us":782,"replication_time_us":13503}]]}
I20260925 13:57:20.379444 22528 raft_consensus.cc:2804] T 56a39663479d40be8bbd1dfd4c91040c P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:20.379786 22528 raft_consensus.cc:493] T 56a39663479d40be8bbd1dfd4c91040c P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:20.380093 22528 raft_consensus.cc:3060] T 56a39663479d40be8bbd1dfd4c91040c P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:20.389065 22214 ts_tablet_manager.cc:1929] T 72fc427f45934cba91e52ad84275c461 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:20.389520 22214 log.cc:1199] T 72fc427f45934cba91e52ad84275c461 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/72fc427f45934cba91e52ad84275c461
I20260925 13:57:20.391251 22214 ts_tablet_manager.cc:1950] T 72fc427f45934cba91e52ad84275c461 P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:20.395608 22416 ts_tablet_manager.cc:1929] T f3b0e015d5dd4a668b6a04b49f628070 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:20.398746 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60256642160a44a2bab77b97d78c27a8" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2"
I20260925 13:57:20.399128 21794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56a39663479d40be8bbd1dfd4c91040c" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2" is_pre_election: true
I20260925 13:57:20.399333 21795 raft_consensus.cc:3060] T 60256642160a44a2bab77b97d78c27a8 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:20.406076 22416 log.cc:1199] T f3b0e015d5dd4a668b6a04b49f628070 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/f3b0e015d5dd4a668b6a04b49f628070
I20260925 13:57:20.406003 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d4d7d706824892b075ab1aa4fbaf85" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2"
I20260925 13:57:20.406755 22416 ts_tablet_manager.cc:1950] T f3b0e015d5dd4a668b6a04b49f628070 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:20.407259 21797 raft_consensus.cc:3060] T 76d4d7d706824892b075ab1aa4fbaf85 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:20.406220 21795 raft_consensus.cc:2468] T 60256642160a44a2bab77b97d78c27a8 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 1.
I20260925 13:57:20.391939 22212 tablet_bootstrap.cc:492] T 9745537e91d645fe9874373fe6237a9b P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:20.412731 22212 ts_tablet_manager.cc:1403] T 9745537e91d645fe9874373fe6237a9b P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.151s user 0.016s sys 0.012s
I20260925 13:57:20.413430 22214 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50052 (request call id 104) took 1274 ms. Trace:
I20260925 13:57:20.415228 22212 raft_consensus.cc:359] T 9745537e91d645fe9874373fe6237a9b P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.415882 22212 raft_consensus.cc:385] T 9745537e91d645fe9874373fe6237a9b P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:20.416142 22212 raft_consensus.cc:740] T 9745537e91d645fe9874373fe6237a9b P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:20.416829 22212 consensus_queue.cc:260] T 9745537e91d645fe9874373fe6237a9b P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.418244 22212 ts_tablet_manager.cc:1434] T 9745537e91d645fe9874373fe6237a9b P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260925 13:57:20.426031 22293 tablet_bootstrap.cc:654] T 56a39663479d40be8bbd1dfd4c91040c P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
W20260925 13:57:20.428254 21893 leader_election.cc:343] T 56a39663479d40be8bbd1dfd4c91040c P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:57:20.430125 21893 leader_election.cc:304] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters: ee02f77594864349982492862d668a36
I20260925 13:57:20.432065 22547 raft_consensus.cc:2804] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:20.432638 22464 consensus_queue.cc:1048] T f5bd96a2bdb842e9b3bc1f706786d6b1 P ee02f77594864349982492862d668a36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:20.433857 22547 raft_consensus.cc:697] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: Becoming Leader. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Running, Role: LEADER
I20260925 13:57:20.435511 22547 consensus_queue.cc:237] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.440091 21579 catalog_manager.cc:5838] T 76d4d7d706824892b075ab1aa4fbaf85 P 145e520e0a5a445d943a53cf8c27efee reported cstate change: term changed from 0 to 1, leader changed from <none> to 145e520e0a5a445d943a53cf8c27efee (127.18.182.194). New cstate: current_term: 1 leader_uuid: "145e520e0a5a445d943a53cf8c27efee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: UNKNOWN } } }
I20260925 13:57:20.442659 21581 catalog_manager.cc:5938] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873) does not have the latest schema for tablet 9745537e91d645fe9874373fe6237a9b (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]). Expected version 1 got 0
I20260925 13:57:20.442278 22214 rpcz_store.cc:276] 0925 13:57:19.139148 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:19.139327 (+ 179us) service_pool.cc:224] Handling call
0925 13:57:20.100429 (+961102us) ts_tablet_manager.cc:1151] Deleting tablet 72fc427f45934cba91e52ad84275c461
0925 13:57:20.100479 (+ 50us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:57:20.198312 (+ 97833us) tablet_metadata.cc:665] Metadata flushed
0925 13:57:20.388940 (+190628us) tablet_metadata.cc:665] Metadata flushed
0925 13:57:20.413336 (+ 24396us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38519,"tablet-delete.queue_time_us":960319}
I20260925 13:57:20.444159 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet f3b0e015d5dd4a668b6a04b49f628070 (table default.table_bd714c618f7a4f16a69a69d5932b2014 [id=dfd05c11872140cfa86b2628c33ef6d8]) successfully deleted
I20260925 13:57:20.440011 21777 tablet_service.cc:1511] Processing CreateTablet for tablet 195c48349fb5410495299c9e71a72ad8 (DEFAULT_TABLE table=default.table_bd714c618f7a4f16a69a69d5932b2014 [id=dfd05c11872140cfa86b2628c33ef6d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:20.445158 21777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 195c48349fb5410495299c9e71a72ad8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:20.445799 22214 tablet_replica.cc:333] T 3a8c53e379a9407c920d91b05ebd2afc P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:20.447688 22214 raft_consensus.cc:2243] T 3a8c53e379a9407c920d91b05ebd2afc P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:20.448099 22214 raft_consensus.cc:2272] T 3a8c53e379a9407c920d91b05ebd2afc P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:20.449771 22214 ts_tablet_manager.cc:1916] T 3a8c53e379a9407c920d91b05ebd2afc P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:20.451591 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet 72fc427f45934cba91e52ad84275c461 (table default.table_e0eb6c1d49224d1cb9b4f6d06f0f7a29 [id=420cd09cc1184f6e9fb38b2cbc3f6fce]) successfully deleted
I20260925 13:57:20.452772 21580 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35994:
name: "default.Table_f644beebc9834408bed8d57d2ac5c2ee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:57:20.458555 21775 tablet_service.cc:1511] Processing CreateTablet for tablet 1928234ffd3947a78e10d4c43e32aeb8 (DEFAULT_TABLE table=default.table_fce30b75d46f47089d7a31e04841dcdf [id=ec768fd401eb409aaa08df2a6cdfa2f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:20.462455 21582 catalog_manager.cc:2213] created HMS catalog entry for table [id=af0f962d5d5d4a17a3f576ab6a3b4df0]
I20260925 13:57:20.463186 22085 tablet_service.cc:1511] Processing CreateTablet for tablet 195c48349fb5410495299c9e71a72ad8 (DEFAULT_TABLE table=default.table_bd714c618f7a4f16a69a69d5932b2014 [id=dfd05c11872140cfa86b2628c33ef6d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:20.464243 22085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 195c48349fb5410495299c9e71a72ad8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:20.470491 21775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1928234ffd3947a78e10d4c43e32aeb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:20.475282 21582 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35994 (request call id 88) took 2016 ms. Trace:
I20260925 13:57:20.475598 21582 rpcz_store.cc:276] 0925 13:57:18.459316 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:18.459458 (+ 142us) service_pool.cc:224] Handling call
0925 13:57:19.055752 (+596294us) catalog_manager.cc:2140] Acquired catalog manager lock
0925 13:57:19.057153 (+ 1401us) catalog_manager.cc:2186] Created new table and tablet info
0925 13:57:20.462408 (+1405255us) catalog_manager.cc:2212] Created new table in HMS catalog
0925 13:57:20.474680 (+ 12272us) catalog_manager.cc:2246] Wrote table and tablets to system table
0925 13:57:20.474982 (+ 302us) catalog_manager.cc:2280] Inserted table and tablets into CatalogManager maps
0925 13:57:20.475229 (+ 247us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:57:20.464141 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:57:20.464491 (+ 350us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:57:20.464513 (+ 22us) write_op.cc:435] Acquired schema lock
0925 13:57:20.464525 (+ 12us) tablet.cc:662] Decoding operations
0925 13:57:20.464767 (+ 242us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:57:20.464797 (+ 30us) write_op.cc:641] Partition lock acquired for write op
0925 13:57:20.464817 (+ 20us) tablet.cc:685] Acquiring locks for 2 operations
0925 13:57:20.465124 (+ 307us) tablet.cc:701] Row locks acquired
0925 13:57:20.465143 (+ 19us) write_op.cc:260] PREPARE: finished
0925 13:57:20.465246 (+ 103us) write_op.cc:270] Start()
0925 13:57:20.465322 (+ 76us) write_op.cc:276] Timestamp: P: 1790344640464686 usec, L: 0
0925 13:57:20.465338 (+ 16us) op_driver.cc:348] REPLICATION: starting
0925 13:57:20.465786 (+ 448us) log.cc:844] Serialized 433 byte log entry
0925 13:57:20.473196 (+ 7410us) op_driver.cc:464] REPLICATION: finished
0925 13:57:20.473550 (+ 354us) write_op.cc:301] APPLY: starting
0925 13:57:20.473593 (+ 43us) tablet.cc:1394] starting BulkCheckPresence
0925 13:57:20.473677 (+ 84us) tablet.cc:1397] finished BulkCheckPresence
0925 13:57:20.473763 (+ 86us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:57:20.474076 (+ 313us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:57:20.474181 (+ 105us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 13:57:20.474218 (+ 37us) write_op.cc:312] APPLY: finished
0925 13:57:20.474344 (+ 126us) log.cc:844] Serialized 29 byte log entry
0925 13:57:20.474451 (+ 107us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:20.474502 (+ 51us) write_op.cc:454] Released schema lock
0925 13:57:20.474526 (+ 24us) write_op.cc:341] FINISH: Updating metrics
0925 13:57:20.474701 (+ 175us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:20.474722 (+ 21us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":1295724,"Hive Metastore.run_cpu_time_us":555,"Hive Metastore.run_wall_time_us":109041,"child_traces":[["op",{"apply.queue_time_us":231,"apply.run_cpu_time_us":1127,"apply.run_wall_time_us":1127,"num_ops":2,"prepare.queue_time_us":334,"prepare.run_cpu_time_us":1937,"prepare.run_wall_time_us":4990,"raft.queue_time_us":428,"raft.run_cpu_time_us":12,"raft.run_wall_time_us":12,"replication_time_us":7775,"spinlock_wait_cycles":27904}]]}
I20260925 13:57:20.478724 21583 catalog_manager.cc:5938] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873) does not have the latest schema for tablet 9745537e91d645fe9874373fe6237a9b (table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f]). Expected version 1 got 0
I20260925 13:57:20.480717 22517 raft_consensus.cc:493] T 24dd39b302344f8d8f5166e9d641d13a P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:20.481170 22517 raft_consensus.cc:515] T 24dd39b302344f8d8f5166e9d641d13a P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.482321 21797 raft_consensus.cc:2468] T 76d4d7d706824892b075ab1aa4fbaf85 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 1.
I20260925 13:57:20.484373 22517 leader_election.cc:290] T 24dd39b302344f8d8f5166e9d641d13a P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:20.485150 22528 raft_consensus.cc:515] T 56a39663479d40be8bbd1dfd4c91040c P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.486703 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dd39b302344f8d8f5166e9d641d13a" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:20.487290 22107 raft_consensus.cc:2468] T 24dd39b302344f8d8f5166e9d641d13a P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 0.
I20260925 13:57:20.488104 22528 leader_election.cc:290] T 56a39663479d40be8bbd1dfd4c91040c P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Requested vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:20.488786 21896 leader_election.cc:304] T 24dd39b302344f8d8f5166e9d641d13a P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:20.490978 22547 raft_consensus.cc:2804] T 24dd39b302344f8d8f5166e9d641d13a P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:20.491293 22547 raft_consensus.cc:493] T 24dd39b302344f8d8f5166e9d641d13a P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:20.491580 22547 raft_consensus.cc:3060] T 24dd39b302344f8d8f5166e9d641d13a P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:20.496145 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56a39663479d40be8bbd1dfd4c91040c" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:20.496639 22107 raft_consensus.cc:3060] T 56a39663479d40be8bbd1dfd4c91040c P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:20.502766 21940 tablet_service.cc:1511] Processing CreateTablet for tablet 195c48349fb5410495299c9e71a72ad8 (DEFAULT_TABLE table=default.table_bd714c618f7a4f16a69a69d5932b2014 [id=dfd05c11872140cfa86b2628c33ef6d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:20.503682 21940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 195c48349fb5410495299c9e71a72ad8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:20.509959 21938 tablet_service.cc:1511] Processing CreateTablet for tablet 1928234ffd3947a78e10d4c43e32aeb8 (DEFAULT_TABLE table=default.table_fce30b75d46f47089d7a31e04841dcdf [id=ec768fd401eb409aaa08df2a6cdfa2f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:20.510859 21938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1928234ffd3947a78e10d4c43e32aeb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:20.514518 22293 tablet_bootstrap.cc:492] T 56a39663479d40be8bbd1dfd4c91040c P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:20.514881 22293 ts_tablet_manager.cc:1403] T 56a39663479d40be8bbd1dfd4c91040c P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.228s user 0.010s sys 0.000s
I20260925 13:57:20.516009 22087 tablet_service.cc:1511] Processing CreateTablet for tablet 1928234ffd3947a78e10d4c43e32aeb8 (DEFAULT_TABLE table=default.table_fce30b75d46f47089d7a31e04841dcdf [id=ec768fd401eb409aaa08df2a6cdfa2f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:20.516933 22087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1928234ffd3947a78e10d4c43e32aeb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:20.519546 21583 master_service.cc:960] ReplaceTablet: received request to replace tablet f481b714a5f34fff8f74ba30bdd5dd35 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:20.517609 22293 raft_consensus.cc:359] T 56a39663479d40be8bbd1dfd4c91040c P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.523773 22293 raft_consensus.cc:385] T 56a39663479d40be8bbd1dfd4c91040c P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:20.524039 22293 raft_consensus.cc:740] T 56a39663479d40be8bbd1dfd4c91040c P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:20.524629 22293 consensus_queue.cc:260] T 56a39663479d40be8bbd1dfd4c91040c P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.526515 22293 ts_tablet_manager.cc:1434] T 56a39663479d40be8bbd1dfd4c91040c P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.011s user 0.000s sys 0.003s
I20260925 13:57:20.536414 22547 raft_consensus.cc:515] T 24dd39b302344f8d8f5166e9d641d13a P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.542858 22211 tablet_bootstrap.cc:492] T 1928234ffd3947a78e10d4c43e32aeb8 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:20.527282 22293 tablet_bootstrap.cc:492] T 9745537e91d645fe9874373fe6237a9b P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:20.546919 22212 tablet_bootstrap.cc:492] T 1928234ffd3947a78e10d4c43e32aeb8 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:20.548926 22211 tablet_bootstrap.cc:654] T 1928234ffd3947a78e10d4c43e32aeb8 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:20.555061 22547 leader_election.cc:290] T 24dd39b302344f8d8f5166e9d641d13a P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Requested vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:20.556255 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dd39b302344f8d8f5166e9d641d13a" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2" is_pre_election: true
I20260925 13:57:20.556806 21797 raft_consensus.cc:2468] T 24dd39b302344f8d8f5166e9d641d13a P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 0.
I20260925 13:57:20.534098 22415 ts_tablet_manager.cc:1929] T f3b0e015d5dd4a668b6a04b49f628070 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:20.559739 22415 log.cc:1199] T f3b0e015d5dd4a668b6a04b49f628070 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/f3b0e015d5dd4a668b6a04b49f628070
I20260925 13:57:20.560950 22415 ts_tablet_manager.cc:1950] T f3b0e015d5dd4a668b6a04b49f628070 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:20.564491 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet f3b0e015d5dd4a668b6a04b49f628070 (table default.table_bd714c618f7a4f16a69a69d5932b2014 [id=dfd05c11872140cfa86b2628c33ef6d8]) successfully deleted
I20260925 13:57:20.565806 21579 master_service.cc:960] ReplaceTablet: received request to replace tablet 9745537e91d645fe9874373fe6237a9b from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:20.566723 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56a39663479d40be8bbd1dfd4c91040c" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2"
I20260925 13:57:20.567224 21795 raft_consensus.cc:3060] T 56a39663479d40be8bbd1dfd4c91040c P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:20.574497 21795 raft_consensus.cc:2468] T 56a39663479d40be8bbd1dfd4c91040c P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 1.
I20260925 13:57:20.576467 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dd39b302344f8d8f5166e9d641d13a" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2"
I20260925 13:57:20.576947 21795 raft_consensus.cc:3060] T 24dd39b302344f8d8f5166e9d641d13a P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:20.579141 21893 leader_election.cc:304] T 56a39663479d40be8bbd1dfd4c91040c P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:20.579622 21583 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet f481b714a5f34fff8f74ba30bdd5dd35
I20260925 13:57:20.579993 22211 tablet_bootstrap.cc:492] T 1928234ffd3947a78e10d4c43e32aeb8 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:20.580389 22211 ts_tablet_manager.cc:1403] T 1928234ffd3947a78e10d4c43e32aeb8 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.038s user 0.009s sys 0.000s
I20260925 13:57:20.580523 22547 raft_consensus.cc:2804] T 56a39663479d40be8bbd1dfd4c91040c P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:20.582918 22547 raft_consensus.cc:697] T 56a39663479d40be8bbd1dfd4c91040c P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: Becoming Leader. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Running, Role: LEADER
I20260925 13:57:20.585887 21582 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35994:
name: "default.Table_1baae11db5a24d50bd596853233007cb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:57:20.588475 21776 tablet_service.cc:1558] Processing DeleteTablet for tablet f481b714a5f34fff8f74ba30bdd5dd35 with delete_type TABLET_DATA_DELETED (replaced by tablet 794784c42d27441f829f954b6929a097) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:20.590379 22415 tablet_replica.cc:333] T f481b714a5f34fff8f74ba30bdd5dd35 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:20.582809 22211 raft_consensus.cc:359] T 1928234ffd3947a78e10d4c43e32aeb8 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.591002 22211 raft_consensus.cc:385] T 1928234ffd3947a78e10d4c43e32aeb8 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:20.591167 22415 raft_consensus.cc:2243] T f481b714a5f34fff8f74ba30bdd5dd35 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:20.591267 22211 raft_consensus.cc:740] T 1928234ffd3947a78e10d4c43e32aeb8 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:20.591666 22415 raft_consensus.cc:2272] T f481b714a5f34fff8f74ba30bdd5dd35 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:20.595397 21938 tablet_service.cc:1558] Processing DeleteTablet for tablet f481b714a5f34fff8f74ba30bdd5dd35 with delete_type TABLET_DATA_DELETED (replaced by tablet 794784c42d27441f829f954b6929a097) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:20.596233 22416 tablet_replica.cc:333] T f481b714a5f34fff8f74ba30bdd5dd35 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:20.596738 22087 tablet_service.cc:1558] Processing DeleteTablet for tablet f481b714a5f34fff8f74ba30bdd5dd35 with delete_type TABLET_DATA_DELETED (replaced by tablet 794784c42d27441f829f954b6929a097) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:20.596921 22416 raft_consensus.cc:2243] T f481b714a5f34fff8f74ba30bdd5dd35 P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:20.597225 22212 tablet_bootstrap.cc:654] T 1928234ffd3947a78e10d4c43e32aeb8 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:20.597566 22416 pending_rounds.cc:70] T f481b714a5f34fff8f74ba30bdd5dd35 P 145e520e0a5a445d943a53cf8c27efee: Trying to abort 1 pending ops.
I20260925 13:57:20.597796 22416 pending_rounds.cc:77] T f481b714a5f34fff8f74ba30bdd5dd35 P 145e520e0a5a445d943a53cf8c27efee: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251644308738048 op_type: NO_OP noop_request { }
I20260925 13:57:20.598176 22416 raft_consensus.cc:2889] T f481b714a5f34fff8f74ba30bdd5dd35 P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:57:20.598495 22416 raft_consensus.cc:2272] T f481b714a5f34fff8f74ba30bdd5dd35 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:20.598510 22106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dd39b302344f8d8f5166e9d641d13a" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:20.599047 22106 raft_consensus.cc:3060] T 24dd39b302344f8d8f5166e9d641d13a P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:20.599886 21583 catalog_manager.cc:6934] ReplaceTablet: tablet f481b714a5f34fff8f74ba30bdd5dd35 deleted and replaced by tablet 794784c42d27441f829f954b6929a097
I20260925 13:57:20.600164 21584 catalog_manager.cc:2213] created HMS catalog entry for table [id=bf8cef2239904fd989622f9426412635]
I20260925 13:57:20.601403 22416 ts_tablet_manager.cc:1916] T f481b714a5f34fff8f74ba30bdd5dd35 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:20.601999 22211 consensus_queue.cc:260] T 1928234ffd3947a78e10d4c43e32aeb8 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.605950 22211 ts_tablet_manager.cc:1434] T 1928234ffd3947a78e10d4c43e32aeb8 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.025s user 0.006s sys 0.000s
I20260925 13:57:20.610936 22106 raft_consensus.cc:2468] T 24dd39b302344f8d8f5166e9d641d13a P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 1.
I20260925 13:57:20.615607 21896 leader_election.cc:304] T 24dd39b302344f8d8f5166e9d641d13a P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36; no voters:
W20260925 13:57:20.616423 22043 consensus_peers.cc:597] T f66354b75eb8446396a27c3d563191cc P ee02f77594864349982492862d668a36 -> Peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Couldn't send request to peer 145e520e0a5a445d943a53cf8c27efee. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f66354b75eb8446396a27c3d563191cc. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:57:20.620182 22517 raft_consensus.cc:2804] T 24dd39b302344f8d8f5166e9d641d13a P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:20.620577 22517 raft_consensus.cc:697] T 24dd39b302344f8d8f5166e9d641d13a P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: Becoming Leader. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Running, Role: LEADER
I20260925 13:57:20.621330 22517 consensus_queue.cc:237] T 24dd39b302344f8d8f5166e9d641d13a P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.527241 22107 raft_consensus.cc:2468] T 56a39663479d40be8bbd1dfd4c91040c P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 1.
I20260925 13:57:20.622267 21643 catalog_manager.cc:2811] Deleting table default.table_01a6dec392d24e9f89578f6f4274c704 [id=9d18288ef656432994079ec83988ec3f] in response to Hive Metastore notification log event 19
I20260925 13:57:20.583686 22547 consensus_queue.cc:237] T 56a39663479d40be8bbd1dfd4c91040c P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.638649 22416 ts_tablet_manager.cc:1929] T f481b714a5f34fff8f74ba30bdd5dd35 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:20.639021 22416 log.cc:1199] T f481b714a5f34fff8f74ba30bdd5dd35 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/f481b714a5f34fff8f74ba30bdd5dd35
I20260925 13:57:20.639721 22416 ts_tablet_manager.cc:1950] T f481b714a5f34fff8f74ba30bdd5dd35 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:20.641582 21584 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35994 (request call id 84) took 2641 ms. Trace:
I20260925 13:57:20.642009 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet f481b714a5f34fff8f74ba30bdd5dd35 (table default.table_61d48f9061c741fc8aa556e3edd984fb [id=7377fa3ea9664b47a930d9c5f88c60fa]) successfully deleted
I20260925 13:57:20.641917 21584 rpcz_store.cc:276] 0925 13:57:18.000360 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:18.003982 (+ 3622us) service_pool.cc:224] Handling call
0925 13:57:19.049840 (+1045858us) catalog_manager.cc:2140] Acquired catalog manager lock
0925 13:57:19.060282 (+ 10442us) catalog_manager.cc:2186] Created new table and tablet info
0925 13:57:20.600122 (+1539840us) catalog_manager.cc:2212] Created new table in HMS catalog
0925 13:57:20.640923 (+ 40801us) catalog_manager.cc:2246] Wrote table and tablets to system table
0925 13:57:20.641244 (+ 321us) catalog_manager.cc:2280] Inserted table and tablets into CatalogManager maps
0925 13:57:20.641525 (+ 281us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:57:20.623637 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:57:20.624035 (+ 398us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:57:20.624055 (+ 20us) write_op.cc:435] Acquired schema lock
0925 13:57:20.624067 (+ 12us) tablet.cc:662] Decoding operations
0925 13:57:20.624331 (+ 264us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:57:20.624378 (+ 47us) write_op.cc:641] Partition lock acquired for write op
0925 13:57:20.624408 (+ 30us) tablet.cc:685] Acquiring locks for 2 operations
0925 13:57:20.624642 (+ 234us) tablet.cc:701] Row locks acquired
0925 13:57:20.624659 (+ 17us) write_op.cc:260] PREPARE: finished
0925 13:57:20.624779 (+ 120us) write_op.cc:270] Start()
0925 13:57:20.624884 (+ 105us) write_op.cc:276] Timestamp: P: 1790344640624199 usec, L: 0
0925 13:57:20.624904 (+ 20us) op_driver.cc:348] REPLICATION: starting
0925 13:57:20.631046 (+ 6142us) log.cc:844] Serialized 433 byte log entry
0925 13:57:20.632568 (+ 1522us) spinlock_profiling.cc:243] Waited 5.54 ms on lock 0x7b54000203d4. stack: 00007fd96d084323 00007fd96d084220 00007fd96cc28574 0000000000516964 00000000005168f9 00007fd974181aa1 00007fd9742145e8 00007fd9746a4741 00007fd9746a3f18 00007fd9746a8bb1 00007fd9746a8b69 00007fd9746a8af9 00007fd9746a8ac1 00007fd9746a7ced 00000000005bd4e4 00000000005bd319
0925 13:57:20.638969 (+ 6401us) op_driver.cc:464] REPLICATION: finished
0925 13:57:20.639508 (+ 539us) write_op.cc:301] APPLY: starting
0925 13:57:20.639560 (+ 52us) tablet.cc:1394] starting BulkCheckPresence
0925 13:57:20.639653 (+ 93us) tablet.cc:1397] finished BulkCheckPresence
0925 13:57:20.639667 (+ 14us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:57:20.639983 (+ 316us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:57:20.640117 (+ 134us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 13:57:20.640157 (+ 40us) write_op.cc:312] APPLY: finished
0925 13:57:20.640298 (+ 141us) log.cc:844] Serialized 29 byte log entry
0925 13:57:20.640422 (+ 124us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:20.640478 (+ 56us) write_op.cc:454] Released schema lock
0925 13:57:20.640503 (+ 25us) write_op.cc:341] FINISH: Updating metrics
0925 13:57:20.640676 (+ 173us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:20.640694 (+ 18us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":1401670,"Hive Metastore.run_cpu_time_us":584,"Hive Metastore.run_wall_time_us":137372,"child_traces":[["op",{"apply.queue_time_us":386,"apply.run_cpu_time_us":1152,"apply.run_wall_time_us":1153,"num_ops":2,"prepare.queue_time_us":21933,"prepare.run_cpu_time_us":2779,"prepare.run_wall_time_us":11423,"raft.queue_time_us":74,"raft.run_cpu_time_us":708,"raft.run_wall_time_us":707,"replication_time_us":13982,"spinlock_wait_cycles":12735872}]]}
I20260925 13:57:20.660097 21776 tablet_service.cc:1511] Processing CreateTablet for tablet d9dbc7ac9d0142c8a92262d2117ee6ec (DEFAULT_TABLE table=default.table_2138b0cdfa7c4afda3a919dd0945a2d1 [id=012dc51aeefc492ca9aade1c75b97020]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:20.663815 21938 tablet_service.cc:1511] Processing CreateTablet for tablet d9dbc7ac9d0142c8a92262d2117ee6ec (DEFAULT_TABLE table=default.table_2138b0cdfa7c4afda3a919dd0945a2d1 [id=012dc51aeefc492ca9aade1c75b97020]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:20.664881 21938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9dbc7ac9d0142c8a92262d2117ee6ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:20.668327 22211 tablet_bootstrap.cc:492] T 195c48349fb5410495299c9e71a72ad8 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:20.673434 22211 tablet_bootstrap.cc:654] T 195c48349fb5410495299c9e71a72ad8 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:20.673379 22212 tablet_bootstrap.cc:492] T 1928234ffd3947a78e10d4c43e32aeb8 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:20.673789 22212 ts_tablet_manager.cc:1403] T 1928234ffd3947a78e10d4c43e32aeb8 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.127s user 0.013s sys 0.000s
I20260925 13:57:20.683697 22221 webserver.cc:533] Webserver started at http://127.18.182.253:33753/ using document root <none> and password file <none>
I20260925 13:57:20.685106 22221 fs_manager.cc:362] Metadata directory not provided
I20260925 13:57:20.685401 22221 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:57:20.692557 22211 tablet_bootstrap.cc:492] T 195c48349fb5410495299c9e71a72ad8 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:20.692919 22211 ts_tablet_manager.cc:1403] T 195c48349fb5410495299c9e71a72ad8 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.025s user 0.011s sys 0.000s
I20260925 13:57:20.694593 21584 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35994:
name: "default.Table_fc718993756d497eb9df6af8c1651362"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:57:20.695636 22211 raft_consensus.cc:359] T 195c48349fb5410495299c9e71a72ad8 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.696311 22211 raft_consensus.cc:385] T 195c48349fb5410495299c9e71a72ad8 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:20.696527 22211 raft_consensus.cc:740] T 195c48349fb5410495299c9e71a72ad8 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:20.697069 22293 tablet_bootstrap.cc:700] T 9745537e91d645fe9874373fe6237a9b P 5e2731975dbf433aade4a859df9699e2: Time spent opening tablet: real 0.107s user 0.001s sys 0.000s
I20260925 13:57:20.697167 22211 consensus_queue.cc:260] T 195c48349fb5410495299c9e71a72ad8 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.697744 22293 tablet_bootstrap.cc:654] T 9745537e91d645fe9874373fe6237a9b P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:20.698832 22211 ts_tablet_manager.cc:1434] T 195c48349fb5410495299c9e71a72ad8 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260925 13:57:20.698648 22212 raft_consensus.cc:359] T 1928234ffd3947a78e10d4c43e32aeb8 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.699301 22212 raft_consensus.cc:385] T 1928234ffd3947a78e10d4c43e32aeb8 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:20.699584 22212 raft_consensus.cc:740] T 1928234ffd3947a78e10d4c43e32aeb8 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:20.700326 22212 consensus_queue.cc:260] T 1928234ffd3947a78e10d4c43e32aeb8 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.706276 21795 raft_consensus.cc:2468] T 24dd39b302344f8d8f5166e9d641d13a P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 1.
I20260925 13:57:20.716756 22212 ts_tablet_manager.cc:1434] T 1928234ffd3947a78e10d4c43e32aeb8 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.043s user 0.007s sys 0.000s
I20260925 13:57:20.724261 22211 tablet_bootstrap.cc:492] T d9dbc7ac9d0142c8a92262d2117ee6ec P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:20.729748 22221 fs_manager.cc:714] Time spent opening directory manager: real 0.030s user 0.006s sys 0.002s
I20260925 13:57:20.730656 22211 tablet_bootstrap.cc:654] T d9dbc7ac9d0142c8a92262d2117ee6ec P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:20.732276 21776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9dbc7ac9d0142c8a92262d2117ee6ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:20.749150 22596 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:57:20.751704 22212 tablet_bootstrap.cc:492] T 195c48349fb5410495299c9e71a72ad8 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:20.753206 22214 ts_tablet_manager.cc:1929] T 3a8c53e379a9407c920d91b05ebd2afc P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:20.756310 22221 fs_manager.cc:730] Time spent opening block manager: real 0.023s user 0.005s sys 0.002s
I20260925 13:57:20.756659 22221 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-1/data,/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-1/wal
uuid: "3ea1e11bc6d84114b7616815b6af4635"
format_stamp: "Formatted at 2026-09-25 13:56:55 on dist-test-slave-nlqf"
I20260925 13:57:20.760197 22221 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:57:20.767225 21579 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet 9745537e91d645fe9874373fe6237a9b
I20260925 13:57:20.768602 22415 ts_tablet_manager.cc:1916] T f481b714a5f34fff8f74ba30bdd5dd35 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:20.780891 21775 tablet_service.cc:1558] Processing DeleteTablet for tablet 9745537e91d645fe9874373fe6237a9b with delete_type TABLET_DATA_DELETED (replaced by tablet b250001674694b48b3bc1a3491516da5) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:20.787786 22214 log.cc:1199] T 3a8c53e379a9407c920d91b05ebd2afc P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/3a8c53e379a9407c920d91b05ebd2afc
I20260925 13:57:20.788729 22214 ts_tablet_manager.cc:1950] T 3a8c53e379a9407c920d91b05ebd2afc P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:20.792786 21938 tablet_service.cc:1558] Processing DeleteTablet for tablet 9745537e91d645fe9874373fe6237a9b with delete_type TABLET_DATA_DELETED (replaced by tablet b250001674694b48b3bc1a3491516da5) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:20.793555 22416 tablet_replica.cc:333] T 9745537e91d645fe9874373fe6237a9b P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:20.794283 22416 raft_consensus.cc:2243] T 9745537e91d645fe9874373fe6237a9b P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:20.794370 21579 catalog_manager.cc:6934] ReplaceTablet: tablet 9745537e91d645fe9874373fe6237a9b deleted and replaced by tablet b250001674694b48b3bc1a3491516da5
I20260925 13:57:20.794732 22416 raft_consensus.cc:2272] T 9745537e91d645fe9874373fe6237a9b P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:20.796980 22416 ts_tablet_manager.cc:1916] T 9745537e91d645fe9874373fe6237a9b P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:20.802886 21643 catalog_manager.cc:6128] Not sending DeleteTablet requests; no consensus state for tablet b250001674694b48b3bc1a3491516da5
W20260925 13:57:20.806329 21581 catalog_manager.cc:5671] Ignoring report from unknown tablet 60256642160a44a2bab77b97d78c27a8
I20260925 13:57:20.784013 22085 tablet_service.cc:1511] Processing CreateTablet for tablet d9dbc7ac9d0142c8a92262d2117ee6ec (DEFAULT_TABLE table=default.table_2138b0cdfa7c4afda3a919dd0945a2d1 [id=012dc51aeefc492ca9aade1c75b97020]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:20.811791 22085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9dbc7ac9d0142c8a92262d2117ee6ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:20.814199 21777 tablet_service.cc:1511] Processing CreateTablet for tablet fb781a032ba649de8a214925fdb374b5 (DEFAULT_TABLE table=default.table_d8ce7cda4ecf4422a5e74c80d7ebf4da [id=af0f962d5d5d4a17a3f576ab6a3b4df0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:20.815133 21777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb781a032ba649de8a214925fdb374b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:20.822315 21775 tablet_service.cc:1558] Processing DeleteTablet for tablet f3b0e015d5dd4a668b6a04b49f628070 with delete_type TABLET_DATA_DELETED (replaced by tablet 195c48349fb5410495299c9e71a72ad8) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:20.758632 22212 tablet_bootstrap.cc:654] T 195c48349fb5410495299c9e71a72ad8 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:20.799613 22087 tablet_service.cc:1558] Processing DeleteTablet for tablet 9745537e91d645fe9874373fe6237a9b with delete_type TABLET_DATA_DELETED (replaced by tablet b250001674694b48b3bc1a3491516da5) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:20.828181 22416 ts_tablet_manager.cc:1929] T 9745537e91d645fe9874373fe6237a9b P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:20.828514 22416 log.cc:1199] T 9745537e91d645fe9874373fe6237a9b P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/9745537e91d645fe9874373fe6237a9b
I20260925 13:57:20.829123 22416 ts_tablet_manager.cc:1950] T 9745537e91d645fe9874373fe6237a9b P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:20.833739 22214 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50052 (request call id 107) took 1197 ms. Trace:
I20260925 13:57:20.834038 22214 rpcz_store.cc:276] 0925 13:57:19.636050 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:19.636141 (+ 91us) service_pool.cc:224] Handling call
0925 13:57:20.445676 (+809535us) ts_tablet_manager.cc:1151] Deleting tablet 3a8c53e379a9407c920d91b05ebd2afc
0925 13:57:20.445713 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:57:20.540514 (+ 94801us) tablet_metadata.cc:665] Metadata flushed
0925 13:57:20.753084 (+212570us) tablet_metadata.cc:665] Metadata flushed
0925 13:57:20.833687 (+ 80603us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":788486}
I20260925 13:57:20.840768 22293 tablet_bootstrap.cc:492] T 9745537e91d645fe9874373fe6237a9b P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:20.841189 22293 ts_tablet_manager.cc:1403] T 9745537e91d645fe9874373fe6237a9b P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.314s user 0.007s sys 0.001s
I20260925 13:57:20.843418 22293 raft_consensus.cc:359] T 9745537e91d645fe9874373fe6237a9b P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.844017 22293 raft_consensus.cc:385] T 9745537e91d645fe9874373fe6237a9b P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:20.844259 22293 raft_consensus.cc:740] T 9745537e91d645fe9874373fe6237a9b P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:20.852180 22211 tablet_bootstrap.cc:492] T d9dbc7ac9d0142c8a92262d2117ee6ec P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:20.852649 22211 ts_tablet_manager.cc:1403] T d9dbc7ac9d0142c8a92262d2117ee6ec P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.130s user 0.010s sys 0.005s
I20260925 13:57:20.854974 22211 raft_consensus.cc:359] T d9dbc7ac9d0142c8a92262d2117ee6ec P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.855662 22211 raft_consensus.cc:385] T d9dbc7ac9d0142c8a92262d2117ee6ec P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:20.855937 22211 raft_consensus.cc:740] T d9dbc7ac9d0142c8a92262d2117ee6ec P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:20.856653 22211 consensus_queue.cc:260] T d9dbc7ac9d0142c8a92262d2117ee6ec P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.859298 22293 consensus_queue.cc:260] T 9745537e91d645fe9874373fe6237a9b P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.864840 21938 tablet_service.cc:1511] Processing CreateTablet for tablet fb781a032ba649de8a214925fdb374b5 (DEFAULT_TABLE table=default.table_d8ce7cda4ecf4422a5e74c80d7ebf4da [id=af0f962d5d5d4a17a3f576ab6a3b4df0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:20.865799 21938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb781a032ba649de8a214925fdb374b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:20.866494 22293 ts_tablet_manager.cc:1434] T 9745537e91d645fe9874373fe6237a9b P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.025s user 0.003s sys 0.004s
I20260925 13:57:20.876295 22293 tablet_bootstrap.cc:492] T 195c48349fb5410495299c9e71a72ad8 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:20.890388 22211 ts_tablet_manager.cc:1434] T d9dbc7ac9d0142c8a92262d2117ee6ec P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.037s user 0.004s sys 0.000s
I20260925 13:57:20.898792 22293 tablet_bootstrap.cc:654] T 195c48349fb5410495299c9e71a72ad8 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
W20260925 13:57:20.862287 22040 consensus_peers.cc:597] T f66354b75eb8446396a27c3d563191cc P ee02f77594864349982492862d668a36 -> Peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Couldn't send request to peer 5e2731975dbf433aade4a859df9699e2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f66354b75eb8446396a27c3d563191cc. This is attempt 1: this message will repeat every 5th retry.
W20260925 13:57:20.908142 22040 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.193:39881, user_credentials={real_user=slave}} blocked reactor thread for 45964.1us
I20260925 13:57:20.888476 22087 tablet_service.cc:1511] Processing CreateTablet for tablet fb781a032ba649de8a214925fdb374b5 (DEFAULT_TABLE table=default.table_d8ce7cda4ecf4422a5e74c80d7ebf4da [id=af0f962d5d5d4a17a3f576ab6a3b4df0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:20.909336 22087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb781a032ba649de8a214925fdb374b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:20.920115 22293 tablet_bootstrap.cc:492] T 195c48349fb5410495299c9e71a72ad8 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:20.920507 22293 ts_tablet_manager.cc:1403] T 195c48349fb5410495299c9e71a72ad8 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.044s user 0.006s sys 0.003s
I20260925 13:57:20.921715 22214 tablet_replica.cc:333] T f66354b75eb8446396a27c3d563191cc P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:20.922488 22214 raft_consensus.cc:2243] T f66354b75eb8446396a27c3d563191cc P ee02f77594864349982492862d668a36 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:20.922911 22547 raft_consensus.cc:493] T 195c48349fb5410495299c9e71a72ad8 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:20.923435 22547 raft_consensus.cc:515] T 195c48349fb5410495299c9e71a72ad8 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.925450 22547 leader_election.cc:290] T 195c48349fb5410495299c9e71a72ad8 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:20.927215 22293 raft_consensus.cc:359] T 195c48349fb5410495299c9e71a72ad8 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.927928 22293 raft_consensus.cc:385] T 195c48349fb5410495299c9e71a72ad8 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:20.928220 22293 raft_consensus.cc:740] T 195c48349fb5410495299c9e71a72ad8 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:20.928392 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "195c48349fb5410495299c9e71a72ad8" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:20.929056 22293 consensus_queue.cc:260] T 195c48349fb5410495299c9e71a72ad8 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:20.929823 21896 leader_election.cc:343] T 195c48349fb5410495299c9e71a72ad8 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:57:20.930758 22212 tablet_bootstrap.cc:492] T 195c48349fb5410495299c9e71a72ad8 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:20.931205 22212 ts_tablet_manager.cc:1403] T 195c48349fb5410495299c9e71a72ad8 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.186s user 0.009s sys 0.000s
I20260925 13:57:20.931339 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "195c48349fb5410495299c9e71a72ad8" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2" is_pre_election: true
I20260925 13:57:20.931913 21795 raft_consensus.cc:2468] T 195c48349fb5410495299c9e71a72ad8 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 0.
I20260925 13:57:20.933105 21893 leader_election.cc:304] T 195c48349fb5410495299c9e71a72ad8 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters: ee02f77594864349982492862d668a36
I20260925 13:57:20.934535 22547 raft_consensus.cc:2804] T 195c48349fb5410495299c9e71a72ad8 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:20.934851 22547 raft_consensus.cc:493] T 195c48349fb5410495299c9e71a72ad8 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:20.935184 22547 raft_consensus.cc:3060] T 195c48349fb5410495299c9e71a72ad8 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:20.936570 22293 ts_tablet_manager.cc:1434] T 195c48349fb5410495299c9e71a72ad8 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.016s user 0.005s sys 0.000s
I20260925 13:57:20.937544 22211 tablet_bootstrap.cc:492] T fb781a032ba649de8a214925fdb374b5 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:20.942595 22293 tablet_bootstrap.cc:492] T 1928234ffd3947a78e10d4c43e32aeb8 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:20.948117 22547 raft_consensus.cc:515] T 195c48349fb5410495299c9e71a72ad8 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.948570 22293 tablet_bootstrap.cc:654] T 1928234ffd3947a78e10d4c43e32aeb8 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:20.950721 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "195c48349fb5410495299c9e71a72ad8" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2"
I20260925 13:57:20.951252 21795 raft_consensus.cc:3060] T 195c48349fb5410495299c9e71a72ad8 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:20.921001 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet 3a8c53e379a9407c920d91b05ebd2afc (table default.table_b17685eaeada404790103bd89711435e [id=df6afcfb1e2c463493b14228e19ade26]) successfully deleted
I20260925 13:57:20.957823 21795 raft_consensus.cc:2468] T 195c48349fb5410495299c9e71a72ad8 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 1.
I20260925 13:57:20.958949 21893 leader_election.cc:304] T 195c48349fb5410495299c9e71a72ad8 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:20.959766 22517 raft_consensus.cc:2804] T 195c48349fb5410495299c9e71a72ad8 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:20.960120 22293 tablet_bootstrap.cc:492] T 1928234ffd3947a78e10d4c43e32aeb8 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:20.960207 22517 raft_consensus.cc:697] T 195c48349fb5410495299c9e71a72ad8 P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: Becoming Leader. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Running, Role: LEADER
I20260925 13:57:20.960480 22293 ts_tablet_manager.cc:1403] T 1928234ffd3947a78e10d4c43e32aeb8 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.018s user 0.013s sys 0.002s
I20260925 13:57:20.960961 22517 consensus_queue.cc:237] T 195c48349fb5410495299c9e71a72ad8 P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.962728 22293 raft_consensus.cc:359] T 1928234ffd3947a78e10d4c43e32aeb8 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.963507 22293 raft_consensus.cc:385] T 1928234ffd3947a78e10d4c43e32aeb8 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:20.963752 22293 raft_consensus.cc:740] T 1928234ffd3947a78e10d4c43e32aeb8 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:20.964411 22293 consensus_queue.cc:260] T 1928234ffd3947a78e10d4c43e32aeb8 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.967286 21583 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35994:
name: "default.Table_30772fdea84947818aff6f6522c73f4b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:57:20.971257 22293 ts_tablet_manager.cc:1434] T 1928234ffd3947a78e10d4c43e32aeb8 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20260925 13:57:20.971908 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "195c48349fb5410495299c9e71a72ad8" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
W20260925 13:57:20.973145 21896 leader_election.cc:343] T 195c48349fb5410495299c9e71a72ad8 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:57:20.974452 22293 tablet_bootstrap.cc:492] T fb781a032ba649de8a214925fdb374b5 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:20.975126 22214 pending_rounds.cc:70] T f66354b75eb8446396a27c3d563191cc P ee02f77594864349982492862d668a36: Trying to abort 1 pending ops.
I20260925 13:57:20.975467 22214 pending_rounds.cc:77] T f66354b75eb8446396a27c3d563191cc P ee02f77594864349982492862d668a36: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251644589215744 op_type: NO_OP noop_request { }
I20260925 13:57:20.976761 22214 raft_consensus.cc:2889] T f66354b75eb8446396a27c3d563191cc P ee02f77594864349982492862d668a36 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:57:20.977210 22214 raft_consensus.cc:2272] T f66354b75eb8446396a27c3d563191cc P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:20.979570 22293 tablet_bootstrap.cc:654] T fb781a032ba649de8a214925fdb374b5 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:20.979604 22547 leader_election.cc:290] T 195c48349fb5410495299c9e71a72ad8 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Requested vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:20.980057 22464 raft_consensus.cc:493] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:20.980463 22464 raft_consensus.cc:515] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.982285 22464 leader_election.cc:290] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 13:57:20.984539 22043 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
W20260925 13:57:20.985613 22040 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
I20260925 13:57:20.985960 22040 leader_election.cc:304] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:20.987481 22464 raft_consensus.cc:2749] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:57:20.988965 22212 raft_consensus.cc:359] T 195c48349fb5410495299c9e71a72ad8 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.990064 22212 raft_consensus.cc:385] T 195c48349fb5410495299c9e71a72ad8 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:20.990449 22212 raft_consensus.cc:740] T 195c48349fb5410495299c9e71a72ad8 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:20.994642 22579 raft_consensus.cc:493] T 1928234ffd3947a78e10d4c43e32aeb8 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:20.995095 22579 raft_consensus.cc:515] T 1928234ffd3947a78e10d4c43e32aeb8 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:20.997031 22579 leader_election.cc:290] T 1928234ffd3947a78e10d4c43e32aeb8 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:21.001072 22211 tablet_bootstrap.cc:654] T fb781a032ba649de8a214925fdb374b5 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:21.002916 21581 master_service.cc:960] ReplaceTablet: received request to replace tablet d267def3f0864435a0d2c40fd7052eab from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:21.002772 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1928234ffd3947a78e10d4c43e32aeb8" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:21.003389 22107 raft_consensus.cc:2468] T 1928234ffd3947a78e10d4c43e32aeb8 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:21.004547 21732 leader_election.cc:304] T 1928234ffd3947a78e10d4c43e32aeb8 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:21.005264 22579 raft_consensus.cc:2804] T 1928234ffd3947a78e10d4c43e32aeb8 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:21.005585 22579 raft_consensus.cc:493] T 1928234ffd3947a78e10d4c43e32aeb8 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:21.005901 22579 raft_consensus.cc:3060] T 1928234ffd3947a78e10d4c43e32aeb8 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:21.012722 22579 raft_consensus.cc:515] T 1928234ffd3947a78e10d4c43e32aeb8 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.013862 22212 consensus_queue.cc:260] T 195c48349fb5410495299c9e71a72ad8 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:21.005038 21643 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 56a39663479d40be8bbd1dfd4c91040c consensus state has no leader
I20260925 13:57:21.014963 22214 ts_tablet_manager.cc:1916] T f66354b75eb8446396a27c3d563191cc P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:21.019961 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1928234ffd3947a78e10d4c43e32aeb8" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:21.021171 22579 leader_election.cc:290] T 1928234ffd3947a78e10d4c43e32aeb8 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Requested vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:21.021364 22107 raft_consensus.cc:3060] T 1928234ffd3947a78e10d4c43e32aeb8 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:21.023007 22293 tablet_bootstrap.cc:492] T fb781a032ba649de8a214925fdb374b5 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:21.023404 22293 ts_tablet_manager.cc:1403] T fb781a032ba649de8a214925fdb374b5 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.049s user 0.014s sys 0.002s
I20260925 13:57:21.025671 22293 raft_consensus.cc:359] T fb781a032ba649de8a214925fdb374b5 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.026291 22293 raft_consensus.cc:385] T fb781a032ba649de8a214925fdb374b5 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:21.026531 22293 raft_consensus.cc:740] T fb781a032ba649de8a214925fdb374b5 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:21.027508 22293 consensus_queue.cc:260] T fb781a032ba649de8a214925fdb374b5 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.029137 22107 raft_consensus.cc:2468] T 1928234ffd3947a78e10d4c43e32aeb8 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:21.029387 22293 ts_tablet_manager.cc:1434] T fb781a032ba649de8a214925fdb374b5 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260925 13:57:21.030290 21732 leader_election.cc:304] T 1928234ffd3947a78e10d4c43e32aeb8 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:21.035270 22579 raft_consensus.cc:2804] T 1928234ffd3947a78e10d4c43e32aeb8 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:21.035748 22579 raft_consensus.cc:697] T 1928234ffd3947a78e10d4c43e32aeb8 P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Becoming Leader. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Running, Role: LEADER
I20260925 13:57:21.036763 22579 consensus_queue.cc:237] T 1928234ffd3947a78e10d4c43e32aeb8 P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.040941 22293 tablet_bootstrap.cc:492] T d9dbc7ac9d0142c8a92262d2117ee6ec P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:21.048249 22293 tablet_bootstrap.cc:654] T d9dbc7ac9d0142c8a92262d2117ee6ec P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:21.050540 21776 tablet_service.cc:1511] Processing CreateTablet for tablet 286159c4dd10418da42040e53c1814f1 (DEFAULT_TABLE table=default.table_dc864d7428e5405da360d2f379eef37b [id=bf8cef2239904fd989622f9426412635]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:21.051827 21776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 286159c4dd10418da42040e53c1814f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:21.055629 21777 tablet_service.cc:1511] Processing CreateTablet for tablet 794784c42d27441f829f954b6929a097 (DEFAULT_TABLE table=default.table_61d48f9061c741fc8aa556e3edd984fb [id=7377fa3ea9664b47a930d9c5f88c60fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:21.056509 21777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 794784c42d27441f829f954b6929a097. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:21.057554 22087 tablet_service.cc:1511] Processing CreateTablet for tablet 286159c4dd10418da42040e53c1814f1 (DEFAULT_TABLE table=default.table_dc864d7428e5405da360d2f379eef37b [id=bf8cef2239904fd989622f9426412635]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:21.060012 22087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 286159c4dd10418da42040e53c1814f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:21.065382 22293 tablet_bootstrap.cc:492] T d9dbc7ac9d0142c8a92262d2117ee6ec P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:21.065796 22293 ts_tablet_manager.cc:1403] T d9dbc7ac9d0142c8a92262d2117ee6ec P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.025s user 0.010s sys 0.000s
I20260925 13:57:21.068007 22293 raft_consensus.cc:359] T d9dbc7ac9d0142c8a92262d2117ee6ec P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.068588 22293 raft_consensus.cc:385] T d9dbc7ac9d0142c8a92262d2117ee6ec P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:21.068828 22293 raft_consensus.cc:740] T d9dbc7ac9d0142c8a92262d2117ee6ec P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:21.069499 22293 consensus_queue.cc:260] T d9dbc7ac9d0142c8a92262d2117ee6ec P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.073558 22293 ts_tablet_manager.cc:1434] T d9dbc7ac9d0142c8a92262d2117ee6ec P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.008s user 0.004s sys 0.000s
I20260925 13:57:21.077246 22085 tablet_service.cc:1511] Processing CreateTablet for tablet 794784c42d27441f829f954b6929a097 (DEFAULT_TABLE table=default.table_61d48f9061c741fc8aa556e3edd984fb [id=7377fa3ea9664b47a930d9c5f88c60fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:21.077589 22464 raft_consensus.cc:493] T 9745537e91d645fe9874373fe6237a9b P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:21.078019 22464 raft_consensus.cc:515] T 9745537e91d645fe9874373fe6237a9b P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.079833 22464 leader_election.cc:290] T 9745537e91d645fe9874373fe6237a9b P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 13:57:21.081981 22043 leader_election.cc:343] T 9745537e91d645fe9874373fe6237a9b P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 9745537e91d645fe9874373fe6237a9b
I20260925 13:57:21.082540 22212 ts_tablet_manager.cc:1434] T 195c48349fb5410495299c9e71a72ad8 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.151s user 0.006s sys 0.000s
I20260925 13:57:21.083340 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9745537e91d645fe9874373fe6237a9b" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2" is_pre_election: true
I20260925 13:57:21.083909 21795 raft_consensus.cc:2468] T 9745537e91d645fe9874373fe6237a9b P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee02f77594864349982492862d668a36 in term 0.
I20260925 13:57:21.086856 22040 leader_election.cc:304] T 9745537e91d645fe9874373fe6237a9b P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2, ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee
I20260925 13:57:21.088228 22464 raft_consensus.cc:2804] T 9745537e91d645fe9874373fe6237a9b P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:21.088552 22464 raft_consensus.cc:493] T 9745537e91d645fe9874373fe6237a9b P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:21.088845 22464 raft_consensus.cc:3060] T 9745537e91d645fe9874373fe6237a9b P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:21.094735 22212 tablet_bootstrap.cc:492] T fb781a032ba649de8a214925fdb374b5 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:21.101670 22464 raft_consensus.cc:515] T 9745537e91d645fe9874373fe6237a9b P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.102239 21588 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:35994 (request call id 69) took 4569 ms. Trace:
I20260925 13:57:21.001889 21958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1928234ffd3947a78e10d4c43e32aeb8" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:21.102708 21588 rpcz_store.cc:276] 0925 13:57:16.532540 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:16.532975 (+ 435us) service_pool.cc:224] Handling call
0925 13:57:16.541534 (+ 8559us) spinlock_profiling.cc:243] Waited 6.12 ms on lock 0x7b54000203d4. stack: 00007fd96d084323 00007fd96d084220 00007fd96cc28574 0000000000516964 00000000005168f9 00007fd9741825c0 00007fd9742216fa 00007fd975f92d8c 00007fd976092876 00007fd973c41b34 00007fd973c41ac2 00007fd973c419f1 00007fd973c4196c 00007fd973c40c22 00007fd971ee709c 00007fd971ee64d6
0925 13:57:18.481498 (+1939964us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 13:57:21.102176 (+2620678us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":89,"Hive Metastore.run_cpu_time_us":309,"Hive Metastore.run_wall_time_us":262112,"spinlock_wait_cycles":14080384}
I20260925 13:57:21.102468 22085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 794784c42d27441f829f954b6929a097. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:21.103698 21958 raft_consensus.cc:2468] T 1928234ffd3947a78e10d4c43e32aeb8 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:21.018065 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1928234ffd3947a78e10d4c43e32aeb8" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
I20260925 13:57:21.104677 21960 raft_consensus.cc:3060] T 1928234ffd3947a78e10d4c43e32aeb8 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:21.108829 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9745537e91d645fe9874373fe6237a9b" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2"
I20260925 13:57:21.109303 21795 raft_consensus.cc:3060] T 9745537e91d645fe9874373fe6237a9b P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:21.111611 22212 tablet_bootstrap.cc:654] T fb781a032ba649de8a214925fdb374b5 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:21.111832 21960 raft_consensus.cc:2468] T 1928234ffd3947a78e10d4c43e32aeb8 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:21.056317 21938 tablet_service.cc:1511] Processing CreateTablet for tablet 286159c4dd10418da42040e53c1814f1 (DEFAULT_TABLE table=default.table_dc864d7428e5405da360d2f379eef37b [id=bf8cef2239904fd989622f9426412635]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:21.073037 21940 tablet_service.cc:1511] Processing CreateTablet for tablet 794784c42d27441f829f954b6929a097 (DEFAULT_TABLE table=default.table_61d48f9061c741fc8aa556e3edd984fb [id=7377fa3ea9664b47a930d9c5f88c60fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:21.114334 21940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 794784c42d27441f829f954b6929a097. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:21.115957 22415 ts_tablet_manager.cc:1929] T f481b714a5f34fff8f74ba30bdd5dd35 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:21.116308 22415 log.cc:1199] T f481b714a5f34fff8f74ba30bdd5dd35 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/f481b714a5f34fff8f74ba30bdd5dd35
I20260925 13:57:21.116920 22415 ts_tablet_manager.cc:1950] T f481b714a5f34fff8f74ba30bdd5dd35 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:21.116885 21795 raft_consensus.cc:2468] T 9745537e91d645fe9874373fe6237a9b P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee02f77594864349982492862d668a36 in term 1.
I20260925 13:57:21.118086 22040 leader_election.cc:304] T 9745537e91d645fe9874373fe6237a9b P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:21.119109 22415 tablet_replica.cc:333] T 9745537e91d645fe9874373fe6237a9b P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:21.119289 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet f481b714a5f34fff8f74ba30bdd5dd35 (table default.table_61d48f9061c741fc8aa556e3edd984fb [id=7377fa3ea9664b47a930d9c5f88c60fa]) successfully deleted
I20260925 13:57:21.123528 21586 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:35994 (request call id 70) took 4590 ms. Trace:
I20260925 13:57:21.123893 21586 rpcz_store.cc:276] 0925 13:57:16.532848 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:16.533686 (+ 838us) service_pool.cc:224] Handling call
0925 13:57:16.545262 (+ 11576us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b54000203d4. stack: 00007fd96d084323 00007fd96d084220 00007fd96cc28574 0000000000516964 00000000005168f9 00007fd9741825c0 00007fd9742216fa 00007fd975f92d8c 00007fd976092876 00007fd973c41b34 00007fd973c41ac2 00007fd973c419f1 00007fd973c4196c 00007fd973c40c22 00007fd971ee709c 00007fd971ee64d6
0925 13:57:19.031118 (+2485856us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 13:57:21.123470 (+2092352us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":214,"Hive Metastore.run_cpu_time_us":335,"Hive Metastore.run_wall_time_us":350854,"spinlock_wait_cycles":24969984}
I20260925 13:57:21.125142 21586 master_service.cc:960] ReplaceTablet: received request to replace tablet f68ccd8a7f4a4e518c2584e579de0c87 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:21.125545 22464 leader_election.cc:290] T 9745537e91d645fe9874373fe6237a9b P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 election: Requested vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:21.125982 22464 raft_consensus.cc:2804] T 9745537e91d645fe9874373fe6237a9b P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:21.126331 22464 raft_consensus.cc:697] T 9745537e91d645fe9874373fe6237a9b P ee02f77594864349982492862d668a36 [term 1 LEADER]: Becoming Leader. State: Replica: ee02f77594864349982492862d668a36, State: Running, Role: LEADER
I20260925 13:57:21.128468 22547 consensus_queue.cc:1048] T 76d4d7d706824892b075ab1aa4fbaf85 P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 13:57:21.132216 21580 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_f644beebc9834408bed8d57d2ac5c2ee in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 13:57:21.136399 22043 leader_election.cc:343] T 9745537e91d645fe9874373fe6237a9b P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 9745537e91d645fe9874373fe6237a9b
I20260925 13:57:21.136780 22415 raft_consensus.cc:2243] T 9745537e91d645fe9874373fe6237a9b P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260925 13:57:21.136982 21583 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_30772fdea84947818aff6f6522c73f4b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:57:21.137212 22415 raft_consensus.cc:2272] T 9745537e91d645fe9874373fe6237a9b P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 13:57:21.137341 21584 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_fc718993756d497eb9df6af8c1651362 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:57:21.137950 21938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 286159c4dd10418da42040e53c1814f1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:57:21.138775 21582 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_1baae11db5a24d50bd596853233007cb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:57:21.131137 21587 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:35994 (request call id 71) took 4563 ms. Trace:
I20260925 13:57:21.140944 21587 rpcz_store.cc:276] 0925 13:57:16.568402 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:16.571332 (+ 2930us) service_pool.cc:224] Handling call
0925 13:57:19.031168 (+2459836us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 13:57:21.128423 (+2097255us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 13:57:21.130217 (+ 1794us) catalog_manager.cc:3764] Apply alter schema
0925 13:57:21.131070 (+ 853us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":350879,"Hive Metastore.run_cpu_time_us":1822,"Hive Metastore.run_wall_time_us":600897}
I20260925 13:57:21.143023 22415 ts_tablet_manager.cc:1916] T 9745537e91d645fe9874373fe6237a9b P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:21.145634 22211 tablet_bootstrap.cc:492] T fb781a032ba649de8a214925fdb374b5 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:21.146034 22211 ts_tablet_manager.cc:1403] T fb781a032ba649de8a214925fdb374b5 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.209s user 0.008s sys 0.004s
I20260925 13:57:21.127038 22464 consensus_queue.cc:237] T 9745537e91d645fe9874373fe6237a9b P ee02f77594864349982492862d668a36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:21.160785 22040 consensus_peers.cc:597] T f3b0e015d5dd4a668b6a04b49f628070 P ee02f77594864349982492862d668a36 -> Peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Couldn't send request to peer 5e2731975dbf433aade4a859df9699e2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f3b0e015d5dd4a668b6a04b49f628070. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:57:21.162818 21586 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet f68ccd8a7f4a4e518c2584e579de0c87
I20260925 13:57:21.162986 22293 tablet_bootstrap.cc:492] T 286159c4dd10418da42040e53c1814f1 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:21.165221 21775 tablet_service.cc:1558] Processing DeleteTablet for tablet f68ccd8a7f4a4e518c2584e579de0c87 with delete_type TABLET_DATA_DELETED (replaced by tablet c42767d8f674410188835ceefa644eb5) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:21.165381 21937 tablet_service.cc:1558] Processing DeleteTablet for tablet f68ccd8a7f4a4e518c2584e579de0c87 with delete_type TABLET_DATA_DELETED (replaced by tablet c42767d8f674410188835ceefa644eb5) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:21.165987 22416 tablet_replica.cc:333] T f68ccd8a7f4a4e518c2584e579de0c87 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:21.166664 22416 raft_consensus.cc:2243] T f68ccd8a7f4a4e518c2584e579de0c87 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:21.167160 22416 raft_consensus.cc:2272] T f68ccd8a7f4a4e518c2584e579de0c87 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:21.167913 22547 consensus_queue.cc:1048] T 76d4d7d706824892b075ab1aa4fbaf85 P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:21.170452 22293 tablet_bootstrap.cc:654] T 286159c4dd10418da42040e53c1814f1 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:21.174220 22416 ts_tablet_manager.cc:1916] T f68ccd8a7f4a4e518c2584e579de0c87 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:21.148320 22211 raft_consensus.cc:359] T fb781a032ba649de8a214925fdb374b5 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.195840 22211 raft_consensus.cc:385] T fb781a032ba649de8a214925fdb374b5 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:21.196082 22211 raft_consensus.cc:740] T fb781a032ba649de8a214925fdb374b5 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:21.196703 22211 consensus_queue.cc:260] T fb781a032ba649de8a214925fdb374b5 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:21.200260 21572 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 56a39663479d40be8bbd1dfd4c91040c (table default.table_05ac73cb4d774c0fbd89f99093663e76 [id=df6afcfb1e2c463493b14228e19ade26], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 56a39663479d40be8bbd1dfd4c91040c consensus state has no leader
I20260925 13:57:21.200760 22214 ts_tablet_manager.cc:1929] T f66354b75eb8446396a27c3d563191cc P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:21.201133 22214 log.cc:1199] T f66354b75eb8446396a27c3d563191cc P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/f66354b75eb8446396a27c3d563191cc
I20260925 13:57:21.201588 22211 ts_tablet_manager.cc:1434] T fb781a032ba649de8a214925fdb374b5 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.055s user 0.006s sys 0.001s
I20260925 13:57:21.206130 22212 tablet_bootstrap.cc:492] T fb781a032ba649de8a214925fdb374b5 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:21.206511 22212 ts_tablet_manager.cc:1403] T fb781a032ba649de8a214925fdb374b5 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.112s user 0.007s sys 0.003s
I20260925 13:57:21.207142 22214 ts_tablet_manager.cc:1950] T f66354b75eb8446396a27c3d563191cc P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:21.209740 22214 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50052 (request call id 108) took 1500 ms. Trace:
I20260925 13:57:21.210042 22214 rpcz_store.cc:276] 0925 13:57:19.708978 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:19.719719 (+ 10741us) service_pool.cc:224] Handling call
0925 13:57:20.921569 (+1201850us) ts_tablet_manager.cc:1151] Deleting tablet f66354b75eb8446396a27c3d563191cc
0925 13:57:20.921611 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:57:21.144463 (+222852us) tablet_metadata.cc:665] Metadata flushed
0925 13:57:21.200646 (+ 56183us) tablet_metadata.cc:665] Metadata flushed
0925 13:57:21.209686 (+ 9040us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1194033}
I20260925 13:57:21.213507 22212 raft_consensus.cc:359] T fb781a032ba649de8a214925fdb374b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.214116 22212 raft_consensus.cc:385] T fb781a032ba649de8a214925fdb374b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:21.214353 22212 raft_consensus.cc:740] T fb781a032ba649de8a214925fdb374b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:21.219564 22214 tablet_replica.cc:333] T f3b0e015d5dd4a668b6a04b49f628070 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:21.220108 21586 catalog_manager.cc:6934] ReplaceTablet: tablet f68ccd8a7f4a4e518c2584e579de0c87 deleted and replaced by tablet c42767d8f674410188835ceefa644eb5
I20260925 13:57:21.220876 22085 tablet_service.cc:1558] Processing DeleteTablet for tablet f68ccd8a7f4a4e518c2584e579de0c87 with delete_type TABLET_DATA_DELETED (replaced by tablet c42767d8f674410188835ceefa644eb5) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:21.221577 22212 consensus_queue.cc:260] T fb781a032ba649de8a214925fdb374b5 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.224265 22214 raft_consensus.cc:2243] T f3b0e015d5dd4a668b6a04b49f628070 P ee02f77594864349982492862d668a36 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:21.224962 22214 raft_consensus.cc:2272] T f3b0e015d5dd4a668b6a04b49f628070 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:21.228950 22212 ts_tablet_manager.cc:1434] T fb781a032ba649de8a214925fdb374b5 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.022s user 0.004s sys 0.002s
I20260925 13:57:21.233251 22221 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260925 13:57:21.235824 22293 tablet_bootstrap.cc:492] T 286159c4dd10418da42040e53c1814f1 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:21.236119 22212 tablet_bootstrap.cc:492] T d9dbc7ac9d0142c8a92262d2117ee6ec P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:21.236349 22293 ts_tablet_manager.cc:1403] T 286159c4dd10418da42040e53c1814f1 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.073s user 0.012s sys 0.001s
I20260925 13:57:21.235700 22214 ts_tablet_manager.cc:1916] T f3b0e015d5dd4a668b6a04b49f628070 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:21.243278 22293 raft_consensus.cc:359] T 286159c4dd10418da42040e53c1814f1 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.244069 22212 tablet_bootstrap.cc:654] T d9dbc7ac9d0142c8a92262d2117ee6ec P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:21.244287 22211 tablet_bootstrap.cc:492] T 794784c42d27441f829f954b6929a097 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:21.252704 22416 ts_tablet_manager.cc:1929] T f68ccd8a7f4a4e518c2584e579de0c87 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:21.253028 22416 log.cc:1199] T f68ccd8a7f4a4e518c2584e579de0c87 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/f68ccd8a7f4a4e518c2584e579de0c87
I20260925 13:57:21.253650 22416 ts_tablet_manager.cc:1950] T f68ccd8a7f4a4e518c2584e579de0c87 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:21.235301 22221 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:57:21.255443 22293 raft_consensus.cc:385] T 286159c4dd10418da42040e53c1814f1 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:21.255694 22293 raft_consensus.cc:740] T 286159c4dd10418da42040e53c1814f1 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:21.255930 22221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:57:21.256462 22293 consensus_queue.cc:260] T 286159c4dd10418da42040e53c1814f1 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.267733 22214 ts_tablet_manager.cc:1929] T f3b0e015d5dd4a668b6a04b49f628070 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:21.268234 22214 log.cc:1199] T f3b0e015d5dd4a668b6a04b49f628070 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/f3b0e015d5dd4a668b6a04b49f628070
I20260925 13:57:21.269912 22293 ts_tablet_manager.cc:1434] T 286159c4dd10418da42040e53c1814f1 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.033s user 0.006s sys 0.000s
I20260925 13:57:21.270907 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet f68ccd8a7f4a4e518c2584e579de0c87 (table default.table_b54e9b49662b4a8e892615fec1080a87 [id=722129e8c51a45f8ae31843d0d3f0a47]) successfully deleted
I20260925 13:57:21.271616 22211 tablet_bootstrap.cc:654] T 794784c42d27441f829f954b6929a097 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:21.275848 21586 master_service.cc:960] ReplaceTablet: received request to replace tablet 286159c4dd10418da42040e53c1814f1 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:21.278986 22293 tablet_bootstrap.cc:492] T 794784c42d27441f829f954b6929a097 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:21.283478 22212 tablet_bootstrap.cc:492] T d9dbc7ac9d0142c8a92262d2117ee6ec P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:21.283946 22212 ts_tablet_manager.cc:1403] T d9dbc7ac9d0142c8a92262d2117ee6ec P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.048s user 0.002s sys 0.009s
I20260925 13:57:21.287057 22212 raft_consensus.cc:359] T d9dbc7ac9d0142c8a92262d2117ee6ec P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.289072 22212 raft_consensus.cc:385] T d9dbc7ac9d0142c8a92262d2117ee6ec P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:21.289589 22214 ts_tablet_manager.cc:1950] T f3b0e015d5dd4a668b6a04b49f628070 P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:21.291834 22415 ts_tablet_manager.cc:1929] T 9745537e91d645fe9874373fe6237a9b P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:21.292200 22415 log.cc:1199] T 9745537e91d645fe9874373fe6237a9b P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/9745537e91d645fe9874373fe6237a9b
I20260925 13:57:21.292819 22415 ts_tablet_manager.cc:1950] T 9745537e91d645fe9874373fe6237a9b P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:21.289830 22212 raft_consensus.cc:740] T d9dbc7ac9d0142c8a92262d2117ee6ec P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:21.298096 22212 consensus_queue.cc:260] T d9dbc7ac9d0142c8a92262d2117ee6ec P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.305208 22214 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50052 (request call id 111) took 1073 ms. Trace:
I20260925 13:57:21.307163 22214 rpcz_store.cc:276] 0925 13:57:20.231629 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:20.232147 (+ 518us) service_pool.cc:224] Handling call
0925 13:57:21.219422 (+987275us) ts_tablet_manager.cc:1151] Deleting tablet f3b0e015d5dd4a668b6a04b49f628070
0925 13:57:21.219465 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:57:21.251590 (+ 32125us) tablet_metadata.cc:665] Metadata flushed
0925 13:57:21.267617 (+ 16027us) tablet_metadata.cc:665] Metadata flushed
0925 13:57:21.305154 (+ 37537us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":986743}
I20260925 13:57:21.311733 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet f3b0e015d5dd4a668b6a04b49f628070 (table default.table_bd714c618f7a4f16a69a69d5932b2014 [id=dfd05c11872140cfa86b2628c33ef6d8]) successfully deleted
I20260925 13:57:21.308380 22212 ts_tablet_manager.cc:1434] T d9dbc7ac9d0142c8a92262d2117ee6ec P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.024s user 0.005s sys 0.000s
I20260925 13:57:21.312530 22214 tablet_replica.cc:333] T f481b714a5f34fff8f74ba30bdd5dd35 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:21.314567 22214 raft_consensus.cc:2243] T f481b714a5f34fff8f74ba30bdd5dd35 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:21.313786 22212 tablet_bootstrap.cc:492] T 794784c42d27441f829f954b6929a097 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:21.315261 21586 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet 286159c4dd10418da42040e53c1814f1
I20260925 13:57:21.317358 21938 tablet_service.cc:1558] Processing DeleteTablet for tablet 286159c4dd10418da42040e53c1814f1 with delete_type TABLET_DATA_DELETED (replaced by tablet ef57aa72abb2405b8187a7ed56d57f96) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:21.319140 22087 tablet_service.cc:1558] Processing DeleteTablet for tablet 286159c4dd10418da42040e53c1814f1 with delete_type TABLET_DATA_DELETED (replaced by tablet ef57aa72abb2405b8187a7ed56d57f96) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:21.319530 22214 raft_consensus.cc:2272] T f481b714a5f34fff8f74ba30bdd5dd35 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 13:57:21.321223 21575 catalog_manager.cc:5043] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): delete failed for tablet 286159c4dd10418da42040e53c1814f1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 286159c4dd10418da42040e53c1814f1 already in progress: creating tablet
I20260925 13:57:21.322070 21586 catalog_manager.cc:6934] ReplaceTablet: tablet 286159c4dd10418da42040e53c1814f1 deleted and replaced by tablet ef57aa72abb2405b8187a7ed56d57f96
I20260925 13:57:21.328649 22214 ts_tablet_manager.cc:1916] T f481b714a5f34fff8f74ba30bdd5dd35 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:21.329082 21777 tablet_service.cc:1558] Processing DeleteTablet for tablet 286159c4dd10418da42040e53c1814f1 with delete_type TABLET_DATA_DELETED (replaced by tablet ef57aa72abb2405b8187a7ed56d57f96) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:21.336897 22293 tablet_bootstrap.cc:654] T 794784c42d27441f829f954b6929a097 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:21.338920 21579 catalog_manager.cc:5838] T 56a39663479d40be8bbd1dfd4c91040c P 145e520e0a5a445d943a53cf8c27efee reported cstate change: term changed from 0 to 1, leader changed from <none> to 145e520e0a5a445d943a53cf8c27efee (127.18.182.194). New cstate: current_term: 1 leader_uuid: "145e520e0a5a445d943a53cf8c27efee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: UNKNOWN } } }
I20260925 13:57:21.341297 21579 catalog_manager.cc:5938] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291) does not have the latest schema for tablet 56a39663479d40be8bbd1dfd4c91040c (table default.table_05ac73cb4d774c0fbd89f99093663e76 [id=df6afcfb1e2c463493b14228e19ade26]). Expected version 1 got 0
I20260925 13:57:21.342090 21579 catalog_manager.cc:5838] T 24dd39b302344f8d8f5166e9d641d13a P 145e520e0a5a445d943a53cf8c27efee reported cstate change: term changed from 0 to 1, leader changed from <none> to 145e520e0a5a445d943a53cf8c27efee (127.18.182.194). New cstate: current_term: 1 leader_uuid: "145e520e0a5a445d943a53cf8c27efee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: UNKNOWN } } }
W20260925 13:57:21.345790 21572 catalog_manager.cc:5030] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): delete failed for tablet f3b0e015d5dd4a668b6a04b49f628070 because the tablet was not found. No further retry: Not found: Tablet not found: f3b0e015d5dd4a668b6a04b49f628070
I20260925 13:57:21.347368 22212 tablet_bootstrap.cc:654] T 794784c42d27441f829f954b6929a097 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:21.347630 21579 catalog_manager.cc:5838] T 195c48349fb5410495299c9e71a72ad8 P 145e520e0a5a445d943a53cf8c27efee reported cstate change: term changed from 0 to 1, leader changed from <none> to 145e520e0a5a445d943a53cf8c27efee (127.18.182.194). New cstate: current_term: 1 leader_uuid: "145e520e0a5a445d943a53cf8c27efee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: UNKNOWN } } }
I20260925 13:57:21.350766 22415 tablet_replica.cc:333] T f68ccd8a7f4a4e518c2584e579de0c87 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:21.351588 22415 raft_consensus.cc:2243] T f68ccd8a7f4a4e518c2584e579de0c87 P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:21.352618 22415 raft_consensus.cc:2272] T f68ccd8a7f4a4e518c2584e579de0c87 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:21.358723 21777 tablet_service.cc:1558] Processing DeleteTablet for tablet 9745537e91d645fe9874373fe6237a9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:57:20 UTC) from {username='slave'} at 127.0.0.1:34008
I20260925 13:57:21.359131 22579 raft_consensus.cc:493] T d9dbc7ac9d0142c8a92262d2117ee6ec P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260925 13:57:21.359807 21573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.253:46177, user_credentials={real_user=slave}} blocked reactor thread for 76831.2us
I20260925 13:57:21.359580 22579 raft_consensus.cc:515] T d9dbc7ac9d0142c8a92262d2117ee6ec P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.361532 22579 leader_election.cc:290] T d9dbc7ac9d0142c8a92262d2117ee6ec P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:21.363608 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9dbc7ac9d0142c8a92262d2117ee6ec" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:21.363762 21587 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35994:
table { table_name: "default.Table_8008d16a92cf4c0dafb258a1122bffb5" } modify_external_catalogs: true
I20260925 13:57:21.364172 21960 raft_consensus.cc:2468] T d9dbc7ac9d0142c8a92262d2117ee6ec P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:21.364447 21587 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35994:
table { table_name: "default.Table_8008d16a92cf4c0dafb258a1122bffb5" } modify_external_catalogs: true
I20260925 13:57:21.365142 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9dbc7ac9d0142c8a92262d2117ee6ec" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:21.365708 22107 raft_consensus.cc:2468] T d9dbc7ac9d0142c8a92262d2117ee6ec P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
W20260925 13:57:21.365862 21573 consensus_peers.cc:597] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 -> Peer 3ea1e11bc6d84114b7616815b6af4635 (127.18.182.253:46177): Couldn't send request to peer 3ea1e11bc6d84114b7616815b6af4635. Status: Network error: Client connection negotiation failed: client connection to 127.18.182.253:46177: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260925 13:57:21.367298 21732 leader_election.cc:304] T d9dbc7ac9d0142c8a92262d2117ee6ec P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:21.368198 21588 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35994:
table { table_name: "default.Table_fce30b75d46f47089d7a31e04841dcdf" } modify_external_catalogs: true
I20260925 13:57:21.368510 22579 raft_consensus.cc:2804] T d9dbc7ac9d0142c8a92262d2117ee6ec P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:21.368824 22579 raft_consensus.cc:493] T d9dbc7ac9d0142c8a92262d2117ee6ec P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:21.369156 22579 raft_consensus.cc:3060] T d9dbc7ac9d0142c8a92262d2117ee6ec P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:21.369048 21588 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35994:
table { table_name: "default.Table_fce30b75d46f47089d7a31e04841dcdf" } modify_external_catalogs: true
I20260925 13:57:21.375398 22579 raft_consensus.cc:515] T d9dbc7ac9d0142c8a92262d2117ee6ec P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.377233 22579 leader_election.cc:290] T d9dbc7ac9d0142c8a92262d2117ee6ec P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Requested vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:21.379122 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9dbc7ac9d0142c8a92262d2117ee6ec" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
I20260925 13:57:21.379652 21960 raft_consensus.cc:3060] T d9dbc7ac9d0142c8a92262d2117ee6ec P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:21.379904 22415 ts_tablet_manager.cc:1916] T f68ccd8a7f4a4e518c2584e579de0c87 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:21.380878 22293 tablet_bootstrap.cc:492] T 794784c42d27441f829f954b6929a097 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:21.381254 22293 ts_tablet_manager.cc:1403] T 794784c42d27441f829f954b6929a097 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.102s user 0.006s sys 0.007s
I20260925 13:57:21.381858 22212 tablet_bootstrap.cc:492] T 794784c42d27441f829f954b6929a097 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:21.382267 22212 ts_tablet_manager.cc:1403] T 794784c42d27441f829f954b6929a097 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.069s user 0.009s sys 0.000s
I20260925 13:57:21.382768 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9dbc7ac9d0142c8a92262d2117ee6ec" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:21.383257 22107 raft_consensus.cc:3060] T d9dbc7ac9d0142c8a92262d2117ee6ec P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:21.390628 22293 raft_consensus.cc:359] T 794784c42d27441f829f954b6929a097 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.391431 22293 raft_consensus.cc:385] T 794784c42d27441f829f954b6929a097 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:21.391723 22293 raft_consensus.cc:740] T 794784c42d27441f829f954b6929a097 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:21.392445 22293 consensus_queue.cc:260] T 794784c42d27441f829f954b6929a097 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.393769 21960 raft_consensus.cc:2468] T d9dbc7ac9d0142c8a92262d2117ee6ec P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:21.394812 21732 leader_election.cc:304] T d9dbc7ac9d0142c8a92262d2117ee6ec P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:21.394960 22212 raft_consensus.cc:359] T 794784c42d27441f829f954b6929a097 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.395599 22212 raft_consensus.cc:385] T 794784c42d27441f829f954b6929a097 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:21.395692 22579 raft_consensus.cc:2804] T d9dbc7ac9d0142c8a92262d2117ee6ec P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:21.395843 22212 raft_consensus.cc:740] T 794784c42d27441f829f954b6929a097 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:21.396111 22579 raft_consensus.cc:697] T d9dbc7ac9d0142c8a92262d2117ee6ec P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Becoming Leader. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Running, Role: LEADER
I20260925 13:57:21.396836 22579 consensus_queue.cc:237] T d9dbc7ac9d0142c8a92262d2117ee6ec P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.402391 22107 raft_consensus.cc:2468] T d9dbc7ac9d0142c8a92262d2117ee6ec P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:21.408393 22211 tablet_bootstrap.cc:492] T 794784c42d27441f829f954b6929a097 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:21.408816 22211 ts_tablet_manager.cc:1403] T 794784c42d27441f829f954b6929a097 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.165s user 0.012s sys 0.007s
I20260925 13:57:21.413502 22293 ts_tablet_manager.cc:1434] T 794784c42d27441f829f954b6929a097 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.032s user 0.006s sys 0.000s
I20260925 13:57:21.414445 21581 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35994:
table { table_name: "default.Table_bd714c618f7a4f16a69a69d5932b2014" } new_table_name: "default.Table_97e22ed3359342caaff80295b5d37db2" modify_external_catalogs: true
I20260925 13:57:21.414907 22211 raft_consensus.cc:359] T 794784c42d27441f829f954b6929a097 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.415587 22211 raft_consensus.cc:385] T 794784c42d27441f829f954b6929a097 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:21.415884 22211 raft_consensus.cc:740] T 794784c42d27441f829f954b6929a097 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:21.420197 21938 tablet_service.cc:1511] Processing CreateTablet for tablet ef57aa72abb2405b8187a7ed56d57f96 (DEFAULT_TABLE table=default.table_dc864d7428e5405da360d2f379eef37b [id=bf8cef2239904fd989622f9426412635]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:21.420486 22211 consensus_queue.cc:260] T 794784c42d27441f829f954b6929a097 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.421228 21938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef57aa72abb2405b8187a7ed56d57f96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:21.423245 22212 consensus_queue.cc:260] T 794784c42d27441f829f954b6929a097 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.423979 21777 tablet_service.cc:1511] Processing CreateTablet for tablet ef57aa72abb2405b8187a7ed56d57f96 (DEFAULT_TABLE table=default.table_dc864d7428e5405da360d2f379eef37b [id=bf8cef2239904fd989622f9426412635]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:21.424814 21777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef57aa72abb2405b8187a7ed56d57f96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:21.436139 22212 ts_tablet_manager.cc:1434] T 794784c42d27441f829f954b6929a097 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.054s user 0.007s sys 0.000s
I20260925 13:57:21.436504 22608 raft_consensus.cc:493] T fb781a032ba649de8a214925fdb374b5 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:21.436908 22608 raft_consensus.cc:515] T fb781a032ba649de8a214925fdb374b5 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.438290 22212 tablet_bootstrap.cc:492] T 286159c4dd10418da42040e53c1814f1 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:21.438735 22608 leader_election.cc:290] T fb781a032ba649de8a214925fdb374b5 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:21.427230 22214 ts_tablet_manager.cc:1929] T f481b714a5f34fff8f74ba30bdd5dd35 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:21.442207 22211 ts_tablet_manager.cc:1434] T 794784c42d27441f829f954b6929a097 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.033s user 0.006s sys 0.000s
I20260925 13:57:21.442262 22214 log.cc:1199] T f481b714a5f34fff8f74ba30bdd5dd35 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/f481b714a5f34fff8f74ba30bdd5dd35
I20260925 13:57:21.442854 22214 ts_tablet_manager.cc:1950] T f481b714a5f34fff8f74ba30bdd5dd35 P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:21.445012 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb781a032ba649de8a214925fdb374b5" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:21.445608 22107 raft_consensus.cc:2468] T fb781a032ba649de8a214925fdb374b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:21.446645 21732 leader_election.cc:304] T fb781a032ba649de8a214925fdb374b5 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:21.447404 22608 raft_consensus.cc:2804] T fb781a032ba649de8a214925fdb374b5 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:21.447696 22608 raft_consensus.cc:493] T fb781a032ba649de8a214925fdb374b5 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:21.447991 22608 raft_consensus.cc:3060] T fb781a032ba649de8a214925fdb374b5 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:21.448800 22214 tablet_replica.cc:333] T 9745537e91d645fe9874373fe6237a9b P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:21.448954 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet f481b714a5f34fff8f74ba30bdd5dd35 (table default.table_61d48f9061c741fc8aa556e3edd984fb [id=7377fa3ea9664b47a930d9c5f88c60fa]) successfully deleted
I20260925 13:57:21.449468 22214 raft_consensus.cc:2243] T 9745537e91d645fe9874373fe6237a9b P ee02f77594864349982492862d668a36 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:21.450066 22214 pending_rounds.cc:70] T 9745537e91d645fe9874373fe6237a9b P ee02f77594864349982492862d668a36: Trying to abort 1 pending ops.
I20260925 13:57:21.450258 22214 pending_rounds.cc:77] T 9745537e91d645fe9874373fe6237a9b P ee02f77594864349982492862d668a36: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333251650216329216 op_type: NO_OP noop_request { }
I20260925 13:57:21.450430 22211 tablet_bootstrap.cc:492] T 286159c4dd10418da42040e53c1814f1 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:21.450542 22214 raft_consensus.cc:2889] T 9745537e91d645fe9874373fe6237a9b P ee02f77594864349982492862d668a36 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 13:57:21.450832 22214 raft_consensus.cc:2272] T 9745537e91d645fe9874373fe6237a9b P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:21.454272 22608 raft_consensus.cc:515] T fb781a032ba649de8a214925fdb374b5 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.456843 21958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb781a032ba649de8a214925fdb374b5" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
I20260925 13:57:21.457345 21958 raft_consensus.cc:3060] T fb781a032ba649de8a214925fdb374b5 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:21.458005 22212 tablet_bootstrap.cc:654] T 286159c4dd10418da42040e53c1814f1 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:21.464020 21958 raft_consensus.cc:2468] T fb781a032ba649de8a214925fdb374b5 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:21.466456 22211 tablet_bootstrap.cc:654] T 286159c4dd10418da42040e53c1814f1 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:21.466460 22214 ts_tablet_manager.cc:1916] T 9745537e91d645fe9874373fe6237a9b P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:21.469368 21732 leader_election.cc:304] T fb781a032ba649de8a214925fdb374b5 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:21.455267 22415 ts_tablet_manager.cc:1929] T f68ccd8a7f4a4e518c2584e579de0c87 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:21.471642 22415 log.cc:1199] T f68ccd8a7f4a4e518c2584e579de0c87 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/f68ccd8a7f4a4e518c2584e579de0c87
I20260925 13:57:21.472288 22415 ts_tablet_manager.cc:1950] T f68ccd8a7f4a4e518c2584e579de0c87 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:21.473722 22087 tablet_service.cc:1511] Processing CreateTablet for tablet ef57aa72abb2405b8187a7ed56d57f96 (DEFAULT_TABLE table=default.table_dc864d7428e5405da360d2f379eef37b [id=bf8cef2239904fd989622f9426412635]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:21.474625 22087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef57aa72abb2405b8187a7ed56d57f96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:21.474946 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet f68ccd8a7f4a4e518c2584e579de0c87 (table default.table_b54e9b49662b4a8e892615fec1080a87 [id=722129e8c51a45f8ae31843d0d3f0a47]) successfully deleted
I20260925 13:57:21.474529 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb781a032ba649de8a214925fdb374b5" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:21.476112 21960 raft_consensus.cc:2376] T fb781a032ba649de8a214925fdb374b5 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1. Re-sending same reply.
I20260925 13:57:21.476253 22415 tablet_replica.cc:333] T 286159c4dd10418da42040e53c1814f1 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:21.476831 22415 raft_consensus.cc:2243] T 286159c4dd10418da42040e53c1814f1 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:21.477213 22415 raft_consensus.cc:2272] T 286159c4dd10418da42040e53c1814f1 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:21.480414 22608 leader_election.cc:290] T fb781a032ba649de8a214925fdb374b5 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Requested vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:21.457655 21586 catalog_manager.cc:5838] T 1928234ffd3947a78e10d4c43e32aeb8 P 5e2731975dbf433aade4a859df9699e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e2731975dbf433aade4a859df9699e2 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "5e2731975dbf433aade4a859df9699e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: UNKNOWN } } }
I20260925 13:57:21.493198 21940 tablet_service.cc:1511] Processing CreateTablet for tablet c42767d8f674410188835ceefa644eb5 (DEFAULT_TABLE table=default.table_b54e9b49662b4a8e892615fec1080a87 [id=722129e8c51a45f8ae31843d0d3f0a47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:21.494063 21940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c42767d8f674410188835ceefa644eb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:21.494014 22085 tablet_service.cc:1511] Processing CreateTablet for tablet c42767d8f674410188835ceefa644eb5 (DEFAULT_TABLE table=default.table_b54e9b49662b4a8e892615fec1080a87 [id=722129e8c51a45f8ae31843d0d3f0a47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:21.494863 22085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c42767d8f674410188835ceefa644eb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:21.497417 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb781a032ba649de8a214925fdb374b5" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:21.497864 22107 raft_consensus.cc:3060] T fb781a032ba649de8a214925fdb374b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:21.481745 21776 tablet_service.cc:1511] Processing CreateTablet for tablet c42767d8f674410188835ceefa644eb5 (DEFAULT_TABLE table=default.table_b54e9b49662b4a8e892615fec1080a87 [id=722129e8c51a45f8ae31843d0d3f0a47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:21.504678 21776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c42767d8f674410188835ceefa644eb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:21.514112 22293 tablet_bootstrap.cc:492] T ef57aa72abb2405b8187a7ed56d57f96 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:21.518920 22293 tablet_bootstrap.cc:654] T ef57aa72abb2405b8187a7ed56d57f96 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:21.482172 22415 ts_tablet_manager.cc:1916] T 286159c4dd10418da42040e53c1814f1 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:21.519542 22107 raft_consensus.cc:2468] T fb781a032ba649de8a214925fdb374b5 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:21.485736 22633 raft_consensus.cc:2804] T fb781a032ba649de8a214925fdb374b5 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:21.523643 22633 raft_consensus.cc:697] T fb781a032ba649de8a214925fdb374b5 P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Becoming Leader. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Running, Role: LEADER
I20260925 13:57:21.525075 22633 consensus_queue.cc:237] T fb781a032ba649de8a214925fdb374b5 P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.527585 22592 consensus_queue.cc:1048] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:21.543478 22293 tablet_bootstrap.cc:492] T ef57aa72abb2405b8187a7ed56d57f96 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:21.543846 22293 ts_tablet_manager.cc:1403] T ef57aa72abb2405b8187a7ed56d57f96 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.030s user 0.008s sys 0.000s
I20260925 13:57:21.548026 22592 consensus_queue.cc:1048] T 24dd39b302344f8d8f5166e9d641d13a P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:21.554886 22214 ts_tablet_manager.cc:1929] T 9745537e91d645fe9874373fe6237a9b P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:21.555234 22214 log.cc:1199] T 9745537e91d645fe9874373fe6237a9b P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/9745537e91d645fe9874373fe6237a9b
I20260925 13:57:21.558287 22211 tablet_bootstrap.cc:492] T 286159c4dd10418da42040e53c1814f1 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:21.558687 22211 ts_tablet_manager.cc:1403] T 286159c4dd10418da42040e53c1814f1 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.108s user 0.007s sys 0.006s
I20260925 13:57:21.555914 22214 ts_tablet_manager.cc:1950] T 9745537e91d645fe9874373fe6237a9b P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:21.564152 22211 raft_consensus.cc:359] T 286159c4dd10418da42040e53c1814f1 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.564738 22211 raft_consensus.cc:385] T 286159c4dd10418da42040e53c1814f1 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:21.564965 22211 raft_consensus.cc:740] T 286159c4dd10418da42040e53c1814f1 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:21.565650 22293 raft_consensus.cc:359] T ef57aa72abb2405b8187a7ed56d57f96 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.566211 22293 raft_consensus.cc:385] T ef57aa72abb2405b8187a7ed56d57f96 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:21.566448 22293 raft_consensus.cc:740] T ef57aa72abb2405b8187a7ed56d57f96 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:21.568884 22212 tablet_bootstrap.cc:492] T 286159c4dd10418da42040e53c1814f1 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:21.569281 22212 ts_tablet_manager.cc:1403] T 286159c4dd10418da42040e53c1814f1 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.131s user 0.008s sys 0.000s
I20260925 13:57:21.570648 22592 consensus_queue.cc:1048] T 24dd39b302344f8d8f5166e9d641d13a P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:21.571372 22415 ts_tablet_manager.cc:1929] T 286159c4dd10418da42040e53c1814f1 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:21.571704 22415 log.cc:1199] T 286159c4dd10418da42040e53c1814f1 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/286159c4dd10418da42040e53c1814f1
I20260925 13:57:21.572304 22415 ts_tablet_manager.cc:1950] T 286159c4dd10418da42040e53c1814f1 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:21.573572 21572 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet 286159c4dd10418da42040e53c1814f1 (table default.table_dc864d7428e5405da360d2f379eef37b [id=bf8cef2239904fd989622f9426412635]) successfully deleted
I20260925 13:57:21.574381 22214 tablet_replica.cc:333] T f68ccd8a7f4a4e518c2584e579de0c87 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:21.575050 22214 raft_consensus.cc:2243] T f68ccd8a7f4a4e518c2584e579de0c87 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:21.575505 22214 raft_consensus.cc:2272] T f68ccd8a7f4a4e518c2584e579de0c87 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:21.578915 22211 consensus_queue.cc:260] T 286159c4dd10418da42040e53c1814f1 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:21.581504 21572 catalog_manager.cc:5030] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): delete failed for tablet 9745537e91d645fe9874373fe6237a9b because the tablet was not found. No further retry: Not found: Tablet not found: 9745537e91d645fe9874373fe6237a9b
I20260925 13:57:21.577129 22214 ts_tablet_manager.cc:1916] T f68ccd8a7f4a4e518c2584e579de0c87 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:21.583693 22212 raft_consensus.cc:359] T 286159c4dd10418da42040e53c1814f1 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.584240 22212 raft_consensus.cc:385] T 286159c4dd10418da42040e53c1814f1 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:21.584465 22212 raft_consensus.cc:740] T 286159c4dd10418da42040e53c1814f1 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:21.567168 22293 consensus_queue.cc:260] T ef57aa72abb2405b8187a7ed56d57f96 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.602692 22212 consensus_queue.cc:260] T 286159c4dd10418da42040e53c1814f1 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.612617 22592 raft_consensus.cc:493] T 286159c4dd10418da42040e53c1814f1 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:21.613308 22293 ts_tablet_manager.cc:1434] T ef57aa72abb2405b8187a7ed56d57f96 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.069s user 0.005s sys 0.000s
I20260925 13:57:21.613207 22592 raft_consensus.cc:515] T 286159c4dd10418da42040e53c1814f1 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.617244 22212 ts_tablet_manager.cc:1434] T 286159c4dd10418da42040e53c1814f1 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.048s user 0.007s sys 0.000s
I20260925 13:57:21.617317 22592 leader_election.cc:290] T 286159c4dd10418da42040e53c1814f1 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 13:57:21.619208 21893 leader_election.cc:343] T 286159c4dd10418da42040e53c1814f1 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 286159c4dd10418da42040e53c1814f1
I20260925 13:57:21.620252 22106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "286159c4dd10418da42040e53c1814f1" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:21.621310 22293 tablet_bootstrap.cc:492] T c42767d8f674410188835ceefa644eb5 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:21.623720 22106 raft_consensus.cc:2468] T 286159c4dd10418da42040e53c1814f1 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 0.
I20260925 13:57:21.621932 22214 ts_tablet_manager.cc:1929] T f68ccd8a7f4a4e518c2584e579de0c87 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:21.624931 21940 tablet_service.cc:1558] Processing DeleteTablet for tablet 286159c4dd10418da42040e53c1814f1 with delete_type TABLET_DATA_DELETED (replaced by tablet ef57aa72abb2405b8187a7ed56d57f96) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:21.625119 22214 log.cc:1199] T f68ccd8a7f4a4e518c2584e579de0c87 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/f68ccd8a7f4a4e518c2584e579de0c87
I20260925 13:57:21.625099 21896 leader_election.cc:304] T 286159c4dd10418da42040e53c1814f1 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36; no voters: 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:21.625816 22214 ts_tablet_manager.cc:1950] T f68ccd8a7f4a4e518c2584e579de0c87 P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:21.625941 22517 raft_consensus.cc:493] T 794784c42d27441f829f954b6929a097 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260925 13:57:21.626384 21575 catalog_manager.cc:5043] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): delete failed for tablet 286159c4dd10418da42040e53c1814f1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 286159c4dd10418da42040e53c1814f1 already in progress: creating tablet
I20260925 13:57:21.626339 22517 raft_consensus.cc:515] T 794784c42d27441f829f954b6929a097 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.628247 22517 leader_election.cc:290] T 794784c42d27441f829f954b6929a097 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:21.628682 22517 raft_consensus.cc:2804] T 286159c4dd10418da42040e53c1814f1 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:21.628768 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet f68ccd8a7f4a4e518c2584e579de0c87 (table default.table_b54e9b49662b4a8e892615fec1080a87 [id=722129e8c51a45f8ae31843d0d3f0a47]) successfully deleted
I20260925 13:57:21.628984 22517 raft_consensus.cc:493] T 286159c4dd10418da42040e53c1814f1 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:21.629272 22517 raft_consensus.cc:3060] T 286159c4dd10418da42040e53c1814f1 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
W20260925 13:57:21.638093 21896 consensus_peers.cc:597] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee -> Peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Couldn't send request to peer ee02f77594864349982492862d668a36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60256642160a44a2bab77b97d78c27a8. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:57:21.643622 22517 raft_consensus.cc:515] T 286159c4dd10418da42040e53c1814f1 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.636108 22214 tablet_replica.cc:333] T 286159c4dd10418da42040e53c1814f1 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:21.646009 22214 raft_consensus.cc:2243] T 286159c4dd10418da42040e53c1814f1 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:21.646443 22214 raft_consensus.cc:2272] T 286159c4dd10418da42040e53c1814f1 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:21.638803 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "794784c42d27441f829f954b6929a097" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:21.647507 22211 ts_tablet_manager.cc:1434] T 286159c4dd10418da42040e53c1814f1 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.089s user 0.008s sys 0.000s
I20260925 13:57:21.647486 22107 raft_consensus.cc:2468] T 794784c42d27441f829f954b6929a097 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 0.
I20260925 13:57:21.648463 22214 ts_tablet_manager.cc:1916] T 286159c4dd10418da42040e53c1814f1 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:21.648469 21896 leader_election.cc:304] T 794784c42d27441f829f954b6929a097 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:21.628299 22212 tablet_bootstrap.cc:492] T c42767d8f674410188835ceefa644eb5 P ee02f77594864349982492862d668a36: Bootstrap starting.
W20260925 13:57:21.659185 21586 catalog_manager.cc:5671] Ignoring report from unknown tablet 60256642160a44a2bab77b97d78c27a8
I20260925 13:57:21.659761 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "794784c42d27441f829f954b6929a097" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2" is_pre_election: true
I20260925 13:57:21.660878 21795 raft_consensus.cc:2468] T 794784c42d27441f829f954b6929a097 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 0.
I20260925 13:57:21.664875 22211 tablet_bootstrap.cc:492] T c42767d8f674410188835ceefa644eb5 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:21.662295 22592 raft_consensus.cc:2804] T 794784c42d27441f829f954b6929a097 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:21.668823 21586 catalog_manager.cc:5838] T fb781a032ba649de8a214925fdb374b5 P 5e2731975dbf433aade4a859df9699e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e2731975dbf433aade4a859df9699e2 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "5e2731975dbf433aade4a859df9699e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: UNKNOWN } } }
I20260925 13:57:21.670861 22214 ts_tablet_manager.cc:1929] T 286159c4dd10418da42040e53c1814f1 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:21.671188 22214 log.cc:1199] T 286159c4dd10418da42040e53c1814f1 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/286159c4dd10418da42040e53c1814f1
I20260925 13:57:21.671810 22214 ts_tablet_manager.cc:1950] T 286159c4dd10418da42040e53c1814f1 P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:21.673056 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet 286159c4dd10418da42040e53c1814f1 (table default.table_dc864d7428e5405da360d2f379eef37b [id=bf8cef2239904fd989622f9426412635]) successfully deleted
I20260925 13:57:21.669610 22592 raft_consensus.cc:493] T 794784c42d27441f829f954b6929a097 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:21.675400 22592 raft_consensus.cc:3060] T 794784c42d27441f829f954b6929a097 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:21.663262 22293 tablet_bootstrap.cc:654] T c42767d8f674410188835ceefa644eb5 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
W20260925 13:57:21.674890 21893 leader_election.cc:343] T 286159c4dd10418da42040e53c1814f1 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 286159c4dd10418da42040e53c1814f1
I20260925 13:57:21.683238 22107 raft_consensus.cc:1275] T 56a39663479d40be8bbd1dfd4c91040c P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Refusing update from remote peer 145e520e0a5a445d943a53cf8c27efee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:57:21.685065 22653 consensus_queue.cc:1048] T 56a39663479d40be8bbd1dfd4c91040c P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 13:57:21.688419 21797 raft_consensus.cc:1275] T 56a39663479d40be8bbd1dfd4c91040c P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Refusing update from remote peer 145e520e0a5a445d943a53cf8c27efee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:57:21.680325 22211 tablet_bootstrap.cc:654] T c42767d8f674410188835ceefa644eb5 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:21.690268 22653 consensus_queue.cc:1048] T 56a39663479d40be8bbd1dfd4c91040c P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 13:57:21.670503 21586 catalog_manager.cc:5838] T d9dbc7ac9d0142c8a92262d2117ee6ec P 5e2731975dbf433aade4a859df9699e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e2731975dbf433aade4a859df9699e2 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "5e2731975dbf433aade4a859df9699e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: UNKNOWN } } }
I20260925 13:57:21.704056 22087 tablet_service.cc:1558] Processing DeleteTablet for tablet 286159c4dd10418da42040e53c1814f1 with delete_type TABLET_DATA_DELETED (replaced by tablet ef57aa72abb2405b8187a7ed56d57f96) from {username='slave'} at 127.0.0.1:50052
W20260925 13:57:21.705917 21575 catalog_manager.cc:5030] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): delete failed for tablet 286159c4dd10418da42040e53c1814f1 because the tablet was not found. No further retry: Not found: Tablet not found: 286159c4dd10418da42040e53c1814f1
I20260925 13:57:21.713580 22212 tablet_bootstrap.cc:654] T c42767d8f674410188835ceefa644eb5 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:21.716729 22517 leader_election.cc:290] T 286159c4dd10418da42040e53c1814f1 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Requested vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 13:57:21.718500 21896 leader_election.cc:343] T 286159c4dd10418da42040e53c1814f1 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Not found: Tablet not found: 286159c4dd10418da42040e53c1814f1
I20260925 13:57:21.718917 21896 leader_election.cc:304] T 286159c4dd10418da42040e53c1814f1 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee; no voters: 5e2731975dbf433aade4a859df9699e2, ee02f77594864349982492862d668a36
I20260925 13:57:21.720904 22211 tablet_bootstrap.cc:492] T c42767d8f674410188835ceefa644eb5 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:21.721326 22211 ts_tablet_manager.cc:1403] T c42767d8f674410188835ceefa644eb5 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.073s user 0.008s sys 0.000s
I20260925 13:57:21.724007 22547 raft_consensus.cc:2749] T 286159c4dd10418da42040e53c1814f1 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260925 13:57:21.724154 22659 tablet.cc:1751] T 56a39663479d40be8bbd1dfd4c91040c P 145e520e0a5a445d943a53cf8c27efee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:57:21.732793 22293 tablet_bootstrap.cc:492] T c42767d8f674410188835ceefa644eb5 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:21.733287 22293 ts_tablet_manager.cc:1403] T c42767d8f674410188835ceefa644eb5 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.112s user 0.009s sys 0.002s
I20260925 13:57:21.734229 22635 raft_consensus.cc:493] T ef57aa72abb2405b8187a7ed56d57f96 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:21.734664 22635 raft_consensus.cc:515] T ef57aa72abb2405b8187a7ed56d57f96 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.736668 22635 leader_election.cc:290] T ef57aa72abb2405b8187a7ed56d57f96 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:21.735498 22660 tablet.cc:1751] T 56a39663479d40be8bbd1dfd4c91040c P 5e2731975dbf433aade4a859df9699e2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:57:21.739513 22293 raft_consensus.cc:359] T c42767d8f674410188835ceefa644eb5 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.740212 22293 raft_consensus.cc:385] T c42767d8f674410188835ceefa644eb5 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:21.739904 22211 raft_consensus.cc:359] T c42767d8f674410188835ceefa644eb5 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.740468 22293 raft_consensus.cc:740] T c42767d8f674410188835ceefa644eb5 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:21.740550 22211 raft_consensus.cc:385] T c42767d8f674410188835ceefa644eb5 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:21.740801 22211 raft_consensus.cc:740] T c42767d8f674410188835ceefa644eb5 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:21.741088 22293 consensus_queue.cc:260] T c42767d8f674410188835ceefa644eb5 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.741411 22211 consensus_queue.cc:260] T c42767d8f674410188835ceefa644eb5 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.745713 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef57aa72abb2405b8187a7ed56d57f96" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:21.747682 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef57aa72abb2405b8187a7ed56d57f96" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
W20260925 13:57:21.749158 21732 leader_election.cc:343] T ef57aa72abb2405b8187a7ed56d57f96 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Illegal state: must be running to vote when last-logged opid is not known
W20260925 13:57:21.749759 21732 leader_election.cc:343] T ef57aa72abb2405b8187a7ed56d57f96 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:57:21.750151 21732 leader_election.cc:304] T ef57aa72abb2405b8187a7ed56d57f96 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
I20260925 13:57:21.750800 22635 raft_consensus.cc:2749] T ef57aa72abb2405b8187a7ed56d57f96 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:57:21.754739 22652 log.cc:927] Time spent T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee: Append to log took a long time: real 0.067s user 0.000s sys 0.000s
I20260925 13:57:21.756675 22616 raft_consensus.cc:493] T 195c48349fb5410495299c9e71a72ad8 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:21.757277 22616 raft_consensus.cc:515] T 195c48349fb5410495299c9e71a72ad8 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.764453 22616 leader_election.cc:290] T 195c48349fb5410495299c9e71a72ad8 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:21.764601 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "195c48349fb5410495299c9e71a72ad8" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:21.765388 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "195c48349fb5410495299c9e71a72ad8" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2" is_pre_election: true
I20260925 13:57:21.767180 21796 raft_consensus.cc:2393] T 195c48349fb5410495299c9e71a72ad8 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ee02f77594864349982492862d668a36 in current term 1: Already voted for candidate 145e520e0a5a445d943a53cf8c27efee in this term.
I20260925 13:57:21.768460 22040 leader_election.cc:304] T 195c48349fb5410495299c9e71a72ad8 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:21.769851 22616 raft_consensus.cc:3060] T 195c48349fb5410495299c9e71a72ad8 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:21.777134 22212 tablet_bootstrap.cc:492] T c42767d8f674410188835ceefa644eb5 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:21.777508 22212 ts_tablet_manager.cc:1403] T c42767d8f674410188835ceefa644eb5 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.149s user 0.009s sys 0.003s
I20260925 13:57:21.779965 22211 ts_tablet_manager.cc:1434] T c42767d8f674410188835ceefa644eb5 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.058s user 0.007s sys 0.000s
I20260925 13:57:21.780838 21940 tablet_service.cc:1558] Processing DeleteTablet for tablet 286159c4dd10418da42040e53c1814f1 with delete_type TABLET_DATA_DELETED (replaced by tablet ef57aa72abb2405b8187a7ed56d57f96) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:21.780738 22212 raft_consensus.cc:359] T c42767d8f674410188835ceefa644eb5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.781369 22212 raft_consensus.cc:385] T c42767d8f674410188835ceefa644eb5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:21.781507 22416 tablet_replica.cc:333] T 286159c4dd10418da42040e53c1814f1 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:21.781627 22212 raft_consensus.cc:740] T c42767d8f674410188835ceefa644eb5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:21.782174 22416 raft_consensus.cc:2243] T 286159c4dd10418da42040e53c1814f1 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:21.782330 22211 tablet_bootstrap.cc:492] T ef57aa72abb2405b8187a7ed56d57f96 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:21.782609 22416 raft_consensus.cc:2272] T 286159c4dd10418da42040e53c1814f1 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:21.782303 22212 consensus_queue.cc:260] T c42767d8f674410188835ceefa644eb5 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:21.782931 21579 catalog_manager.cc:5671] Ignoring report from unknown tablet 60256642160a44a2bab77b97d78c27a8
I20260925 13:57:21.783892 21586 master_service.cc:960] ReplaceTablet: received request to replace tablet f5bd96a2bdb842e9b3bc1f706786d6b1 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:21.788269 22416 ts_tablet_manager.cc:1916] T 286159c4dd10418da42040e53c1814f1 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:21.792253 22211 tablet_bootstrap.cc:654] T ef57aa72abb2405b8187a7ed56d57f96 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:21.794155 22616 raft_consensus.cc:2749] T 195c48349fb5410495299c9e71a72ad8 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:57:21.800045 22293 ts_tablet_manager.cc:1434] T c42767d8f674410188835ceefa644eb5 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.066s user 0.007s sys 0.000s
I20260925 13:57:21.802619 22416 ts_tablet_manager.cc:1929] T 286159c4dd10418da42040e53c1814f1 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 13:57:21.809649 22416 log.cc:1199] T 286159c4dd10418da42040e53c1814f1 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/286159c4dd10418da42040e53c1814f1
I20260925 13:57:21.810675 22416 ts_tablet_manager.cc:1950] T 286159c4dd10418da42040e53c1814f1 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:21.813647 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet 286159c4dd10418da42040e53c1814f1 (table default.table_dc864d7428e5405da360d2f379eef37b [id=bf8cef2239904fd989622f9426412635]) successfully deleted
I20260925 13:57:21.817168 22211 tablet_bootstrap.cc:492] T ef57aa72abb2405b8187a7ed56d57f96 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:21.817824 22211 ts_tablet_manager.cc:1403] T ef57aa72abb2405b8187a7ed56d57f96 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.036s user 0.012s sys 0.006s
I20260925 13:57:21.827469 22211 raft_consensus.cc:359] T ef57aa72abb2405b8187a7ed56d57f96 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.828486 22211 raft_consensus.cc:385] T ef57aa72abb2405b8187a7ed56d57f96 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:21.828894 22211 raft_consensus.cc:740] T ef57aa72abb2405b8187a7ed56d57f96 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:21.830581 22211 consensus_queue.cc:260] T ef57aa72abb2405b8187a7ed56d57f96 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.834685 22211 ts_tablet_manager.cc:1434] T ef57aa72abb2405b8187a7ed56d57f96 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.016s user 0.007s sys 0.000s
I20260925 13:57:21.840610 22212 ts_tablet_manager.cc:1434] T c42767d8f674410188835ceefa644eb5 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.063s user 0.006s sys 0.001s
W20260925 13:57:21.845392 22662 log.cc:927] Time spent T 56a39663479d40be8bbd1dfd4c91040c P ee02f77594864349982492862d668a36: Append to log took a long time: real 0.067s user 0.001s sys 0.000s
I20260925 13:57:21.848120 22212 tablet_bootstrap.cc:492] T ef57aa72abb2405b8187a7ed56d57f96 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:21.723249 22592 raft_consensus.cc:515] T 794784c42d27441f829f954b6929a097 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.856652 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "794784c42d27441f829f954b6929a097" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2"
I20260925 13:57:21.857324 21796 raft_consensus.cc:3060] T 794784c42d27441f829f954b6929a097 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:21.867463 21796 raft_consensus.cc:2468] T 794784c42d27441f829f954b6929a097 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 1.
I20260925 13:57:21.874687 22669 tablet.cc:1751] T 56a39663479d40be8bbd1dfd4c91040c P ee02f77594864349982492862d668a36: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:57:21.876894 22653 consensus_queue.cc:1048] T 195c48349fb5410495299c9e71a72ad8 P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:21.877291 22212 tablet_bootstrap.cc:654] T ef57aa72abb2405b8187a7ed56d57f96 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:21.888025 21893 leader_election.cc:304] T 794784c42d27441f829f954b6929a097 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:21.888809 22653 raft_consensus.cc:2804] T 794784c42d27441f829f954b6929a097 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:21.889266 22653 raft_consensus.cc:697] T 794784c42d27441f829f954b6929a097 P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: Becoming Leader. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Running, Role: LEADER
I20260925 13:57:21.890009 22653 consensus_queue.cc:237] T 794784c42d27441f829f954b6929a097 P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.900136 22106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "794784c42d27441f829f954b6929a097" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:21.900635 22106 raft_consensus.cc:3060] T 794784c42d27441f829f954b6929a097 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:21.937487 22655 consensus_queue.cc:1048] T 195c48349fb5410495299c9e71a72ad8 P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 13:57:21.953069 22212 tablet_bootstrap.cc:492] T ef57aa72abb2405b8187a7ed56d57f96 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:21.903519 22592 leader_election.cc:290] T 794784c42d27441f829f954b6929a097 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Requested vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:21.953554 22212 ts_tablet_manager.cc:1403] T ef57aa72abb2405b8187a7ed56d57f96 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.106s user 0.008s sys 0.000s
I20260925 13:57:21.959965 22212 raft_consensus.cc:359] T ef57aa72abb2405b8187a7ed56d57f96 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.960613 22212 raft_consensus.cc:385] T ef57aa72abb2405b8187a7ed56d57f96 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:21.960857 22212 raft_consensus.cc:740] T ef57aa72abb2405b8187a7ed56d57f96 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:21.975592 22212 consensus_queue.cc:260] T ef57aa72abb2405b8187a7ed56d57f96 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:21.992236 22212 ts_tablet_manager.cc:1434] T ef57aa72abb2405b8187a7ed56d57f96 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.038s user 0.007s sys 0.000s
I20260925 13:57:22.012331 22635 raft_consensus.cc:493] T c42767d8f674410188835ceefa644eb5 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:22.012761 22635 raft_consensus.cc:515] T c42767d8f674410188835ceefa644eb5 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:22.014441 22635 leader_election.cc:290] T c42767d8f674410188835ceefa644eb5 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:22.015959 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c42767d8f674410188835ceefa644eb5" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:22.016111 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c42767d8f674410188835ceefa644eb5" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:22.016515 22107 raft_consensus.cc:2468] T c42767d8f674410188835ceefa644eb5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:22.016561 21960 raft_consensus.cc:2468] T c42767d8f674410188835ceefa644eb5 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:22.017429 21732 leader_election.cc:304] T c42767d8f674410188835ceefa644eb5 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:22.018525 22635 raft_consensus.cc:2804] T c42767d8f674410188835ceefa644eb5 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:22.018860 22635 raft_consensus.cc:493] T c42767d8f674410188835ceefa644eb5 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:22.019196 22635 raft_consensus.cc:3060] T c42767d8f674410188835ceefa644eb5 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:22.026018 22635 raft_consensus.cc:515] T c42767d8f674410188835ceefa644eb5 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:22.028549 22106 raft_consensus.cc:2468] T 794784c42d27441f829f954b6929a097 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 1.
I20260925 13:57:22.027391 22633 consensus_queue.cc:1048] T 1928234ffd3947a78e10d4c43e32aeb8 P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:22.056022 21958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c42767d8f674410188835ceefa644eb5" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
I20260925 13:57:22.056613 21958 raft_consensus.cc:3060] T c42767d8f674410188835ceefa644eb5 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:22.061398 22635 leader_election.cc:290] T c42767d8f674410188835ceefa644eb5 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Requested vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:22.065443 21958 raft_consensus.cc:2468] T c42767d8f674410188835ceefa644eb5 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:22.066576 21732 leader_election.cc:304] T c42767d8f674410188835ceefa644eb5 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:22.067751 22106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c42767d8f674410188835ceefa644eb5" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:22.068359 22106 raft_consensus.cc:3060] T c42767d8f674410188835ceefa644eb5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:22.071543 22635 raft_consensus.cc:2804] T c42767d8f674410188835ceefa644eb5 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:22.072708 22635 raft_consensus.cc:697] T c42767d8f674410188835ceefa644eb5 P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Becoming Leader. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Running, Role: LEADER
I20260925 13:57:22.073608 22635 consensus_queue.cc:237] T c42767d8f674410188835ceefa644eb5 P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:22.121932 22633 consensus_queue.cc:1048] T 1928234ffd3947a78e10d4c43e32aeb8 P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:22.133276 22106 raft_consensus.cc:2468] T c42767d8f674410188835ceefa644eb5 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:22.222838 22710 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.182.253:46177 every 8 connection(s)
I20260925 13:57:22.223471 22221 rpc_server.cc:307] RPC server started. Bound to: 127.18.182.253:46177
I20260925 13:57:22.227567 22221 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/master-1/data/info.pb
I20260925 13:57:22.231451 21585 catalog_manager.cc:3541] renamed table default.table_61d48f9061c741fc8aa556e3edd984fb [id=7377fa3ea9664b47a930d9c5f88c60fa] in HMS: new name default.table_683c0db2850941be84798e9dca3fe490
I20260925 13:57:22.236883 19163 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskYmJttM/build/tsan/bin/kudu as pid 22221
I20260925 13:57:22.257808 22711 sys_catalog.cc:263] Verifying existing consensus state
I20260925 13:57:22.264245 22714 raft_consensus.cc:493] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a9bf5a5925ac42769e474adc84a1b725)
I20260925 13:57:22.271580 22714 raft_consensus.cc:515] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:22.282624 22699 consensus_queue.cc:1048] T fb781a032ba649de8a214925fdb374b5 P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:22.286664 21598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e942d7f2e4864b5ca08e6e830085caa0" candidate_term: 4 candidate_status { last_received { term: 3 index: 172 } } ignore_live_leader: false dest_uuid: "a9bf5a5925ac42769e474adc84a1b725" is_pre_election: true
I20260925 13:57:22.288759 22714 leader_election.cc:290] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3ea1e11bc6d84114b7616815b6af4635 (127.18.182.253:46177), a9bf5a5925ac42769e474adc84a1b725 (127.18.182.252:39283)
I20260925 13:57:22.308158 22699 consensus_queue.cc:1048] T fb781a032ba649de8a214925fdb374b5 P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:22.339946 22711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635: Bootstrap starting.
I20260925 13:57:22.358696 22635 consensus_queue.cc:1048] T d9dbc7ac9d0142c8a92262d2117ee6ec P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:22.396494 22008 heartbeater.cc:344] Connected to a master server at 127.18.182.253:46177
I20260925 13:57:22.399526 22153 heartbeater.cc:344] Connected to a master server at 127.18.182.253:46177
I20260925 13:57:22.403838 21843 heartbeater.cc:344] Connected to a master server at 127.18.182.253:46177
I20260925 13:57:22.455906 22635 consensus_queue.cc:1048] T d9dbc7ac9d0142c8a92262d2117ee6ec P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:22.477871 22711 log.cc:826] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635: Log is configured to *not* fsync() on all Append() calls
I20260925 13:57:22.483850 22616 raft_consensus.cc:493] T ef57aa72abb2405b8187a7ed56d57f96 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:22.484267 22616 raft_consensus.cc:515] T ef57aa72abb2405b8187a7ed56d57f96 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:22.486022 22616 leader_election.cc:290] T ef57aa72abb2405b8187a7ed56d57f96 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:22.486788 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef57aa72abb2405b8187a7ed56d57f96" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2" is_pre_election: true
I20260925 13:57:22.489161 21958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef57aa72abb2405b8187a7ed56d57f96" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:22.489717 21958 raft_consensus.cc:2468] T ef57aa72abb2405b8187a7ed56d57f96 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee02f77594864349982492862d668a36 in term 0.
I20260925 13:57:22.490861 22043 leader_election.cc:304] T ef57aa72abb2405b8187a7ed56d57f96 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:22.492334 22616 raft_consensus.cc:2804] T ef57aa72abb2405b8187a7ed56d57f96 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:22.487316 21797 raft_consensus.cc:2468] T ef57aa72abb2405b8187a7ed56d57f96 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee02f77594864349982492862d668a36 in term 0.
I20260925 13:57:22.492650 22616 raft_consensus.cc:493] T ef57aa72abb2405b8187a7ed56d57f96 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:22.492957 22616 raft_consensus.cc:3060] T ef57aa72abb2405b8187a7ed56d57f96 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:22.499922 22616 raft_consensus.cc:515] T ef57aa72abb2405b8187a7ed56d57f96 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:22.508780 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef57aa72abb2405b8187a7ed56d57f96" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2"
I20260925 13:57:22.509282 21797 raft_consensus.cc:3060] T ef57aa72abb2405b8187a7ed56d57f96 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:22.509760 22616 leader_election.cc:290] T ef57aa72abb2405b8187a7ed56d57f96 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 election: Requested vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:22.511665 21958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef57aa72abb2405b8187a7ed56d57f96" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
I20260925 13:57:22.512228 21958 raft_consensus.cc:3060] T ef57aa72abb2405b8187a7ed56d57f96 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:22.521550 21797 raft_consensus.cc:2468] T ef57aa72abb2405b8187a7ed56d57f96 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee02f77594864349982492862d668a36 in term 1.
I20260925 13:57:22.524199 21958 raft_consensus.cc:2468] T ef57aa72abb2405b8187a7ed56d57f96 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee02f77594864349982492862d668a36 in term 1.
I20260925 13:57:22.525360 22040 leader_election.cc:304] T ef57aa72abb2405b8187a7ed56d57f96 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:22.526216 22616 raft_consensus.cc:2804] T ef57aa72abb2405b8187a7ed56d57f96 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:22.537192 22616 raft_consensus.cc:697] T ef57aa72abb2405b8187a7ed56d57f96 P ee02f77594864349982492862d668a36 [term 1 LEADER]: Becoming Leader. State: Replica: ee02f77594864349982492862d668a36, State: Running, Role: LEADER
I20260925 13:57:22.538129 22616 consensus_queue.cc:237] T ef57aa72abb2405b8187a7ed56d57f96 P ee02f77594864349982492862d668a36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:22.586345 21584 catalog_manager.cc:2213] created HMS catalog entry for table [id=e02d9bf4f1ff4d81bb26cddfe336f414]
I20260925 13:57:22.586968 22711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=1 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 13:57:22.591153 22711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635: Bootstrap complete.
I20260925 13:57:22.715211 22711 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:22.720388 22711 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ea1e11bc6d84114b7616815b6af4635, State: Initialized, Role: FOLLOWER
I20260925 13:57:22.723973 22711 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:22.796383 22008 heartbeater.cc:344] Connected to a master server at 127.18.182.253:46177
I20260925 13:57:22.800149 22730 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:57:22.801534 22730 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 13:57:22.826797 21420 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.18.182.253:46177 timed out after 0.550s (SENT) [suppressed 13 similar messages]
I20260925 13:57:22.845654 21582 catalog_manager.cc:2213] created HMS catalog entry for table [id=bb79c1d2d0094785b35250fb1b718b9a]
I20260925 13:57:22.847932 22711 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:57:22.848057 22675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e942d7f2e4864b5ca08e6e830085caa0" candidate_term: 4 candidate_status { last_received { term: 3 index: 172 } } ignore_live_leader: false dest_uuid: "3ea1e11bc6d84114b7616815b6af4635" is_pre_election: true
I20260925 13:57:22.849093 22675 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e942d7f2e4864b5ca08e6e830085caa0 in term 3.
I20260925 13:57:22.850631 22678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e942d7f2e4864b5ca08e6e830085caa0" candidate_term: 4 candidate_status { last_received { term: 3 index: 172 } } ignore_live_leader: false dest_uuid: "3ea1e11bc6d84114b7616815b6af4635" is_pre_election: true
I20260925 13:57:22.851687 21420 leader_election.cc:304] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ea1e11bc6d84114b7616815b6af4635, e942d7f2e4864b5ca08e6e830085caa0; no voters: a9bf5a5925ac42769e474adc84a1b725
I20260925 13:57:22.853745 22735 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260925 13:57:22.854127 22735 raft_consensus.cc:493] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a9bf5a5925ac42769e474adc84a1b725)
I20260925 13:57:22.854511 22735 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 3 FOLLOWER]: Advancing to term 4
I20260925 13:57:22.863257 22592 consensus_queue.cc:1048] T 794784c42d27441f829f954b6929a097 P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 13:57:22.851372 22678 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e942d7f2e4864b5ca08e6e830085caa0 in term 3.
W20260925 13:57:22.867712 22678 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41844 (request call id 0) took 484 ms (client timeout 550 ms). Trace:
W20260925 13:57:22.868099 22678 rpcz_store.cc:269] 0925 13:57:22.383406 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:22.383626 (+ 220us) service_pool.cc:224] Handling call
0925 13:57:22.867636 (+484010us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260925 13:57:22.887566 22735 raft_consensus.cc:515] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:22.890588 22678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e942d7f2e4864b5ca08e6e830085caa0" candidate_term: 4 candidate_status { last_received { term: 3 index: 172 } } ignore_live_leader: false dest_uuid: "3ea1e11bc6d84114b7616815b6af4635"
I20260925 13:57:22.899394 21598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e942d7f2e4864b5ca08e6e830085caa0" candidate_term: 4 candidate_status { last_received { term: 3 index: 172 } } ignore_live_leader: false dest_uuid: "a9bf5a5925ac42769e474adc84a1b725"
I20260925 13:57:22.901070 21422 leader_election.cc:304] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e942d7f2e4864b5ca08e6e830085caa0; no voters: 3ea1e11bc6d84114b7616815b6af4635, a9bf5a5925ac42769e474adc84a1b725
I20260925 13:57:22.901770 22731 consensus_queue.cc:1048] T 794784c42d27441f829f954b6929a097 P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:22.909546 22735 leader_election.cc:290] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [CANDIDATE]: Term 4 election: Requested vote from peers 3ea1e11bc6d84114b7616815b6af4635 (127.18.182.253:46177), a9bf5a5925ac42769e474adc84a1b725 (127.18.182.252:39283)
I20260925 13:57:22.910547 22735 raft_consensus.cc:2749] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260925 13:57:22.918483 21583 catalog_manager.cc:2213] created HMS catalog entry for table [id=bcd7dc63c23c475e943a6d7395fcec10]
I20260925 13:57:22.939720 22153 heartbeater.cc:344] Connected to a master server at 127.18.182.253:46177
W20260925 13:57:22.981349 21422 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.252:39283, user_credentials={real_user=slave}} blocked reactor thread for 80473.3us
I20260925 13:57:22.985098 21843 heartbeater.cc:344] Connected to a master server at 127.18.182.253:46177
I20260925 13:57:22.996744 21580 catalog_manager.cc:2213] created HMS catalog entry for table [id=18fb001639084ecbba7437d57dfd516e]
I20260925 13:57:23.244161 22751 consensus_queue.cc:1048] T c42767d8f674410188835ceefa644eb5 P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:23.303880 22752 consensus_queue.cc:1048] T c42767d8f674410188835ceefa644eb5 P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:23.321461 22730 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9bf5a5925ac42769e474adc84a1b725. Latest consensus state: current_term: 3 leader_uuid: "a9bf5a5925ac42769e474adc84a1b725" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:57:23.322125 22730 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 13:57:23.359512 21573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.253:46177, user_credentials={real_user=slave}} blocked reactor thread for 52177.5us
I20260925 13:57:23.466089 22730 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a9bf5a5925ac42769e474adc84a1b725)
I20260925 13:57:23.466612 22730 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:23.469938 22730 leader_election.cc:290] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e942d7f2e4864b5ca08e6e830085caa0 (127.18.182.254:43163), a9bf5a5925ac42769e474adc84a1b725 (127.18.182.252:39283)
I20260925 13:57:23.526202 22616 consensus_queue.cc:1048] T ef57aa72abb2405b8187a7ed56d57f96 P ee02f77594864349982492862d668a36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:23.544780 21586 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet f5bd96a2bdb842e9b3bc1f706786d6b1
I20260925 13:57:23.546371 21586 catalog_manager.cc:6934] ReplaceTablet: tablet f5bd96a2bdb842e9b3bc1f706786d6b1 deleted and replaced by tablet 88b875abab81457481d4b98570a74ea1
I20260925 13:57:23.546810 21586 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:35994 (request call id 111) took 1936 ms. Trace:
I20260925 13:57:23.549122 22087 tablet_service.cc:1558] Processing DeleteTablet for tablet f5bd96a2bdb842e9b3bc1f706786d6b1 with delete_type TABLET_DATA_DELETED (replaced by tablet 88b875abab81457481d4b98570a74ea1) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:23.554508 21940 tablet_service.cc:1558] Processing DeleteTablet for tablet f5bd96a2bdb842e9b3bc1f706786d6b1 with delete_type TABLET_DATA_DELETED (replaced by tablet 88b875abab81457481d4b98570a74ea1) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:23.547123 21586 rpcz_store.cc:276] 0925 13:57:21.610670 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:21.783789 (+173119us) service_pool.cc:224] Handling call
0925 13:57:23.546732 (+1762943us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:57:21.805313 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:57:21.805777 (+ 464us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:57:21.805807 (+ 30us) write_op.cc:435] Acquired schema lock
0925 13:57:21.805826 (+ 19us) tablet.cc:662] Decoding operations
0925 13:57:21.806151 (+ 325us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:57:21.806198 (+ 47us) write_op.cc:641] Partition lock acquired for write op
0925 13:57:21.806229 (+ 31us) tablet.cc:685] Acquiring locks for 2 operations
0925 13:57:21.806499 (+ 270us) tablet.cc:701] Row locks acquired
0925 13:57:21.806517 (+ 18us) write_op.cc:260] PREPARE: finished
0925 13:57:21.806651 (+ 134us) write_op.cc:270] Start()
0925 13:57:21.806753 (+ 102us) write_op.cc:276] Timestamp: P: 1790344641806594 usec, L: 0
0925 13:57:21.806774 (+ 21us) op_driver.cc:348] REPLICATION: starting
0925 13:57:21.807413 (+ 639us) log.cc:844] Serialized 665 byte log entry
0925 13:57:23.462908 (+1655495us) op_driver.cc:464] REPLICATION: finished
0925 13:57:23.469561 (+ 6653us) write_op.cc:301] APPLY: starting
0925 13:57:23.469662 (+ 101us) tablet.cc:1394] starting BulkCheckPresence
0925 13:57:23.469813 (+ 151us) tablet.cc:1397] finished BulkCheckPresence
0925 13:57:23.484815 (+ 15002us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:57:23.489773 (+ 4958us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:57:23.491277 (+ 1504us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 13:57:23.491561 (+ 284us) write_op.cc:312] APPLY: finished
0925 13:57:23.509381 (+ 17820us) log.cc:844] Serialized 29 byte log entry
0925 13:57:23.509556 (+ 175us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:23.509620 (+ 64us) write_op.cc:454] Released schema lock
0925 13:57:23.509652 (+ 32us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":6014,"num_ops":2,"prepare.queue_time_us":2943,"prepare.run_cpu_time_us":2420,"prepare.run_wall_time_us":2419,"raft.queue_time_us":2389,"raft.run_cpu_time_us":719,"raft.run_wall_time_us":719,"replication_time_us":1655451,"thread_start_us":4262,"threads_started":1}]]}
W20260925 13:57:23.574297 21586 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.195:53791 (request call id 113) took 1779 ms (client timeout 1000 ms). Trace:
W20260925 13:57:23.574921 21586 rpcz_store.cc:269] 0925 13:57:21.794714 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:23.573929 (+1779215us) service_pool.cc:211] Skipping call since client already timed out
0925 13:57:23.574207 (+ 278us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260925 13:57:23.584648 21440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ea1e11bc6d84114b7616815b6af4635" candidate_term: 4 candidate_status { last_received { term: 3 index: 175 } } ignore_live_leader: false dest_uuid: "e942d7f2e4864b5ca08e6e830085caa0" is_pre_election: true
I20260925 13:57:23.585860 21440 raft_consensus.cc:2393] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ea1e11bc6d84114b7616815b6af4635 in current term 4: Already voted for candidate e942d7f2e4864b5ca08e6e830085caa0 in this term.
I20260925 13:57:23.586629 22778 raft_consensus.cc:493] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:23.587134 22778 raft_consensus.cc:515] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:23.590211 21598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e942d7f2e4864b5ca08e6e830085caa0" candidate_term: 5 candidate_status { last_received { term: 3 index: 172 } } ignore_live_leader: false dest_uuid: "a9bf5a5925ac42769e474adc84a1b725" is_pre_election: true
I20260925 13:57:23.591372 21579 catalog_manager.cc:6391] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: default.table_05ac73cb4d774c0fbd89f99093663e76 [id=df6afcfb1e2c463493b14228e19ade26] alter complete (version 1)
W20260925 13:57:23.593593 21579 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.193:57907 (request call id 122) took 1822 ms (client timeout 1000 ms). Trace:
W20260925 13:57:23.593945 21579 rpcz_store.cc:269] 0925 13:57:21.771480 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:21.780420 (+ 8940us) service_pool.cc:224] Handling call
0925 13:57:23.593517 (+1813097us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:57:21.833540 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:57:21.833859 (+ 319us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:57:21.833880 (+ 21us) write_op.cc:435] Acquired schema lock
0925 13:57:21.833891 (+ 11us) tablet.cc:662] Decoding operations
0925 13:57:21.834123 (+ 232us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:57:21.834167 (+ 44us) write_op.cc:641] Partition lock acquired for write op
0925 13:57:21.834196 (+ 29us) tablet.cc:685] Acquiring locks for 1 operations
0925 13:57:21.834370 (+ 174us) tablet.cc:701] Row locks acquired
0925 13:57:21.834387 (+ 17us) write_op.cc:260] PREPARE: finished
0925 13:57:21.834502 (+ 115us) write_op.cc:270] Start()
0925 13:57:21.834594 (+ 92us) write_op.cc:276] Timestamp: P: 1790344641834446 usec, L: 0
0925 13:57:21.834613 (+ 19us) op_driver.cc:348] REPLICATION: starting
0925 13:57:21.835180 (+ 567us) log.cc:844] Serialized 356 byte log entry
0925 13:57:23.468616 (+1633436us) op_driver.cc:464] REPLICATION: finished
0925 13:57:23.586993 (+118377us) write_op.cc:301] APPLY: starting
0925 13:57:23.587046 (+ 53us) tablet.cc:1394] starting BulkCheckPresence
0925 13:57:23.587140 (+ 94us) tablet.cc:1397] finished BulkCheckPresence
0925 13:57:23.587232 (+ 92us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:57:23.587394 (+ 162us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:57:23.587527 (+ 133us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 13:57:23.587571 (+ 44us) write_op.cc:312] APPLY: finished
0925 13:57:23.587728 (+ 157us) log.cc:844] Serialized 23 byte log entry
0925 13:57:23.588350 (+ 622us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:23.588413 (+ 63us) write_op.cc:454] Released schema lock
0925 13:57:23.588443 (+ 30us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":118198,"apply.run_cpu_time_us":1243,"apply.run_wall_time_us":2506,"num_ops":1,"prepare.queue_time_us":8482,"prepare.run_cpu_time_us":1906,"prepare.run_wall_time_us":1906,"raft.queue_time_us":168,"raft.run_cpu_time_us":11,"raft.run_wall_time_us":11,"replication_time_us":1633857,"threads_started":1}]]}
I20260925 13:57:23.597221 22778 leader_election.cc:290] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ea1e11bc6d84114b7616815b6af4635 (127.18.182.253:46177), a9bf5a5925ac42769e474adc84a1b725 (127.18.182.252:39283)
I20260925 13:57:23.601775 22676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e942d7f2e4864b5ca08e6e830085caa0" candidate_term: 5 candidate_status { last_received { term: 3 index: 172 } } ignore_live_leader: false dest_uuid: "3ea1e11bc6d84114b7616815b6af4635" is_pre_election: true
I20260925 13:57:23.604058 21420 leader_election.cc:304] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e942d7f2e4864b5ca08e6e830085caa0; no voters: 3ea1e11bc6d84114b7616815b6af4635, a9bf5a5925ac42769e474adc84a1b725
I20260925 13:57:23.604979 22778 raft_consensus.cc:2749] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260925 13:57:23.621759 22767 catalog_manager.cc:1308] Loaded cluster ID: d5d0bc5567a74d9cad4d92a9d1041fd9
I20260925 13:57:23.622089 22767 catalog_manager.cc:1601] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635: loading cluster ID for follower catalog manager: success
W20260925 13:57:23.624934 21573 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.253:46177, user_credentials={real_user=slave}} blocked reactor thread for 50555us
I20260925 13:57:23.634446 22616 consensus_queue.cc:1048] T ef57aa72abb2405b8187a7ed56d57f96 P ee02f77594864349982492862d668a36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:23.670680 22650 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "5e2731975dbf433aade4a859df9699e2" instance_seqno: 1790344622433678) as {username='slave'} at 127.18.182.193:59841; Asking this server to re-register.
I20260925 13:57:23.677829 21843 heartbeater.cc:461] Registering TS with master...
I20260925 13:57:23.692761 22650 ts_manager.cc:194] Registered new tserver with Master: 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881)
I20260925 13:57:23.693078 22767 catalog_manager.cc:1623] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635: acquiring CA information for follower catalog manager: success
I20260925 13:57:23.708727 22760 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "a9bf5a5925ac42769e474adc84a1b725" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:57:23.709426 22760 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [sys.catalog]: This master's current role is: LEADER
I20260925 13:57:23.719518 22776 tablet_replica.cc:333] T f5bd96a2bdb842e9b3bc1f706786d6b1 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:23.720613 22777 tablet_replica.cc:333] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:23.722947 21579 catalog_manager.cc:5838] T ef57aa72abb2405b8187a7ed56d57f96 P ee02f77594864349982492862d668a36 reported cstate change: term changed from 0 to 1, leader changed from <none> to ee02f77594864349982492862d668a36 (127.18.182.195). New cstate: current_term: 1 leader_uuid: "ee02f77594864349982492862d668a36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: HEALTHY } } }
I20260925 13:57:23.744305 22767 catalog_manager.cc:1651] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260925 13:57:23.751494 21586 catalog_manager.cc:5671] Ignoring report from unknown tablet 60256642160a44a2bab77b97d78c27a8
I20260925 13:57:23.761219 22776 raft_consensus.cc:2243] T f5bd96a2bdb842e9b3bc1f706786d6b1 P ee02f77594864349982492862d668a36 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:23.776011 21582 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35994 (request call id 102) took 3190 ms. Trace:
I20260925 13:57:23.776305 21582 rpcz_store.cc:276] 0925 13:57:20.585314 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:20.585455 (+ 141us) service_pool.cc:224] Handling call
0925 13:57:21.139140 (+553685us) catalog_manager.cc:2140] Acquired catalog manager lock
0925 13:57:21.140714 (+ 1574us) catalog_manager.cc:2186] Created new table and tablet info
0925 13:57:22.845613 (+1704899us) catalog_manager.cc:2212] Created new table in HMS catalog
0925 13:57:23.775424 (+929811us) catalog_manager.cc:2246] Wrote table and tablets to system table
0925 13:57:23.775715 (+ 291us) catalog_manager.cc:2280] Inserted table and tablets into CatalogManager maps
0925 13:57:23.775954 (+ 239us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:57:22.847325 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:57:22.847789 (+ 464us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:57:22.847812 (+ 23us) write_op.cc:435] Acquired schema lock
0925 13:57:22.847826 (+ 14us) tablet.cc:662] Decoding operations
0925 13:57:22.848125 (+ 299us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:57:22.848166 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0925 13:57:22.848196 (+ 30us) tablet.cc:685] Acquiring locks for 2 operations
0925 13:57:22.848423 (+ 227us) tablet.cc:701] Row locks acquired
0925 13:57:22.848458 (+ 35us) write_op.cc:260] PREPARE: finished
0925 13:57:22.848597 (+ 139us) write_op.cc:270] Start()
0925 13:57:22.848693 (+ 96us) write_op.cc:276] Timestamp: P: 1790344642848549 usec, L: 0
0925 13:57:22.848707 (+ 14us) op_driver.cc:348] REPLICATION: starting
0925 13:57:22.849252 (+ 545us) log.cc:844] Serialized 433 byte log entry
0925 13:57:23.750090 (+900838us) op_driver.cc:464] REPLICATION: finished
0925 13:57:23.770088 (+ 19998us) write_op.cc:301] APPLY: starting
0925 13:57:23.770143 (+ 55us) tablet.cc:1394] starting BulkCheckPresence
0925 13:57:23.770250 (+ 107us) tablet.cc:1397] finished BulkCheckPresence
0925 13:57:23.770358 (+ 108us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:57:23.770765 (+ 407us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:57:23.770899 (+ 134us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 13:57:23.770942 (+ 43us) write_op.cc:312] APPLY: finished
0925 13:57:23.771132 (+ 190us) log.cc:844] Serialized 29 byte log entry
0925 13:57:23.771808 (+ 676us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:23.771870 (+ 62us) write_op.cc:454] Released schema lock
0925 13:57:23.771903 (+ 33us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"Hive Metastore.queue_time_us":1444521,"Hive Metastore.run_cpu_time_us":605,"Hive Metastore.run_wall_time_us":259733,"child_traces":[["op",{"apply.queue_time_us":19844,"num_ops":2,"prepare.queue_time_us":137,"prepare.run_cpu_time_us":2138,"prepare.run_wall_time_us":2139,"replication_time_us":901306}]]}
I20260925 13:57:23.779557 21582 master_service.cc:960] ReplaceTablet: received request to replace tablet d267def3f0864435a0d2c40fd7052eab from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:23.782789 21583 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35994 (request call id 104) took 2932 ms. Trace:
I20260925 13:57:23.783041 21583 rpcz_store.cc:276] 0925 13:57:20.850463 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:20.966980 (+116517us) service_pool.cc:224] Handling call
0925 13:57:21.137337 (+170357us) catalog_manager.cc:2140] Acquired catalog manager lock
0925 13:57:21.141645 (+ 4308us) catalog_manager.cc:2186] Created new table and tablet info
0925 13:57:22.918413 (+1776768us) catalog_manager.cc:2212] Created new table in HMS catalog
0925 13:57:23.782254 (+863841us) catalog_manager.cc:2246] Wrote table and tablets to system table
0925 13:57:23.782518 (+ 264us) catalog_manager.cc:2280] Inserted table and tablets into CatalogManager maps
0925 13:57:23.782742 (+ 224us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:57:22.920302 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:57:22.920766 (+ 464us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:57:22.920792 (+ 26us) write_op.cc:435] Acquired schema lock
0925 13:57:22.920808 (+ 16us) tablet.cc:662] Decoding operations
0925 13:57:22.921129 (+ 321us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:57:22.921169 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0925 13:57:22.921195 (+ 26us) tablet.cc:685] Acquiring locks for 2 operations
0925 13:57:22.921418 (+ 223us) tablet.cc:701] Row locks acquired
0925 13:57:22.921435 (+ 17us) write_op.cc:260] PREPARE: finished
0925 13:57:22.921569 (+ 134us) write_op.cc:270] Start()
0925 13:57:22.921671 (+ 102us) write_op.cc:276] Timestamp: P: 1790344642921514 usec, L: 0
0925 13:57:22.921690 (+ 19us) op_driver.cc:348] REPLICATION: starting
0925 13:57:22.922208 (+ 518us) log.cc:844] Serialized 433 byte log entry
0925 13:57:23.750280 (+828072us) op_driver.cc:464] REPLICATION: finished
0925 13:57:23.780357 (+ 30077us) write_op.cc:301] APPLY: starting
0925 13:57:23.780403 (+ 46us) tablet.cc:1394] starting BulkCheckPresence
0925 13:57:23.780502 (+ 99us) tablet.cc:1397] finished BulkCheckPresence
0925 13:57:23.780577 (+ 75us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:57:23.780890 (+ 313us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:57:23.781004 (+ 114us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 13:57:23.781074 (+ 70us) write_op.cc:312] APPLY: finished
0925 13:57:23.781236 (+ 162us) log.cc:844] Serialized 29 byte log entry
0925 13:57:23.781861 (+ 625us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:23.781946 (+ 85us) write_op.cc:454] Released schema lock
0925 13:57:23.782001 (+ 55us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"Hive Metastore.queue_time_us":1703582,"Hive Metastore.run_cpu_time_us":572,"Hive Metastore.run_wall_time_us":72117,"spinlock_wait_cycles":33024,"child_traces":[["op",{"apply.queue_time_us":29955,"num_ops":2,"prepare.queue_time_us":114,"prepare.run_cpu_time_us":2135,"prepare.run_wall_time_us":2136,"replication_time_us":828519,"threads_started":1}]]}
I20260925 13:57:23.793658 22651 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" instance_seqno: 1790344625095189) as {username='slave'} at 127.18.182.194:58885; Asking this server to re-register.
I20260925 13:57:23.796656 22008 heartbeater.cc:461] Registering TS with master...
I20260925 13:57:23.810451 22777 raft_consensus.cc:2243] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:23.815342 22651 ts_manager.cc:194] Registered new tserver with Master: 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:23.843505 21584 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35994 (request call id 103) took 3160 ms. Trace:
I20260925 13:57:23.844198 21580 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35994 (request call id 99) took 3391 ms. Trace:
I20260925 13:57:23.844519 21580 rpcz_store.cc:276] 0925 13:57:20.452329 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:20.452464 (+ 135us) service_pool.cc:224] Handling call
0925 13:57:21.132592 (+680128us) catalog_manager.cc:2140] Acquired catalog manager lock
0925 13:57:21.166270 (+ 33678us) catalog_manager.cc:2186] Created new table and tablet info
0925 13:57:22.996702 (+1830432us) catalog_manager.cc:2212] Created new table in HMS catalog
0925 13:57:23.843609 (+846907us) catalog_manager.cc:2246] Wrote table and tablets to system table
0925 13:57:23.843906 (+ 297us) catalog_manager.cc:2280] Inserted table and tablets into CatalogManager maps
0925 13:57:23.844144 (+ 238us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:57:22.998530 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:57:22.999022 (+ 492us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:57:22.999048 (+ 26us) write_op.cc:435] Acquired schema lock
0925 13:57:22.999065 (+ 17us) tablet.cc:662] Decoding operations
0925 13:57:22.999493 (+ 428us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:57:22.999546 (+ 53us) write_op.cc:641] Partition lock acquired for write op
0925 13:57:22.999573 (+ 27us) tablet.cc:685] Acquiring locks for 2 operations
0925 13:57:22.999835 (+ 262us) tablet.cc:701] Row locks acquired
0925 13:57:22.999858 (+ 23us) write_op.cc:260] PREPARE: finished
0925 13:57:22.999986 (+ 128us) write_op.cc:270] Start()
0925 13:57:23.000087 (+ 101us) write_op.cc:276] Timestamp: P: 1790344642999921 usec, L: 0
0925 13:57:23.000107 (+ 20us) op_driver.cc:348] REPLICATION: starting
0925 13:57:23.000661 (+ 554us) log.cc:844] Serialized 433 byte log entry
0925 13:57:23.840151 (+839490us) op_driver.cc:464] REPLICATION: finished
0925 13:57:23.840860 (+ 709us) write_op.cc:301] APPLY: starting
0925 13:57:23.840917 (+ 57us) tablet.cc:1394] starting BulkCheckPresence
0925 13:57:23.841043 (+ 126us) tablet.cc:1397] finished BulkCheckPresence
0925 13:57:23.841150 (+ 107us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:57:23.841576 (+ 426us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:57:23.841718 (+ 142us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 13:57:23.841763 (+ 45us) write_op.cc:312] APPLY: finished
0925 13:57:23.842064 (+ 301us) log.cc:844] Serialized 29 byte log entry
0925 13:57:23.842209 (+ 145us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:23.842285 (+ 76us) write_op.cc:454] Released schema lock
0925 13:57:23.842315 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0925 13:57:23.842527 (+ 212us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:23.842551 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":1751155,"Hive Metastore.run_cpu_time_us":634,"Hive Metastore.run_wall_time_us":78454,"child_traces":[["op",{"apply.queue_time_us":534,"apply.run_cpu_time_us":1644,"apply.run_wall_time_us":1644,"num_ops":2,"prepare.queue_time_us":182,"prepare.run_cpu_time_us":2303,"prepare.run_wall_time_us":2386,"replication_time_us":839945}]]}
I20260925 13:57:23.843855 21584 rpcz_store.cc:276] 0925 13:57:20.682662 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:20.682783 (+ 121us) service_pool.cc:224] Handling call
0925 13:57:20.694487 (+ 11704us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b54000203d4. stack: 00007fd96d084323 00007fd96d084220 00007fd96cc28574 0000000000516964 00000000005168f9 00007fd9741825c0 00007fd9742216fa 00007fd975f92d8c 00007fd976092566 00007fd973c3d034 00007fd973c3cfc2 00007fd973c3cef1 00007fd973c3ce6c 00007fd973c3c122 00007fd971ee709c 00007fd971ee64d6
0925 13:57:21.137708 (+443221us) catalog_manager.cc:2140] Acquired catalog manager lock
0925 13:57:21.139311 (+ 1603us) catalog_manager.cc:2186] Created new table and tablet info
0925 13:57:22.586308 (+1446997us) catalog_manager.cc:2212] Created new table in HMS catalog
0925 13:57:23.842889 (+1256581us) catalog_manager.cc:2246] Wrote table and tablets to system table
0925 13:57:23.843174 (+ 285us) catalog_manager.cc:2280] Inserted table and tablets into CatalogManager maps
0925 13:57:23.843446 (+ 272us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:57:22.663476 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:57:22.663987 (+ 511us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:57:22.664017 (+ 30us) write_op.cc:435] Acquired schema lock
0925 13:57:22.664036 (+ 19us) tablet.cc:662] Decoding operations
0925 13:57:22.664362 (+ 326us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:57:22.664408 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0925 13:57:22.664434 (+ 26us) tablet.cc:685] Acquiring locks for 2 operations
0925 13:57:22.664674 (+ 240us) tablet.cc:701] Row locks acquired
0925 13:57:22.664687 (+ 13us) write_op.cc:260] PREPARE: finished
0925 13:57:22.664817 (+ 130us) write_op.cc:270] Start()
0925 13:57:22.664918 (+ 101us) write_op.cc:276] Timestamp: P: 1790344642664735 usec, L: 0
0925 13:57:22.664933 (+ 15us) op_driver.cc:348] REPLICATION: starting
0925 13:57:22.665545 (+ 612us) log.cc:844] Serialized 433 byte log entry
0925 13:57:23.749854 (+1084309us) op_driver.cc:464] REPLICATION: finished
0925 13:57:23.750955 (+ 1101us) write_op.cc:301] APPLY: starting
0925 13:57:23.751010 (+ 55us) tablet.cc:1394] starting BulkCheckPresence
0925 13:57:23.751103 (+ 93us) tablet.cc:1397] finished BulkCheckPresence
0925 13:57:23.751218 (+ 115us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:57:23.751612 (+ 394us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:57:23.751753 (+ 141us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 13:57:23.751796 (+ 43us) write_op.cc:312] APPLY: finished
0925 13:57:23.751978 (+ 182us) log.cc:844] Serialized 29 byte log entry
0925 13:57:23.752117 (+ 139us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:23.752178 (+ 61us) write_op.cc:454] Released schema lock
0925 13:57:23.752206 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0925 13:57:23.844520 (+ 92314us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:23.844555 (+ 35us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":1091490,"Hive Metastore.run_cpu_time_us":612,"Hive Metastore.run_wall_time_us":354416,"spinlock_wait_cycles":25172864,"child_traces":[["op",{"apply.queue_time_us":915,"apply.run_cpu_time_us":1513,"apply.run_wall_time_us":93553,"num_ops":2,"prepare.queue_time_us":75512,"prepare.run_cpu_time_us":2350,"prepare.run_wall_time_us":8123,"replication_time_us":1084827,"thread_start_us":54915,"threads_started":1}]]}
I20260925 13:57:23.851542 21580 master_service.cc:960] ReplaceTablet: received request to replace tablet f5bd96a2bdb842e9b3bc1f706786d6b1 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:23.879596 22777 raft_consensus.cc:2272] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:23.884869 22776 raft_consensus.cc:2272] T f5bd96a2bdb842e9b3bc1f706786d6b1 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 13:57:23.893451 21643 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 794784c42d27441f829f954b6929a097 consensus state has no leader
W20260925 13:57:23.898131 21583 catalog_manager.cc:5671] Ignoring report from unknown tablet 60256642160a44a2bab77b97d78c27a8
I20260925 13:57:23.902563 22776 ts_tablet_manager.cc:1916] T f5bd96a2bdb842e9b3bc1f706786d6b1 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:23.905573 21582 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:35994 (request call id 112) took 2282 ms. Trace:
I20260925 13:57:23.905849 21582 rpcz_store.cc:276] 0925 13:57:21.623421 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:23.779456 (+2156035us) service_pool.cc:224] Handling call
0925 13:57:23.905346 (+125890us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b54000203d4. stack: 00007fd96d084323 00007fd96d084220 00007fd96cc28574 0000000000516964 00000000005168f9 00007fd9741825c0 00007fd9742216fa 00007fd975f92d8c 00007fd9760950e1 00007fd973c5dd37 00007fd973c5dcc2 00007fd973c5dbf1 00007fd973c5db6c 00007fd973c5ce22 00007fd971ee709c 00007fd971ee64d6
0925 13:57:23.905525 (+ 179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":287472896}
I20260925 13:57:23.912690 22777 ts_tablet_manager.cc:1916] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 13:57:23.923552 21579 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.195:53791 (request call id 114) took 1116 ms (client timeout 1000 ms). Trace:
W20260925 13:57:23.923864 21579 rpcz_store.cc:269] 0925 13:57:22.807092 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:23.598720 (+791628us) service_pool.cc:224] Handling call
0925 13:57:23.716932 (+118212us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b54000203d4. stack: 00007fd96d084323 00007fd96d084220 00007fd96cc28574 0000000000516964 00000000005168f9 00007fd9741825c0 00007fd9742216fa 00007fd975f92d8c 00007fd97609101e 00007fd973c35fb4 00007fd973c35f42 00007fd973c35e71 00007fd973c35dec 00007fd973c350a2 00007fd971ee709c 00007fd971ee64d6
0925 13:57:23.923492 (+206560us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:57:23.738682 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:57:23.739123 (+ 441us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:57:23.739149 (+ 26us) write_op.cc:435] Acquired schema lock
0925 13:57:23.739165 (+ 16us) tablet.cc:662] Decoding operations
0925 13:57:23.739442 (+ 277us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:57:23.739480 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0925 13:57:23.739505 (+ 25us) tablet.cc:685] Acquiring locks for 1 operations
0925 13:57:23.739672 (+ 167us) tablet.cc:701] Row locks acquired
0925 13:57:23.739688 (+ 16us) write_op.cc:260] PREPARE: finished
0925 13:57:23.739802 (+ 114us) write_op.cc:270] Start()
0925 13:57:23.739881 (+ 79us) write_op.cc:276] Timestamp: P: 1790344643739765 usec, L: 0
0925 13:57:23.739898 (+ 17us) op_driver.cc:348] REPLICATION: starting
0925 13:57:23.740423 (+ 525us) log.cc:844] Serialized 536 byte log entry
0925 13:57:23.895224 (+154801us) op_driver.cc:464] REPLICATION: finished
0925 13:57:23.908480 (+ 13256us) write_op.cc:301] APPLY: starting
0925 13:57:23.908532 (+ 52us) tablet.cc:1394] starting BulkCheckPresence
0925 13:57:23.908607 (+ 75us) tablet.cc:1397] finished BulkCheckPresence
0925 13:57:23.908686 (+ 79us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:57:23.908801 (+ 115us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:57:23.908912 (+ 111us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 13:57:23.908952 (+ 40us) write_op.cc:312] APPLY: finished
0925 13:57:23.909133 (+ 181us) log.cc:844] Serialized 23 byte log entry
0925 13:57:23.909276 (+ 143us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:23.909325 (+ 49us) write_op.cc:454] Released schema lock
0925 13:57:23.909351 (+ 26us) write_op.cc:341] FINISH: Updating metrics
0925 13:57:23.909549 (+ 198us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:23.909570 (+ 21us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":7,"spinlock_wait_cycles":262040448,"child_traces":[["op",{"apply.queue_time_us":13116,"apply.run_cpu_time_us":1045,"apply.run_wall_time_us":1045,"num_ops":1,"prepare.queue_time_us":12784,"prepare.run_cpu_time_us":2035,"prepare.run_wall_time_us":2038,"raft.queue_time_us":68,"raft.run_cpu_time_us":685,"raft.run_wall_time_us":686,"replication_time_us":155227,"spinlock_wait_cycles":135296,"thread_start_us":12821,"threads_started":1}]]}
I20260925 13:57:23.911134 22650 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "ee02f77594864349982492862d668a36" instance_seqno: 1790344627869737) as {username='slave'} at 127.18.182.195:60505; Asking this server to re-register.
I20260925 13:57:23.937568 22153 heartbeater.cc:461] Registering TS with master...
I20260925 13:57:23.938066 21586 catalog_manager.cc:5838] T c42767d8f674410188835ceefa644eb5 P 5e2731975dbf433aade4a859df9699e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e2731975dbf433aade4a859df9699e2 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "5e2731975dbf433aade4a859df9699e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: UNKNOWN } } }
I20260925 13:57:23.943418 22651 ts_manager.cc:194] Registered new tserver with Master: ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:23.946762 21940 tablet_service.cc:1511] Processing CreateTablet for tablet 88b875abab81457481d4b98570a74ea1 (DEFAULT_TABLE table=default.table_fe2e32c0275e4744b1163ae7b22eaf51 [id=9e8b522f92404f2ba5505e6fa689ea5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:23.948186 21940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88b875abab81457481d4b98570a74ea1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:23.950665 22087 tablet_service.cc:1511] Processing CreateTablet for tablet 88b875abab81457481d4b98570a74ea1 (DEFAULT_TABLE table=default.table_fe2e32c0275e4744b1163ae7b22eaf51 [id=9e8b522f92404f2ba5505e6fa689ea5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:23.951689 22087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88b875abab81457481d4b98570a74ea1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:57:23.955530 21575 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 794784c42d27441f829f954b6929a097 (table default.table_683c0db2850941be84798e9dca3fe490 [id=7377fa3ea9664b47a930d9c5f88c60fa], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 794784c42d27441f829f954b6929a097 consensus state has no leader
I20260925 13:57:23.982951 21579 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35994:
name: "default.Table_b80b7994ff294977bd92dc8f9ddd922e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:57:23.993310 21582 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35994:
name: "default.Table_8229f26c3e104bf1885bce04e09f8e29"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:57:24.000751 22776 ts_tablet_manager.cc:1929] T f5bd96a2bdb842e9b3bc1f706786d6b1 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:24.001610 22776 log.cc:1199] T f5bd96a2bdb842e9b3bc1f706786d6b1 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/f5bd96a2bdb842e9b3bc1f706786d6b1
I20260925 13:57:24.002731 22776 ts_tablet_manager.cc:1950] T f5bd96a2bdb842e9b3bc1f706786d6b1 P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:24.006907 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet f5bd96a2bdb842e9b3bc1f706786d6b1 (table default.table_fe2e32c0275e4744b1163ae7b22eaf51 [id=9e8b522f92404f2ba5505e6fa689ea5a]) successfully deleted
W20260925 13:57:24.008118 21586 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.193:57907 (request call id 123) took 1228 ms (client timeout 1000 ms). Trace:
I20260925 13:57:24.010136 22794 tablet_bootstrap.cc:492] T 88b875abab81457481d4b98570a74ea1 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:24.010265 21584 catalog_manager.cc:5838] T 794784c42d27441f829f954b6929a097 P ee02f77594864349982492862d668a36 reported cstate change: term changed from 0 to 1, leader changed from <none> to 145e520e0a5a445d943a53cf8c27efee (127.18.182.194). New cstate: current_term: 1 leader_uuid: "145e520e0a5a445d943a53cf8c27efee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } } }
I20260925 13:57:24.013473 21584 catalog_manager.cc:5938] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873) does not have the latest schema for tablet 794784c42d27441f829f954b6929a097 (table default.table_683c0db2850941be84798e9dca3fe490 [id=7377fa3ea9664b47a930d9c5f88c60fa]). Expected version 1 got 0
I20260925 13:57:24.032514 22794 tablet_bootstrap.cc:654] T 88b875abab81457481d4b98570a74ea1 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:24.032347 22777 ts_tablet_manager.cc:1929] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:24.032931 22777 log.cc:1199] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/f5bd96a2bdb842e9b3bc1f706786d6b1
I20260925 13:57:24.034096 22777 ts_tablet_manager.cc:1950] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:24.039314 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet f5bd96a2bdb842e9b3bc1f706786d6b1 (table default.table_fe2e32c0275e4744b1163ae7b22eaf51 [id=9e8b522f92404f2ba5505e6fa689ea5a]) successfully deleted
I20260925 13:57:24.044557 22795 tablet_bootstrap.cc:492] T 88b875abab81457481d4b98570a74ea1 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:24.052707 21583 catalog_manager.cc:5938] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881) does not have the latest schema for tablet 794784c42d27441f829f954b6929a097 (table default.table_683c0db2850941be84798e9dca3fe490 [id=7377fa3ea9664b47a930d9c5f88c60fa]). Expected version 1 got 0
I20260925 13:57:24.058588 22795 tablet_bootstrap.cc:654] T 88b875abab81457481d4b98570a74ea1 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:24.061110 21580 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35994:
name: "default.Table_8791c6009be44efab81f5f2252a3d8ae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:57:24.008478 21586 rpcz_store.cc:269] 0925 13:57:22.779717 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:23.580993 (+801276us) service_pool.cc:224] Handling call
0925 13:57:23.749424 (+168431us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b54000203d4. stack: 00007fd96d084323 00007fd96d084220 00007fd96cc28574 0000000000516964 00000000005168f9 00007fd9741825c0 00007fd9742216fa 00007fd975f92d8c 00007fd97609101e 00007fd973c35fb4 00007fd973c35f42 00007fd973c35e71 00007fd973c35dec 00007fd973c350a2 00007fd971ee709c 00007fd971ee64d6
0925 13:57:24.008050 (+258626us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:57:23.973248 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:57:23.973681 (+ 433us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:57:23.973708 (+ 27us) write_op.cc:435] Acquired schema lock
0925 13:57:23.973723 (+ 15us) tablet.cc:662] Decoding operations
0925 13:57:23.973999 (+ 276us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:57:23.974031 (+ 32us) write_op.cc:641] Partition lock acquired for write op
0925 13:57:23.974058 (+ 27us) tablet.cc:685] Acquiring locks for 1 operations
0925 13:57:23.974201 (+ 143us) tablet.cc:701] Row locks acquired
0925 13:57:23.974217 (+ 16us) write_op.cc:260] PREPARE: finished
0925 13:57:23.974325 (+ 108us) write_op.cc:270] Start()
0925 13:57:23.974400 (+ 75us) write_op.cc:276] Timestamp: P: 1790344643974265 usec, L: 0
0925 13:57:23.974443 (+ 43us) op_driver.cc:348] REPLICATION: starting
0925 13:57:23.974908 (+ 465us) log.cc:844] Serialized 536 byte log entry
0925 13:57:23.998592 (+ 23684us) op_driver.cc:464] REPLICATION: finished
0925 13:57:23.998808 (+ 216us) write_op.cc:301] APPLY: starting
0925 13:57:23.998858 (+ 50us) tablet.cc:1394] starting BulkCheckPresence
0925 13:57:23.998979 (+ 121us) tablet.cc:1397] finished BulkCheckPresence
0925 13:57:23.999108 (+ 129us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:57:23.999242 (+ 134us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:57:23.999379 (+ 137us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 13:57:23.999430 (+ 51us) write_op.cc:312] APPLY: finished
0925 13:57:23.999577 (+ 147us) log.cc:844] Serialized 23 byte log entry
0925 13:57:23.999714 (+ 137us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:23.999765 (+ 51us) write_op.cc:454] Released schema lock
0925 13:57:23.999793 (+ 28us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":7237,"reported_tablets":9,"spinlock_wait_cycles":377322368,"child_traces":[["op",{"apply.queue_time_us":83,"apply.run_cpu_time_us":1099,"apply.run_wall_time_us":1143,"mutex_wait_us":14971,"num_ops":1,"prepare.queue_time_us":15092,"prepare.run_cpu_time_us":1872,"prepare.run_wall_time_us":1871,"replication_time_us":24058}]]}
I20260925 13:57:24.069100 22087 tablet_service.cc:1558] Processing DeleteTablet for tablet f5bd96a2bdb842e9b3bc1f706786d6b1 with delete_type TABLET_DATA_DELETED (replaced by tablet 88b875abab81457481d4b98570a74ea1) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:24.069610 21586 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35994:
name: "default.Table_4586487550ec46f08eae599e1c34ef17"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:57:24.072762 21575 catalog_manager.cc:5030] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): delete failed for tablet f5bd96a2bdb842e9b3bc1f706786d6b1 because the tablet was not found. No further retry: Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
I20260925 13:57:24.100454 21940 tablet_service.cc:1511] Processing CreateTablet for tablet 565a7e8f453e464b8897d84e99429067 (DEFAULT_TABLE table=default.table_f644beebc9834408bed8d57d2ac5c2ee [id=18fb001639084ecbba7437d57dfd516e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:24.101492 21940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 565a7e8f453e464b8897d84e99429067. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:24.115295 22794 tablet_bootstrap.cc:492] T 88b875abab81457481d4b98570a74ea1 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:24.115795 22794 ts_tablet_manager.cc:1403] T 88b875abab81457481d4b98570a74ea1 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.106s user 0.011s sys 0.004s
I20260925 13:57:24.133219 22795 tablet_bootstrap.cc:492] T 88b875abab81457481d4b98570a74ea1 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:24.133678 22795 ts_tablet_manager.cc:1403] T 88b875abab81457481d4b98570a74ea1 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.089s user 0.006s sys 0.006s
I20260925 13:57:24.133711 21583 master_service.cc:960] ReplaceTablet: received request to replace tablet 60256642160a44a2bab77b97d78c27a8 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:24.136381 22795 raft_consensus.cc:359] T 88b875abab81457481d4b98570a74ea1 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:24.137135 22795 raft_consensus.cc:385] T 88b875abab81457481d4b98570a74ea1 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:24.137396 22795 raft_consensus.cc:740] T 88b875abab81457481d4b98570a74ea1 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:24.138170 22795 consensus_queue.cc:260] T 88b875abab81457481d4b98570a74ea1 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:24.140882 22795 ts_tablet_manager.cc:1434] T 88b875abab81457481d4b98570a74ea1 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.007s user 0.002s sys 0.003s
I20260925 13:57:24.141572 21584 master_service.cc:960] ReplaceTablet: received request to replace tablet f5bd96a2bdb842e9b3bc1f706786d6b1 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:24.146045 22087 tablet_service.cc:1511] Processing CreateTablet for tablet 565a7e8f453e464b8897d84e99429067 (DEFAULT_TABLE table=default.table_f644beebc9834408bed8d57d2ac5c2ee [id=18fb001639084ecbba7437d57dfd516e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:24.118381 22794 raft_consensus.cc:359] T 88b875abab81457481d4b98570a74ea1 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:24.147095 22794 raft_consensus.cc:385] T 88b875abab81457481d4b98570a74ea1 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:24.147497 22087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 565a7e8f453e464b8897d84e99429067. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:57:24.156952 22609 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: connection negotiation to 127.18.182.252:39283 for RPC RequestConsensusVote timed out after 0.684s (ON_OUTBOUND_QUEUE)
I20260925 13:57:24.157094 21938 tablet_service.cc:1511] Processing CreateTablet for tablet 60ef9c152aec465eb0bb7dc286c74277 (DEFAULT_TABLE table=default.table_30772fdea84947818aff6f6522c73f4b [id=bcd7dc63c23c475e943a6d7395fcec10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:24.158247 21938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60ef9c152aec465eb0bb7dc286c74277. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:24.147404 22794 raft_consensus.cc:740] T 88b875abab81457481d4b98570a74ea1 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:24.189344 22794 consensus_queue.cc:260] T 88b875abab81457481d4b98570a74ea1 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:24.192281 22085 tablet_service.cc:1511] Processing CreateTablet for tablet 60ef9c152aec465eb0bb7dc286c74277 (DEFAULT_TABLE table=default.table_30772fdea84947818aff6f6522c73f4b [id=bcd7dc63c23c475e943a6d7395fcec10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:24.200039 22795 tablet_bootstrap.cc:492] T 565a7e8f453e464b8897d84e99429067 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:24.207944 22085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60ef9c152aec465eb0bb7dc286c74277. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:24.209658 21940 tablet_service.cc:1511] Processing CreateTablet for tablet a06a5aac5d464d339c870df4ebd63d92 (DEFAULT_TABLE table=default.table_1baae11db5a24d50bd596853233007cb [id=bb79c1d2d0094785b35250fb1b718b9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:24.216114 21940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a06a5aac5d464d339c870df4ebd63d92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:24.216768 22794 ts_tablet_manager.cc:1434] T 88b875abab81457481d4b98570a74ea1 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.101s user 0.006s sys 0.000s
I20260925 13:57:24.219978 22795 tablet_bootstrap.cc:654] T 565a7e8f453e464b8897d84e99429067 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:24.225510 22083 tablet_service.cc:1511] Processing CreateTablet for tablet a06a5aac5d464d339c870df4ebd63d92 (DEFAULT_TABLE table=default.table_1baae11db5a24d50bd596853233007cb [id=bb79c1d2d0094785b35250fb1b718b9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:24.228737 21937 tablet_service.cc:1511] Processing CreateTablet for tablet 04cd31f29357471a882157829ca700ab (DEFAULT_TABLE table=default.table_fc718993756d497eb9df6af8c1651362 [id=e02d9bf4f1ff4d81bb26cddfe336f414]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:24.229075 22804 tablet.cc:1751] T 794784c42d27441f829f954b6929a097 P ee02f77594864349982492862d668a36: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:57:24.229647 21937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04cd31f29357471a882157829ca700ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:24.237846 22084 tablet_service.cc:1511] Processing CreateTablet for tablet 04cd31f29357471a882157829ca700ab (DEFAULT_TABLE table=default.table_fc718993756d497eb9df6af8c1651362 [id=e02d9bf4f1ff4d81bb26cddfe336f414]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:24.239717 22803 tablet.cc:1751] T 794784c42d27441f829f954b6929a097 P 145e520e0a5a445d943a53cf8c27efee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:57:24.251941 22083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a06a5aac5d464d339c870df4ebd63d92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:24.252962 22805 tablet.cc:1751] T 794784c42d27441f829f954b6929a097 P 5e2731975dbf433aade4a859df9699e2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:57:24.255957 22084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04cd31f29357471a882157829ca700ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:24.272928 22795 tablet_bootstrap.cc:492] T 565a7e8f453e464b8897d84e99429067 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:24.273387 22795 ts_tablet_manager.cc:1403] T 565a7e8f453e464b8897d84e99429067 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.074s user 0.004s sys 0.006s
I20260925 13:57:24.276017 22795 raft_consensus.cc:359] T 565a7e8f453e464b8897d84e99429067 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:24.276678 22795 raft_consensus.cc:385] T 565a7e8f453e464b8897d84e99429067 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:24.276932 22795 raft_consensus.cc:740] T 565a7e8f453e464b8897d84e99429067 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:24.281239 22795 consensus_queue.cc:260] T 565a7e8f453e464b8897d84e99429067 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:24.294898 22795 ts_tablet_manager.cc:1434] T 565a7e8f453e464b8897d84e99429067 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.021s user 0.006s sys 0.001s
I20260925 13:57:24.307423 21587 catalog_manager.cc:2770] deleted HMS catalog entry for table default.table_8008d16a92cf4c0dafb258a1122bffb5 [id=a22f86f8c2404aa69bdeb9883471453c]
I20260925 13:57:24.308815 22794 tablet_bootstrap.cc:492] T 565a7e8f453e464b8897d84e99429067 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:24.320116 22794 tablet_bootstrap.cc:654] T 565a7e8f453e464b8897d84e99429067 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:24.322125 22795 tablet_bootstrap.cc:492] T 60ef9c152aec465eb0bb7dc286c74277 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:24.333503 21584 catalog_manager.cc:6391] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: default.table_683c0db2850941be84798e9dca3fe490 [id=7377fa3ea9664b47a930d9c5f88c60fa] alter complete (version 1)
I20260925 13:57:24.338618 22795 tablet_bootstrap.cc:654] T 60ef9c152aec465eb0bb7dc286c74277 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:24.340245 21585 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:35994 (request call id 94) took 4988 ms. Trace:
I20260925 13:57:24.340656 21585 rpcz_store.cc:276] 0925 13:57:19.352015 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:19.439309 (+ 87294us) service_pool.cc:224] Handling call
0925 13:57:21.127504 (+1688195us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 13:57:24.338090 (+3210586us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 13:57:24.339443 (+ 1353us) catalog_manager.cc:3764] Apply alter schema
0925 13:57:24.340187 (+ 744us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":7549,"Hive Metastore.run_cpu_time_us":1779,"Hive Metastore.run_wall_time_us":1095267}
I20260925 13:57:24.351756 21585 master_service.cc:960] ReplaceTablet: received request to replace tablet 761ede873f5648ccb5216a4c0a8afa2e from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:24.355316 22785 raft_consensus.cc:493] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:24.355801 22785 raft_consensus.cc:515] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:24.357790 22785 leader_election.cc:290] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 13:57:24.361172 22043 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
W20260925 13:57:24.364499 22040 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
I20260925 13:57:24.366110 22040 leader_election.cc:304] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:24.366719 22785 raft_consensus.cc:2749] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:57:24.367856 22794 tablet_bootstrap.cc:492] T 565a7e8f453e464b8897d84e99429067 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:24.368219 22794 ts_tablet_manager.cc:1403] T 565a7e8f453e464b8897d84e99429067 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.060s user 0.008s sys 0.004s
I20260925 13:57:24.370694 22794 raft_consensus.cc:359] T 565a7e8f453e464b8897d84e99429067 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:24.371305 22794 raft_consensus.cc:385] T 565a7e8f453e464b8897d84e99429067 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:24.371603 22794 raft_consensus.cc:740] T 565a7e8f453e464b8897d84e99429067 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:24.373692 21585 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet 761ede873f5648ccb5216a4c0a8afa2e
I20260925 13:57:24.377100 21585 catalog_manager.cc:6934] ReplaceTablet: tablet 761ede873f5648ccb5216a4c0a8afa2e deleted and replaced by tablet 8ace3cd70b45451d8e9fb4b8a8d4dd0d
I20260925 13:57:24.381511 22795 tablet_bootstrap.cc:492] T 60ef9c152aec465eb0bb7dc286c74277 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:24.382298 22795 ts_tablet_manager.cc:1403] T 60ef9c152aec465eb0bb7dc286c74277 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.060s user 0.003s sys 0.009s
I20260925 13:57:24.384871 22795 raft_consensus.cc:359] T 60ef9c152aec465eb0bb7dc286c74277 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:24.385519 22795 raft_consensus.cc:385] T 60ef9c152aec465eb0bb7dc286c74277 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:24.385767 22795 raft_consensus.cc:740] T 60ef9c152aec465eb0bb7dc286c74277 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:24.388348 21584 master_service.cc:960] ReplaceTablet: received request to replace tablet 60256642160a44a2bab77b97d78c27a8 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:24.388872 22795 consensus_queue.cc:260] T 60ef9c152aec465eb0bb7dc286c74277 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:24.390048 22781 raft_consensus.cc:493] T 88b875abab81457481d4b98570a74ea1 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:24.390456 22781 raft_consensus.cc:515] T 88b875abab81457481d4b98570a74ea1 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:24.392292 22781 leader_election.cc:290] T 88b875abab81457481d4b98570a74ea1 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:24.393640 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88b875abab81457481d4b98570a74ea1" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:24.394136 22107 raft_consensus.cc:2468] T 88b875abab81457481d4b98570a74ea1 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 0.
I20260925 13:57:24.390081 22794 consensus_queue.cc:260] T 565a7e8f453e464b8897d84e99429067 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:24.395613 21937 tablet_service.cc:1558] Processing DeleteTablet for tablet 761ede873f5648ccb5216a4c0a8afa2e with delete_type TABLET_DATA_DELETED (replaced by tablet 8ace3cd70b45451d8e9fb4b8a8d4dd0d) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:24.395750 21896 leader_election.cc:304] T 88b875abab81457481d4b98570a74ea1 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:24.396293 22777 tablet_replica.cc:333] T 761ede873f5648ccb5216a4c0a8afa2e P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:24.396822 22781 raft_consensus.cc:2804] T 88b875abab81457481d4b98570a74ea1 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:24.397080 22777 raft_consensus.cc:2243] T 761ede873f5648ccb5216a4c0a8afa2e P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:24.397135 22781 raft_consensus.cc:493] T 88b875abab81457481d4b98570a74ea1 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:24.397511 22781 raft_consensus.cc:3060] T 88b875abab81457481d4b98570a74ea1 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:24.398075 22777 raft_consensus.cc:2272] T 761ede873f5648ccb5216a4c0a8afa2e P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:24.402781 21643 catalog_manager.cc:2811] Deleting table default.table_8008d16a92cf4c0dafb258a1122bffb5 [id=a22f86f8c2404aa69bdeb9883471453c] in response to Hive Metastore notification log event 30
I20260925 13:57:24.405628 22794 ts_tablet_manager.cc:1434] T 565a7e8f453e464b8897d84e99429067 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.037s user 0.001s sys 0.005s
W20260925 13:57:24.407976 21893 leader_election.cc:343] T 88b875abab81457481d4b98570a74ea1 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 88b875abab81457481d4b98570a74ea1
I20260925 13:57:24.406454 22794 tablet_bootstrap.cc:492] T 60ef9c152aec465eb0bb7dc286c74277 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:24.420660 22794 tablet_bootstrap.cc:654] T 60ef9c152aec465eb0bb7dc286c74277 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:24.424288 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet 761ede873f5648ccb5216a4c0a8afa2e with delete_type TABLET_DATA_DELETED (replaced by tablet 8ace3cd70b45451d8e9fb4b8a8d4dd0d) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:24.424850 22776 tablet_replica.cc:333] T 761ede873f5648ccb5216a4c0a8afa2e P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:24.425519 22776 raft_consensus.cc:2243] T 761ede873f5648ccb5216a4c0a8afa2e P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:24.425959 22776 raft_consensus.cc:2272] T 761ede873f5648ccb5216a4c0a8afa2e P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:24.426501 21643 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet 76d4d7d706824892b075ab1aa4fbaf85
I20260925 13:57:24.427901 22776 ts_tablet_manager.cc:1916] T 761ede873f5648ccb5216a4c0a8afa2e P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:24.429039 22795 ts_tablet_manager.cc:1434] T 60ef9c152aec465eb0bb7dc286c74277 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.046s user 0.000s sys 0.005s
W20260925 13:57:24.430754 21582 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_8229f26c3e104bf1885bce04e09f8e29 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 13:57:24.430783 21579 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_b80b7994ff294977bd92dc8f9ddd922e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:57:24.434443 22777 ts_tablet_manager.cc:1916] T 761ede873f5648ccb5216a4c0a8afa2e P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:24.435568 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet 76d4d7d706824892b075ab1aa4fbaf85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:57:24 UTC) from {username='slave'} at 127.0.0.1:50052
W20260925 13:57:24.436318 21580 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_8791c6009be44efab81f5f2252a3d8ae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:57:24.440586 21587 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:35994 (request call id 108) took 3077 ms. Trace:
I20260925 13:57:24.440939 21587 rpcz_store.cc:276] 0925 13:57:21.363343 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:21.363462 (+ 119us) service_pool.cc:224] Handling call
0925 13:57:23.932302 (+2568840us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 13:57:24.440523 (+508221us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":2525,"Hive Metastore.run_cpu_time_us":286,"Hive Metastore.run_wall_time_us":371628}
I20260925 13:57:24.414180 22781 raft_consensus.cc:515] T 88b875abab81457481d4b98570a74ea1 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:24.436293 21937 tablet_service.cc:1558] Processing DeleteTablet for tablet 76d4d7d706824892b075ab1aa4fbaf85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:57:24 UTC) from {username='slave'} at 127.0.0.1:55588
W20260925 13:57:24.444350 21586 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_4586487550ec46f08eae599e1c34ef17 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 13:57:24.450265 21893 leader_election.cc:343] T 88b875abab81457481d4b98570a74ea1 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 88b875abab81457481d4b98570a74ea1
I20260925 13:57:24.439096 22795 tablet_bootstrap.cc:492] T a06a5aac5d464d339c870df4ebd63d92 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:24.454444 21937 tablet_service.cc:1511] Processing CreateTablet for tablet 8ace3cd70b45451d8e9fb4b8a8d4dd0d (DEFAULT_TABLE table=default.table_d6100b0f2b294ab099c179a7681cb4ed [id=74ee67384ecf4220b7d08fdf0637aefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:24.455404 21937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ace3cd70b45451d8e9fb4b8a8d4dd0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:24.456930 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88b875abab81457481d4b98570a74ea1" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:24.457391 22107 raft_consensus.cc:3060] T 88b875abab81457481d4b98570a74ea1 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:24.457701 22781 leader_election.cc:290] T 88b875abab81457481d4b98570a74ea1 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Requested vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:24.463565 22107 raft_consensus.cc:2468] T 88b875abab81457481d4b98570a74ea1 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 1.
I20260925 13:57:24.464613 21896 leader_election.cc:304] T 88b875abab81457481d4b98570a74ea1 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36; no voters: 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:24.465536 22781 raft_consensus.cc:2804] T 88b875abab81457481d4b98570a74ea1 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:24.467415 22781 raft_consensus.cc:697] T 88b875abab81457481d4b98570a74ea1 P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: Becoming Leader. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Running, Role: LEADER
I20260925 13:57:24.468190 22781 consensus_queue.cc:237] T 88b875abab81457481d4b98570a74ea1 P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:24.469730 22795 tablet_bootstrap.cc:654] T a06a5aac5d464d339c870df4ebd63d92 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:24.476954 22083 tablet_service.cc:1511] Processing CreateTablet for tablet 8ace3cd70b45451d8e9fb4b8a8d4dd0d (DEFAULT_TABLE table=default.table_d6100b0f2b294ab099c179a7681cb4ed [id=74ee67384ecf4220b7d08fdf0637aefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:24.479919 22083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ace3cd70b45451d8e9fb4b8a8d4dd0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:24.490406 22592 raft_consensus.cc:493] T 565a7e8f453e464b8897d84e99429067 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:24.490823 22592 raft_consensus.cc:515] T 565a7e8f453e464b8897d84e99429067 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:24.492607 22592 leader_election.cc:290] T 565a7e8f453e464b8897d84e99429067 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 13:57:24.495885 21893 leader_election.cc:343] T 565a7e8f453e464b8897d84e99429067 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 565a7e8f453e464b8897d84e99429067
I20260925 13:57:24.495034 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565a7e8f453e464b8897d84e99429067" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:24.496546 22107 raft_consensus.cc:2468] T 565a7e8f453e464b8897d84e99429067 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 0.
I20260925 13:57:24.497606 21896 leader_election.cc:304] T 565a7e8f453e464b8897d84e99429067 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36; no voters: 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:24.498488 22592 raft_consensus.cc:2804] T 565a7e8f453e464b8897d84e99429067 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:24.498790 22592 raft_consensus.cc:493] T 565a7e8f453e464b8897d84e99429067 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:24.499141 22592 raft_consensus.cc:3060] T 565a7e8f453e464b8897d84e99429067 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:24.505805 22592 raft_consensus.cc:515] T 565a7e8f453e464b8897d84e99429067 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:24.509053 21893 leader_election.cc:343] T 565a7e8f453e464b8897d84e99429067 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 565a7e8f453e464b8897d84e99429067
I20260925 13:57:24.510372 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "565a7e8f453e464b8897d84e99429067" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:24.510841 22107 raft_consensus.cc:3060] T 565a7e8f453e464b8897d84e99429067 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:24.515590 22809 raft_consensus.cc:493] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:24.516171 22809 raft_consensus.cc:515] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:24.516660 22107 raft_consensus.cc:2468] T 565a7e8f453e464b8897d84e99429067 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 1.
I20260925 13:57:24.517638 21896 leader_election.cc:304] T 565a7e8f453e464b8897d84e99429067 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36; no voters: 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:24.518698 22676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e942d7f2e4864b5ca08e6e830085caa0" candidate_term: 5 candidate_status { last_received { term: 3 index: 172 } } ignore_live_leader: false dest_uuid: "3ea1e11bc6d84114b7616815b6af4635" is_pre_election: true
I20260925 13:57:24.521520 22809 leader_election.cc:290] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ea1e11bc6d84114b7616815b6af4635 (127.18.182.253:46177), a9bf5a5925ac42769e474adc84a1b725 (127.18.182.252:39283)
I20260925 13:57:24.523702 21598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e942d7f2e4864b5ca08e6e830085caa0" candidate_term: 5 candidate_status { last_received { term: 3 index: 172 } } ignore_live_leader: false dest_uuid: "a9bf5a5925ac42769e474adc84a1b725" is_pre_election: true
I20260925 13:57:24.525187 21422 leader_election.cc:304] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e942d7f2e4864b5ca08e6e830085caa0; no voters: 3ea1e11bc6d84114b7616815b6af4635, a9bf5a5925ac42769e474adc84a1b725
I20260925 13:57:24.525894 22813 raft_consensus.cc:2804] T 565a7e8f453e464b8897d84e99429067 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:24.528192 22795 tablet_bootstrap.cc:492] T a06a5aac5d464d339c870df4ebd63d92 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:24.533556 22795 ts_tablet_manager.cc:1403] T a06a5aac5d464d339c870df4ebd63d92 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.095s user 0.011s sys 0.000s
I20260925 13:57:24.533697 22809 raft_consensus.cc:2749] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260925 13:57:24.528470 22592 leader_election.cc:290] T 565a7e8f453e464b8897d84e99429067 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Requested vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:24.537367 22795 raft_consensus.cc:359] T a06a5aac5d464d339c870df4ebd63d92 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:24.538105 22795 raft_consensus.cc:385] T a06a5aac5d464d339c870df4ebd63d92 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:24.538410 22795 raft_consensus.cc:740] T a06a5aac5d464d339c870df4ebd63d92 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:24.539186 22795 consensus_queue.cc:260] T a06a5aac5d464d339c870df4ebd63d92 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:24.540679 22592 raft_consensus.cc:493] T 60ef9c152aec465eb0bb7dc286c74277 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:24.541132 22592 raft_consensus.cc:515] T 60ef9c152aec465eb0bb7dc286c74277 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:24.544513 21893 leader_election.cc:343] T 60ef9c152aec465eb0bb7dc286c74277 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 60ef9c152aec465eb0bb7dc286c74277
I20260925 13:57:24.545817 21585 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35994:
table { table_name: "default.Table_1baae11db5a24d50bd596853233007cb" } new_table_name: "default.Table_2a09482ff526433a8d028a964cfdfb0c" modify_external_catalogs: true
I20260925 13:57:24.546483 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60ef9c152aec465eb0bb7dc286c74277" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
W20260925 13:57:24.547592 21896 leader_election.cc:343] T 60ef9c152aec465eb0bb7dc286c74277 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:57:24.547967 21896 leader_election.cc:304] T 60ef9c152aec465eb0bb7dc286c74277 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee; no voters: 5e2731975dbf433aade4a859df9699e2, ee02f77594864349982492862d668a36
I20260925 13:57:24.548717 22592 leader_election.cc:290] T 60ef9c152aec465eb0bb7dc286c74277 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:24.549345 22592 raft_consensus.cc:2749] T 60ef9c152aec465eb0bb7dc286c74277 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:57:24.551209 22795 ts_tablet_manager.cc:1434] T a06a5aac5d464d339c870df4ebd63d92 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.017s user 0.005s sys 0.000s
I20260925 13:57:24.552150 22795 tablet_bootstrap.cc:492] T 04cd31f29357471a882157829ca700ab P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:24.553615 22777 ts_tablet_manager.cc:1929] T 761ede873f5648ccb5216a4c0a8afa2e P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:57:24.553983 22777 log.cc:1199] T 761ede873f5648ccb5216a4c0a8afa2e P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/761ede873f5648ccb5216a4c0a8afa2e
I20260925 13:57:24.556346 22776 ts_tablet_manager.cc:1929] T 761ede873f5648ccb5216a4c0a8afa2e P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:57:24.556785 22776 log.cc:1199] T 761ede873f5648ccb5216a4c0a8afa2e P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/761ede873f5648ccb5216a4c0a8afa2e
I20260925 13:57:24.557555 22776 ts_tablet_manager.cc:1950] T 761ede873f5648ccb5216a4c0a8afa2e P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:24.564256 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet 761ede873f5648ccb5216a4c0a8afa2e (table default.table_d6100b0f2b294ab099c179a7681cb4ed [id=74ee67384ecf4220b7d08fdf0637aefb]) successfully deleted
I20260925 13:57:24.564453 22794 tablet_bootstrap.cc:492] T 60ef9c152aec465eb0bb7dc286c74277 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:24.565588 22794 ts_tablet_manager.cc:1403] T 60ef9c152aec465eb0bb7dc286c74277 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.159s user 0.021s sys 0.008s
I20260925 13:57:24.566399 22776 tablet_replica.cc:333] T 76d4d7d706824892b075ab1aa4fbaf85 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:24.565196 22777 ts_tablet_manager.cc:1950] T 761ede873f5648ccb5216a4c0a8afa2e P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:24.570169 22781 raft_consensus.cc:493] T a06a5aac5d464d339c870df4ebd63d92 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:24.570600 22781 raft_consensus.cc:515] T a06a5aac5d464d339c870df4ebd63d92 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:24.568841 22776 raft_consensus.cc:2243] T 76d4d7d706824892b075ab1aa4fbaf85 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:24.571691 22776 raft_consensus.cc:2272] T 76d4d7d706824892b075ab1aa4fbaf85 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:24.573455 22781 leader_election.cc:290] T a06a5aac5d464d339c870df4ebd63d92 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 13:57:24.574777 21893 leader_election.cc:343] T a06a5aac5d464d339c870df4ebd63d92 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: a06a5aac5d464d339c870df4ebd63d92
I20260925 13:57:24.575522 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a06a5aac5d464d339c870df4ebd63d92" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:24.575835 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet 761ede873f5648ccb5216a4c0a8afa2e (table default.table_d6100b0f2b294ab099c179a7681cb4ed [id=74ee67384ecf4220b7d08fdf0637aefb]) successfully deleted
W20260925 13:57:24.577009 21896 leader_election.cc:343] T a06a5aac5d464d339c870df4ebd63d92 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:57:24.577442 22777 tablet_replica.cc:333] T 76d4d7d706824892b075ab1aa4fbaf85 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:24.575400 22794 raft_consensus.cc:359] T 60ef9c152aec465eb0bb7dc286c74277 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:24.579658 22794 raft_consensus.cc:385] T 60ef9c152aec465eb0bb7dc286c74277 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:24.579931 22794 raft_consensus.cc:740] T 60ef9c152aec465eb0bb7dc286c74277 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:24.580583 22794 consensus_queue.cc:260] T 60ef9c152aec465eb0bb7dc286c74277 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:24.589409 21587 master_service.cc:960] ReplaceTablet: received request to replace tablet 195c48349fb5410495299c9e71a72ad8 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:24.577360 21896 leader_election.cc:304] T a06a5aac5d464d339c870df4ebd63d92 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee; no voters: 5e2731975dbf433aade4a859df9699e2, ee02f77594864349982492862d668a36
I20260925 13:57:24.566566 22795 tablet_bootstrap.cc:654] T 04cd31f29357471a882157829ca700ab P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:24.590101 22794 ts_tablet_manager.cc:1434] T 60ef9c152aec465eb0bb7dc286c74277 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.024s user 0.001s sys 0.005s
I20260925 13:57:24.590847 22814 raft_consensus.cc:2749] T a06a5aac5d464d339c870df4ebd63d92 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:57:24.592237 22776 ts_tablet_manager.cc:1916] T 76d4d7d706824892b075ab1aa4fbaf85 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:24.591810 22777 raft_consensus.cc:2243] T 76d4d7d706824892b075ab1aa4fbaf85 P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:24.594601 22777 raft_consensus.cc:2272] T 76d4d7d706824892b075ab1aa4fbaf85 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:24.609009 21587 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet 195c48349fb5410495299c9e71a72ad8
I20260925 13:57:24.609148 21584 master_service.cc:960] ReplaceTablet: received request to replace tablet 76d4d7d706824892b075ab1aa4fbaf85 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:24.612454 21587 catalog_manager.cc:6934] ReplaceTablet: tablet 195c48349fb5410495299c9e71a72ad8 deleted and replaced by tablet ba1242d8795641e7a89006d154be1216
I20260925 13:57:24.611380 21937 tablet_service.cc:1558] Processing DeleteTablet for tablet 195c48349fb5410495299c9e71a72ad8 with delete_type TABLET_DATA_DELETED (replaced by tablet ba1242d8795641e7a89006d154be1216) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:24.613201 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet 195c48349fb5410495299c9e71a72ad8 with delete_type TABLET_DATA_DELETED (replaced by tablet ba1242d8795641e7a89006d154be1216) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:24.614471 22777 ts_tablet_manager.cc:1916] T 76d4d7d706824892b075ab1aa4fbaf85 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:24.623945 22794 tablet_bootstrap.cc:492] T 04cd31f29357471a882157829ca700ab P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:24.587554 22813 raft_consensus.cc:697] T 565a7e8f453e464b8897d84e99429067 P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: Becoming Leader. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Running, Role: LEADER
I20260925 13:57:24.633178 22813 consensus_queue.cc:237] T 565a7e8f453e464b8897d84e99429067 P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:24.646649 22795 tablet_bootstrap.cc:492] T 04cd31f29357471a882157829ca700ab P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:24.646960 22794 tablet_bootstrap.cc:654] T 04cd31f29357471a882157829ca700ab P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:24.647176 22795 ts_tablet_manager.cc:1403] T 04cd31f29357471a882157829ca700ab P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.095s user 0.015s sys 0.000s
I20260925 13:57:24.650460 21587 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35994:
table { table_name: "default.Table_f644beebc9834408bed8d57d2ac5c2ee" } modify_external_catalogs: true
I20260925 13:57:24.651144 21587 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35994:
table { table_name: "default.Table_f644beebc9834408bed8d57d2ac5c2ee" } modify_external_catalogs: true
I20260925 13:57:24.651109 22795 raft_consensus.cc:359] T 04cd31f29357471a882157829ca700ab P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:24.651743 22795 raft_consensus.cc:385] T 04cd31f29357471a882157829ca700ab P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:24.651998 22795 raft_consensus.cc:740] T 04cd31f29357471a882157829ca700ab P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:24.652647 22795 consensus_queue.cc:260] T 04cd31f29357471a882157829ca700ab P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:24.654846 22795 ts_tablet_manager.cc:1434] T 04cd31f29357471a882157829ca700ab P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260925 13:57:24.655825 22795 tablet_bootstrap.cc:492] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:24.668196 22776 ts_tablet_manager.cc:1929] T 76d4d7d706824892b075ab1aa4fbaf85 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:24.668741 22776 log.cc:1199] T 76d4d7d706824892b075ab1aa4fbaf85 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/76d4d7d706824892b075ab1aa4fbaf85
I20260925 13:57:24.669651 22776 ts_tablet_manager.cc:1950] T 76d4d7d706824892b075ab1aa4fbaf85 P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:24.674099 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet 76d4d7d706824892b075ab1aa4fbaf85 (table default.table_8008d16a92cf4c0dafb258a1122bffb5 [id=a22f86f8c2404aa69bdeb9883471453c]) successfully deleted
I20260925 13:57:24.675994 22777 ts_tablet_manager.cc:1929] T 76d4d7d706824892b075ab1aa4fbaf85 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:24.683478 22777 log.cc:1199] T 76d4d7d706824892b075ab1aa4fbaf85 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/76d4d7d706824892b075ab1aa4fbaf85
I20260925 13:57:24.684448 22777 ts_tablet_manager.cc:1950] T 76d4d7d706824892b075ab1aa4fbaf85 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:24.675449 22795 tablet_bootstrap.cc:654] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:24.685282 22776 tablet_replica.cc:333] T 195c48349fb5410495299c9e71a72ad8 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:24.688069 22776 raft_consensus.cc:2243] T 195c48349fb5410495299c9e71a72ad8 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:24.690201 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet 76d4d7d706824892b075ab1aa4fbaf85 (table default.table_8008d16a92cf4c0dafb258a1122bffb5 [id=a22f86f8c2404aa69bdeb9883471453c]) successfully deleted
I20260925 13:57:24.691480 22776 raft_consensus.cc:2272] T 195c48349fb5410495299c9e71a72ad8 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:24.693558 22777 tablet_replica.cc:333] T 195c48349fb5410495299c9e71a72ad8 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:24.694379 22777 raft_consensus.cc:2243] T 195c48349fb5410495299c9e71a72ad8 P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:24.695191 22776 ts_tablet_manager.cc:1916] T 195c48349fb5410495299c9e71a72ad8 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:24.697539 22777 raft_consensus.cc:2272] T 195c48349fb5410495299c9e71a72ad8 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:24.705240 22794 tablet_bootstrap.cc:492] T 04cd31f29357471a882157829ca700ab P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:24.705703 22794 ts_tablet_manager.cc:1403] T 04cd31f29357471a882157829ca700ab P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.082s user 0.009s sys 0.002s
I20260925 13:57:24.706197 22795 tablet_bootstrap.cc:492] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:24.706630 22795 ts_tablet_manager.cc:1403] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.051s user 0.016s sys 0.000s
I20260925 13:57:24.709172 22794 raft_consensus.cc:359] T 04cd31f29357471a882157829ca700ab P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:24.709228 22795 raft_consensus.cc:359] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:24.709824 22794 raft_consensus.cc:385] T 04cd31f29357471a882157829ca700ab P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:24.709877 22795 raft_consensus.cc:385] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:24.710106 22794 raft_consensus.cc:740] T 04cd31f29357471a882157829ca700ab P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:24.710156 22795 raft_consensus.cc:740] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:24.710850 22795 consensus_queue.cc:260] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:24.713721 22777 ts_tablet_manager.cc:1916] T 195c48349fb5410495299c9e71a72ad8 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:24.727955 22795 ts_tablet_manager.cc:1434] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.021s user 0.007s sys 0.000s
I20260925 13:57:24.729190 22813 raft_consensus.cc:493] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:24.729614 22813 raft_consensus.cc:515] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:24.731578 22813 leader_election.cc:290] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 13:57:24.732990 21893 leader_election.cc:343] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 8ace3cd70b45451d8e9fb4b8a8d4dd0d
I20260925 13:57:24.733098 22106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ace3cd70b45451d8e9fb4b8a8d4dd0d" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
W20260925 13:57:24.734377 21896 leader_election.cc:343] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:57:24.734724 21896 leader_election.cc:304] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee; no voters: 5e2731975dbf433aade4a859df9699e2, ee02f77594864349982492862d668a36
I20260925 13:57:24.735641 22813 raft_consensus.cc:2749] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:57:24.742409 22785 raft_consensus.cc:493] T 60ef9c152aec465eb0bb7dc286c74277 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:24.742821 22785 raft_consensus.cc:515] T 60ef9c152aec465eb0bb7dc286c74277 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:24.744547 22785 leader_election.cc:290] T 60ef9c152aec465eb0bb7dc286c74277 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 13:57:24.746996 22040 leader_election.cc:343] T 60ef9c152aec465eb0bb7dc286c74277 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 60ef9c152aec465eb0bb7dc286c74277
I20260925 13:57:24.748134 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60ef9c152aec465eb0bb7dc286c74277" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:24.748643 21960 raft_consensus.cc:2468] T 60ef9c152aec465eb0bb7dc286c74277 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee02f77594864349982492862d668a36 in term 0.
I20260925 13:57:24.749714 22043 leader_election.cc:304] T 60ef9c152aec465eb0bb7dc286c74277 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36; no voters: 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:24.750765 21937 tablet_service.cc:1511] Processing CreateTablet for tablet ba1242d8795641e7a89006d154be1216 (DEFAULT_TABLE table=default.table_bd714c618f7a4f16a69a69d5932b2014 [id=dfd05c11872140cfa86b2628c33ef6d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:24.751708 21937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba1242d8795641e7a89006d154be1216. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:24.752349 22785 raft_consensus.cc:2804] T 60ef9c152aec465eb0bb7dc286c74277 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:24.752620 22785 raft_consensus.cc:493] T 60ef9c152aec465eb0bb7dc286c74277 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:24.752887 22785 raft_consensus.cc:3060] T 60ef9c152aec465eb0bb7dc286c74277 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:24.756559 22776 ts_tablet_manager.cc:1929] T 195c48349fb5410495299c9e71a72ad8 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:24.756918 22776 log.cc:1199] T 195c48349fb5410495299c9e71a72ad8 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/195c48349fb5410495299c9e71a72ad8
I20260925 13:57:24.757658 22776 ts_tablet_manager.cc:1950] T 195c48349fb5410495299c9e71a72ad8 P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:24.738132 22794 consensus_queue.cc:260] T 04cd31f29357471a882157829ca700ab P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:24.767315 22083 tablet_service.cc:1511] Processing CreateTablet for tablet ba1242d8795641e7a89006d154be1216 (DEFAULT_TABLE table=default.table_bd714c618f7a4f16a69a69d5932b2014 [id=dfd05c11872140cfa86b2628c33ef6d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:24.768301 22083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba1242d8795641e7a89006d154be1216. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:24.777482 22777 ts_tablet_manager.cc:1929] T 195c48349fb5410495299c9e71a72ad8 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:24.777837 22777 log.cc:1199] T 195c48349fb5410495299c9e71a72ad8 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/195c48349fb5410495299c9e71a72ad8
I20260925 13:57:24.778558 22777 ts_tablet_manager.cc:1950] T 195c48349fb5410495299c9e71a72ad8 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:24.792719 22795 tablet_bootstrap.cc:492] T ba1242d8795641e7a89006d154be1216 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:24.759178 22785 raft_consensus.cc:515] T 60ef9c152aec465eb0bb7dc286c74277 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:24.799866 22040 leader_election.cc:343] T 60ef9c152aec465eb0bb7dc286c74277 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 60ef9c152aec465eb0bb7dc286c74277
I20260925 13:57:24.801113 22795 tablet_bootstrap.cc:654] T ba1242d8795641e7a89006d154be1216 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:24.801157 22785 leader_election.cc:290] T 60ef9c152aec465eb0bb7dc286c74277 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 election: Requested vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:24.801622 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60ef9c152aec465eb0bb7dc286c74277" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
I20260925 13:57:24.802094 21960 raft_consensus.cc:3060] T 60ef9c152aec465eb0bb7dc286c74277 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:24.807731 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet 195c48349fb5410495299c9e71a72ad8 (table default.table_bd714c618f7a4f16a69a69d5932b2014 [id=dfd05c11872140cfa86b2628c33ef6d8]) successfully deleted
I20260925 13:57:24.808732 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet 195c48349fb5410495299c9e71a72ad8 (table default.table_bd714c618f7a4f16a69a69d5932b2014 [id=dfd05c11872140cfa86b2628c33ef6d8]) successfully deleted
I20260925 13:57:24.821082 21960 raft_consensus.cc:2468] T 60ef9c152aec465eb0bb7dc286c74277 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee02f77594864349982492862d668a36 in term 1.
I20260925 13:57:24.822244 22043 leader_election.cc:304] T 60ef9c152aec465eb0bb7dc286c74277 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36; no voters: 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:24.822983 22795 tablet_bootstrap.cc:492] T ba1242d8795641e7a89006d154be1216 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:24.823635 22819 raft_consensus.cc:2804] T 60ef9c152aec465eb0bb7dc286c74277 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:24.823838 22795 ts_tablet_manager.cc:1403] T ba1242d8795641e7a89006d154be1216 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.031s user 0.010s sys 0.001s
I20260925 13:57:24.824039 22819 raft_consensus.cc:697] T 60ef9c152aec465eb0bb7dc286c74277 P ee02f77594864349982492862d668a36 [term 1 LEADER]: Becoming Leader. State: Replica: ee02f77594864349982492862d668a36, State: Running, Role: LEADER
I20260925 13:57:24.824759 22819 consensus_queue.cc:237] T 60ef9c152aec465eb0bb7dc286c74277 P ee02f77594864349982492862d668a36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:24.826612 22795 raft_consensus.cc:359] T ba1242d8795641e7a89006d154be1216 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:24.827224 22795 raft_consensus.cc:385] T ba1242d8795641e7a89006d154be1216 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:24.827545 21584 master_service.cc:960] ReplaceTablet: received request to replace tablet fb781a032ba649de8a214925fdb374b5 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:24.827636 22795 raft_consensus.cc:740] T ba1242d8795641e7a89006d154be1216 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:24.828302 22795 consensus_queue.cc:260] T ba1242d8795641e7a89006d154be1216 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:24.833293 22794 ts_tablet_manager.cc:1434] T 04cd31f29357471a882157829ca700ab P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.127s user 0.007s sys 0.000s
I20260925 13:57:24.836728 22794 tablet_bootstrap.cc:492] T a06a5aac5d464d339c870df4ebd63d92 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:24.841171 22795 ts_tablet_manager.cc:1434] T ba1242d8795641e7a89006d154be1216 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.017s user 0.005s sys 0.000s
I20260925 13:57:24.844309 21584 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet fb781a032ba649de8a214925fdb374b5
I20260925 13:57:24.844714 21583 master_service.cc:960] ReplaceTablet: received request to replace tablet 794784c42d27441f829f954b6929a097 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:24.845570 21584 catalog_manager.cc:6934] ReplaceTablet: tablet fb781a032ba649de8a214925fdb374b5 deleted and replaced by tablet 242acfcc09ca4f879e5e44a73ef16cec
I20260925 13:57:24.849234 21940 tablet_service.cc:1558] Processing DeleteTablet for tablet fb781a032ba649de8a214925fdb374b5 with delete_type TABLET_DATA_DELETED (replaced by tablet 242acfcc09ca4f879e5e44a73ef16cec) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:24.849946 22777 tablet_replica.cc:333] T fb781a032ba649de8a214925fdb374b5 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:24.850313 22084 tablet_service.cc:1558] Processing DeleteTablet for tablet fb781a032ba649de8a214925fdb374b5 with delete_type TABLET_DATA_DELETED (replaced by tablet 242acfcc09ca4f879e5e44a73ef16cec) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:24.850765 22777 raft_consensus.cc:2243] T fb781a032ba649de8a214925fdb374b5 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:24.851029 22776 tablet_replica.cc:333] T fb781a032ba649de8a214925fdb374b5 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:24.851266 22777 raft_consensus.cc:2272] T fb781a032ba649de8a214925fdb374b5 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:24.851848 22776 raft_consensus.cc:2243] T fb781a032ba649de8a214925fdb374b5 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:24.852341 22776 raft_consensus.cc:2272] T fb781a032ba649de8a214925fdb374b5 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:24.853284 22777 ts_tablet_manager.cc:1916] T fb781a032ba649de8a214925fdb374b5 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:24.854198 22776 ts_tablet_manager.cc:1916] T fb781a032ba649de8a214925fdb374b5 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:24.865202 22794 tablet_bootstrap.cc:654] T a06a5aac5d464d339c870df4ebd63d92 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:24.876439 21583 catalog_manager.cc:6133] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Sending DeleteTablet for 3 replicas of tablet 794784c42d27441f829f954b6929a097
I20260925 13:57:24.878250 21937 tablet_service.cc:1558] Processing DeleteTablet for tablet 794784c42d27441f829f954b6929a097 with delete_type TABLET_DATA_DELETED (replaced by tablet 09199f2141f746a29f410907e2a5ae37) from {username='slave'} at 127.0.0.1:55588
I20260925 13:57:24.879455 22084 tablet_service.cc:1558] Processing DeleteTablet for tablet 794784c42d27441f829f954b6929a097 with delete_type TABLET_DATA_DELETED (replaced by tablet 09199f2141f746a29f410907e2a5ae37) from {username='slave'} at 127.0.0.1:50052
I20260925 13:57:24.879989 21583 catalog_manager.cc:6934] ReplaceTablet: tablet 794784c42d27441f829f954b6929a097 deleted and replaced by tablet 09199f2141f746a29f410907e2a5ae37
I20260925 13:57:24.882548 22777 ts_tablet_manager.cc:1929] T fb781a032ba649de8a214925fdb374b5 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:24.882915 22777 log.cc:1199] T fb781a032ba649de8a214925fdb374b5 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/fb781a032ba649de8a214925fdb374b5
I20260925 13:57:24.883677 22777 ts_tablet_manager.cc:1950] T fb781a032ba649de8a214925fdb374b5 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:24.886343 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet fb781a032ba649de8a214925fdb374b5 (table default.table_d8ce7cda4ecf4422a5e74c80d7ebf4da [id=af0f962d5d5d4a17a3f576ab6a3b4df0]) successfully deleted
W20260925 13:57:24.898842 22609 leader_election.cc:336] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a9bf5a5925ac42769e474adc84a1b725 (127.18.182.252:39283): Timed out: connection negotiation to 127.18.182.252:39283 for RPC RequestConsensusVote timed out after 0.684s (ON_OUTBOUND_QUEUE)
I20260925 13:57:24.899304 22609 leader_election.cc:304] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ea1e11bc6d84114b7616815b6af4635; no voters: a9bf5a5925ac42769e474adc84a1b725, e942d7f2e4864b5ca08e6e830085caa0
I20260925 13:57:24.900899 22777 tablet_replica.cc:333] T 794784c42d27441f829f954b6929a097 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:24.901700 22777 raft_consensus.cc:2243] T 794784c42d27441f829f954b6929a097 P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:24.902714 22777 raft_consensus.cc:2272] T 794784c42d27441f829f954b6929a097 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:24.908324 22777 ts_tablet_manager.cc:1916] T 794784c42d27441f829f954b6929a097 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:24.932974 22777 ts_tablet_manager.cc:1929] T 794784c42d27441f829f954b6929a097 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:57:24.933346 22777 log.cc:1199] T 794784c42d27441f829f954b6929a097 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/794784c42d27441f829f954b6929a097
I20260925 13:57:24.934063 22777 ts_tablet_manager.cc:1950] T 794784c42d27441f829f954b6929a097 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:24.936754 21575 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet 794784c42d27441f829f954b6929a097 (table default.table_683c0db2850941be84798e9dca3fe490 [id=7377fa3ea9664b47a930d9c5f88c60fa]) successfully deleted
I20260925 13:57:24.957221 22823 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 3 FOLLOWER]: Advancing to term 4
W20260925 13:57:24.956389 22609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.252:39283, user_credentials={real_user=slave}} blocked reactor thread for 57608.4us
I20260925 13:57:24.971747 22794 tablet_bootstrap.cc:492] T a06a5aac5d464d339c870df4ebd63d92 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:24.972168 22794 ts_tablet_manager.cc:1403] T a06a5aac5d464d339c870df4ebd63d92 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.136s user 0.007s sys 0.006s
I20260925 13:57:24.974565 22794 raft_consensus.cc:359] T a06a5aac5d464d339c870df4ebd63d92 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:24.982756 22823 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260925 13:57:24.975186 22794 raft_consensus.cc:385] T a06a5aac5d464d339c870df4ebd63d92 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:24.995532 22794 raft_consensus.cc:740] T a06a5aac5d464d339c870df4ebd63d92 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:24.996338 22794 consensus_queue.cc:260] T a06a5aac5d464d339c870df4ebd63d92 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:24.997920 22676 raft_consensus.cc:1240] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 4 FOLLOWER]: Rejecting Update request from peer a9bf5a5925ac42769e474adc84a1b725 for earlier term 3. Current term is 4. Ops: [3.196-3.196]
W20260925 13:57:25.003217 21732 consensus_peers.cc:597] T fb781a032ba649de8a214925fdb374b5 P 5e2731975dbf433aade4a859df9699e2 -> Peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Couldn't send request to peer ee02f77594864349982492862d668a36. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:57:25.004316 22764 consensus_queue.cc:1059] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 }, Status: INVALID_TERM, Last received: 3.195, Next index: 196, Last known committed idx: 195, Time since last communication: 0.000s
I20260925 13:57:25.005131 22764 raft_consensus.cc:3055] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 3 LEADER]: Stepping down as leader of term 3
I20260925 13:57:25.005484 22764 raft_consensus.cc:740] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a9bf5a5925ac42769e474adc84a1b725, State: Running, Role: LEADER
I20260925 13:57:25.006248 22764 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 195, Committed index: 195, Last appended: 3.196, Last appended by leader: 196, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:25.007289 22764 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 3 FOLLOWER]: Advancing to term 4
I20260925 13:57:25.019461 22776 ts_tablet_manager.cc:1929] T fb781a032ba649de8a214925fdb374b5 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:25.019842 22776 log.cc:1199] T fb781a032ba649de8a214925fdb374b5 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/fb781a032ba649de8a214925fdb374b5
I20260925 13:57:25.022565 22794 ts_tablet_manager.cc:1434] T a06a5aac5d464d339c870df4ebd63d92 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.050s user 0.005s sys 0.001s
I20260925 13:57:25.025523 22776 ts_tablet_manager.cc:1950] T fb781a032ba649de8a214925fdb374b5 P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:25.033408 22794 tablet_bootstrap.cc:492] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:25.036269 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet fb781a032ba649de8a214925fdb374b5 (table default.table_d8ce7cda4ecf4422a5e74c80d7ebf4da [id=af0f962d5d5d4a17a3f576ab6a3b4df0]) successfully deleted
I20260925 13:57:25.037122 22776 tablet_replica.cc:333] T 794784c42d27441f829f954b6929a097 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:25.037851 22776 raft_consensus.cc:2243] T 794784c42d27441f829f954b6929a097 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:25.038332 22776 raft_consensus.cc:2272] T 794784c42d27441f829f954b6929a097 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:25.060166 21584 master_service.cc:960] ReplaceTablet: received request to replace tablet c42767d8f674410188835ceefa644eb5 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:25.060745 22776 ts_tablet_manager.cc:1916] T 794784c42d27441f829f954b6929a097 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:25.062457 22794 tablet_bootstrap.cc:654] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
W20260925 13:57:25.068607 22165 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.18.182.252:39283)
I20260925 13:57:25.086771 22819 raft_consensus.cc:493] T a06a5aac5d464d339c870df4ebd63d92 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:25.087180 22819 raft_consensus.cc:515] T a06a5aac5d464d339c870df4ebd63d92 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:25.088991 22819 leader_election.cc:290] T a06a5aac5d464d339c870df4ebd63d92 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:25.090397 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a06a5aac5d464d339c870df4ebd63d92" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:25.090926 21960 raft_consensus.cc:2468] T a06a5aac5d464d339c870df4ebd63d92 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee02f77594864349982492862d668a36 in term 0.
W20260925 13:57:25.091434 22040 leader_election.cc:343] T a06a5aac5d464d339c870df4ebd63d92 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: a06a5aac5d464d339c870df4ebd63d92
W20260925 13:57:25.092850 21732 consensus_peers.cc:597] T fb781a032ba649de8a214925fdb374b5 P 5e2731975dbf433aade4a859df9699e2 -> Peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Couldn't send request to peer 145e520e0a5a445d943a53cf8c27efee. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fb781a032ba649de8a214925fdb374b5. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:57:25.093828 22043 leader_election.cc:304] T a06a5aac5d464d339c870df4ebd63d92 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36; no voters: 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:25.094800 22819 raft_consensus.cc:2804] T a06a5aac5d464d339c870df4ebd63d92 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:25.095147 22819 raft_consensus.cc:493] T a06a5aac5d464d339c870df4ebd63d92 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:25.095255 21584 master_service.cc:960] ReplaceTablet: received request to replace tablet 76d4d7d706824892b075ab1aa4fbaf85 from {username='slave'} at 127.0.0.1:35994
I20260925 13:57:25.095465 22819 raft_consensus.cc:3060] T a06a5aac5d464d339c870df4ebd63d92 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:25.105717 21588 catalog_manager.cc:2770] deleted HMS catalog entry for table default.table_fce30b75d46f47089d7a31e04841dcdf [id=ec768fd401eb409aaa08df2a6cdfa2f3]
I20260925 13:57:25.102851 22819 raft_consensus.cc:515] T a06a5aac5d464d339c870df4ebd63d92 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:25.112142 22040 leader_election.cc:343] T a06a5aac5d464d339c870df4ebd63d92 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: a06a5aac5d464d339c870df4ebd63d92
I20260925 13:57:25.117412 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a06a5aac5d464d339c870df4ebd63d92" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
I20260925 13:57:25.117936 21960 raft_consensus.cc:3060] T a06a5aac5d464d339c870df4ebd63d92 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:25.123945 21960 raft_consensus.cc:2468] T a06a5aac5d464d339c870df4ebd63d92 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee02f77594864349982492862d668a36 in term 1.
I20260925 13:57:25.124819 22043 leader_election.cc:304] T a06a5aac5d464d339c870df4ebd63d92 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36; no voters: 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:25.125207 22819 leader_election.cc:290] T a06a5aac5d464d339c870df4ebd63d92 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 election: Requested vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:25.125600 22785 raft_consensus.cc:2804] T a06a5aac5d464d339c870df4ebd63d92 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:25.126011 22785 raft_consensus.cc:697] T a06a5aac5d464d339c870df4ebd63d92 P ee02f77594864349982492862d668a36 [term 1 LEADER]: Becoming Leader. State: Replica: ee02f77594864349982492862d668a36, State: Running, Role: LEADER
I20260925 13:57:25.127210 22794 tablet_bootstrap.cc:492] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:25.126785 22785 consensus_queue.cc:237] T a06a5aac5d464d339c870df4ebd63d92 P ee02f77594864349982492862d668a36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:25.127641 22794 ts_tablet_manager.cc:1403] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.094s user 0.007s sys 0.011s
I20260925 13:57:25.146216 22794 raft_consensus.cc:359] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:25.151799 22794 raft_consensus.cc:385] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:25.152125 22794 raft_consensus.cc:740] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:25.152881 22794 consensus_queue.cc:260] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:25.155001 22794 ts_tablet_manager.cc:1434] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.027s user 0.004s sys 0.001s
I20260925 13:57:25.155923 22794 tablet_bootstrap.cc:492] T ba1242d8795641e7a89006d154be1216 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:25.156091 22776 ts_tablet_manager.cc:1929] T 794784c42d27441f829f954b6929a097 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:57:25.156464 22776 log.cc:1199] T 794784c42d27441f829f954b6929a097 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/794784c42d27441f829f954b6929a097
I20260925 13:57:25.157157 22776 ts_tablet_manager.cc:1950] T 794784c42d27441f829f954b6929a097 P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:25.159600 21575 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet 794784c42d27441f829f954b6929a097 (table default.table_683c0db2850941be84798e9dca3fe490 [id=7377fa3ea9664b47a930d9c5f88c60fa]) successfully deleted
I20260925 13:57:25.190430 22794 tablet_bootstrap.cc:654] T ba1242d8795641e7a89006d154be1216 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:25.223511 22794 tablet_bootstrap.cc:492] T ba1242d8795641e7a89006d154be1216 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:25.223951 22794 ts_tablet_manager.cc:1403] T ba1242d8795641e7a89006d154be1216 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.068s user 0.007s sys 0.005s
I20260925 13:57:25.226634 22794 raft_consensus.cc:359] T ba1242d8795641e7a89006d154be1216 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:25.227303 22794 raft_consensus.cc:385] T ba1242d8795641e7a89006d154be1216 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:25.227613 22794 raft_consensus.cc:740] T ba1242d8795641e7a89006d154be1216 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:25.237541 22813 consensus_queue.cc:1048] T 88b875abab81457481d4b98570a74ea1 P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:25.228276 22794 consensus_queue.cc:260] T ba1242d8795641e7a89006d154be1216 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:25.261320 22794 ts_tablet_manager.cc:1434] T ba1242d8795641e7a89006d154be1216 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.037s user 0.005s sys 0.001s
I20260925 13:57:25.292202 21581 catalog_manager.cc:3541] renamed table default.table_bd714c618f7a4f16a69a69d5932b2014 [id=dfd05c11872140cfa86b2628c33ef6d8] in HMS: new name default.table_97e22ed3359342caaff80295b5d37db2
W20260925 13:57:25.314035 21893 consensus_peers.cc:597] T 88b875abab81457481d4b98570a74ea1 P 145e520e0a5a445d943a53cf8c27efee -> Peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Couldn't send request to peer 5e2731975dbf433aade4a859df9699e2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 88b875abab81457481d4b98570a74ea1. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:57:25.350021 22813 raft_consensus.cc:493] T ba1242d8795641e7a89006d154be1216 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:25.350507 22813 raft_consensus.cc:515] T ba1242d8795641e7a89006d154be1216 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:25.352545 22813 leader_election.cc:290] T ba1242d8795641e7a89006d154be1216 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:25.354780 22106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1242d8795641e7a89006d154be1216" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:25.355309 22106 raft_consensus.cc:2468] T ba1242d8795641e7a89006d154be1216 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 0.
I20260925 13:57:25.356436 22834 raft_consensus.cc:493] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:25.357139 21896 leader_election.cc:304] T ba1242d8795641e7a89006d154be1216 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36; no voters:
W20260925 13:57:25.358311 21893 leader_election.cc:343] T ba1242d8795641e7a89006d154be1216 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: ba1242d8795641e7a89006d154be1216
I20260925 13:57:25.359077 22813 raft_consensus.cc:2804] T ba1242d8795641e7a89006d154be1216 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:25.359468 22813 raft_consensus.cc:493] T ba1242d8795641e7a89006d154be1216 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:25.359824 22813 raft_consensus.cc:3060] T ba1242d8795641e7a89006d154be1216 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:25.366361 22813 raft_consensus.cc:515] T ba1242d8795641e7a89006d154be1216 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:25.357968 22834 raft_consensus.cc:515] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:25.377272 22676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e942d7f2e4864b5ca08e6e830085caa0" candidate_term: 5 candidate_status { last_received { term: 3 index: 172 } } ignore_live_leader: false dest_uuid: "3ea1e11bc6d84114b7616815b6af4635" is_pre_election: true
I20260925 13:57:25.381546 22106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba1242d8795641e7a89006d154be1216" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:25.382050 22106 raft_consensus.cc:3060] T ba1242d8795641e7a89006d154be1216 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
W20260925 13:57:25.383968 21893 leader_election.cc:343] T ba1242d8795641e7a89006d154be1216 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: ba1242d8795641e7a89006d154be1216
I20260925 13:57:25.389431 22106 raft_consensus.cc:2468] T ba1242d8795641e7a89006d154be1216 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 1.
I20260925 13:57:25.390555 21896 leader_election.cc:304] T ba1242d8795641e7a89006d154be1216 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36; no voters: 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:25.391080 21598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e942d7f2e4864b5ca08e6e830085caa0" candidate_term: 5 candidate_status { last_received { term: 3 index: 172 } } ignore_live_leader: false dest_uuid: "a9bf5a5925ac42769e474adc84a1b725" is_pre_election: true
I20260925 13:57:25.391491 22814 raft_consensus.cc:2804] T ba1242d8795641e7a89006d154be1216 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:25.391880 21598 raft_consensus.cc:2410] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e942d7f2e4864b5ca08e6e830085caa0 for term 5 because replica has last-logged OpId of term: 3 index: 196, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 172.
I20260925 13:57:25.391912 22814 raft_consensus.cc:697] T ba1242d8795641e7a89006d154be1216 P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: Becoming Leader. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Running, Role: LEADER
I20260925 13:57:25.392699 22814 consensus_queue.cc:237] T ba1242d8795641e7a89006d154be1216 P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:25.393955 21422 leader_election.cc:304] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e942d7f2e4864b5ca08e6e830085caa0; no voters: 3ea1e11bc6d84114b7616815b6af4635, a9bf5a5925ac42769e474adc84a1b725
I20260925 13:57:25.394867 21582 catalog_manager.cc:2213] created HMS catalog entry for table [id=7cba8a57ef5541a881bc69ed98657f63]
W20260925 13:57:25.396754 21582 catalog_manager.cc:2241] Illegal state: an error occurred while writing to the sys-catalog: replica a9bf5a5925ac42769e474adc84a1b725 is not leader of this config: current role FOLLOWER
I20260925 13:57:25.399472 22813 leader_election.cc:290] T ba1242d8795641e7a89006d154be1216 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Requested vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 13:57:25.399722 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:57:25.407290 22834 leader_election.cc:290] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3ea1e11bc6d84114b7616815b6af4635 (127.18.182.253:46177), a9bf5a5925ac42769e474adc84a1b725 (127.18.182.252:39283)
I20260925 13:57:25.409058 22834 raft_consensus.cc:2749] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260925 13:57:25.415669 22813 raft_consensus.cc:493] T 04cd31f29357471a882157829ca700ab P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:25.416105 22813 raft_consensus.cc:515] T 04cd31f29357471a882157829ca700ab P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:25.419602 21893 leader_election.cc:343] T 04cd31f29357471a882157829ca700ab P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 04cd31f29357471a882157829ca700ab
I20260925 13:57:25.421520 22106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04cd31f29357471a882157829ca700ab" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:25.422097 22106 raft_consensus.cc:2468] T 04cd31f29357471a882157829ca700ab P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 0.
I20260925 13:57:25.423314 21896 leader_election.cc:304] T 04cd31f29357471a882157829ca700ab P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36; no voters: 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:25.428277 22592 raft_consensus.cc:2804] T 04cd31f29357471a882157829ca700ab P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:25.428608 22592 raft_consensus.cc:493] T 04cd31f29357471a882157829ca700ab P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:25.428893 22592 raft_consensus.cc:3060] T 04cd31f29357471a882157829ca700ab P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
W20260925 13:57:25.440335 21893 consensus_peers.cc:597] T 565a7e8f453e464b8897d84e99429067 P 145e520e0a5a445d943a53cf8c27efee -> Peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Couldn't send request to peer 5e2731975dbf433aade4a859df9699e2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 565a7e8f453e464b8897d84e99429067. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:57:25.459031 21579 catalog_manager.cc:2213] created HMS catalog entry for table [id=e6c92379188140049e6f4d8839a700f7]
I20260925 13:57:25.459342 22592 raft_consensus.cc:515] T 04cd31f29357471a882157829ca700ab P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:25.460892 21579 catalog_manager.cc:2241] Illegal state: an error occurred while writing to the sys-catalog: replica a9bf5a5925ac42769e474adc84a1b725 is not leader of this config: current role FOLLOWER
I20260925 13:57:25.461737 22813 leader_election.cc:290] T 04cd31f29357471a882157829ca700ab P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 13:57:25.462646 21893 leader_election.cc:343] T 04cd31f29357471a882157829ca700ab P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 04cd31f29357471a882157829ca700ab
I20260925 13:57:25.464704 22106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04cd31f29357471a882157829ca700ab" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:25.465261 22106 raft_consensus.cc:3060] T 04cd31f29357471a882157829ca700ab P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:25.471336 22106 raft_consensus.cc:2468] T 04cd31f29357471a882157829ca700ab P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 1.
W20260925 13:57:25.474143 22837 log.cc:927] Time spent T ba1242d8795641e7a89006d154be1216 P 145e520e0a5a445d943a53cf8c27efee: Append to log took a long time: real 0.069s user 0.000s sys 0.000s
I20260925 13:57:25.474507 22592 leader_election.cc:290] T 04cd31f29357471a882157829ca700ab P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Requested vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:25.476982 21896 leader_election.cc:304] T 04cd31f29357471a882157829ca700ab P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36; no voters: 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:25.477973 22592 raft_consensus.cc:2804] T 04cd31f29357471a882157829ca700ab P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:25.481467 22814 consensus_queue.cc:1048] T 565a7e8f453e464b8897d84e99429067 P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:25.514132 22764 raft_consensus.cc:493] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:25.514951 22764 raft_consensus.cc:515] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:25.517115 22764 leader_election.cc:290] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e942d7f2e4864b5ca08e6e830085caa0 (127.18.182.254:43163), 3ea1e11bc6d84114b7616815b6af4635 (127.18.182.253:46177)
I20260925 13:57:25.521073 22676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9bf5a5925ac42769e474adc84a1b725" candidate_term: 5 candidate_status { last_received { term: 3 index: 196 } } ignore_live_leader: false dest_uuid: "3ea1e11bc6d84114b7616815b6af4635" is_pre_election: true
I20260925 13:57:25.521585 22676 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9bf5a5925ac42769e474adc84a1b725 in term 4.
I20260925 13:57:25.523373 21573 leader_election.cc:304] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea1e11bc6d84114b7616815b6af4635, a9bf5a5925ac42769e474adc84a1b725; no voters:
I20260925 13:57:25.536544 22764 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260925 13:57:25.537051 22764 raft_consensus.cc:493] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:25.540452 22764 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 4 FOLLOWER]: Advancing to term 5
I20260925 13:57:25.559595 22592 raft_consensus.cc:697] T 04cd31f29357471a882157829ca700ab P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: Becoming Leader. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Running, Role: LEADER
I20260925 13:57:25.561441 22592 consensus_queue.cc:237] T 04cd31f29357471a882157829ca700ab P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:25.579483 22840 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:25.584324 21580 catalog_manager.cc:2213] created HMS catalog entry for table [id=616ee6b561b348198e3d268025078dde]
I20260925 13:57:25.584838 22840 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
W20260925 13:57:25.586194 21580 catalog_manager.cc:2241] Illegal state: an error occurred while writing to the sys-catalog: replica a9bf5a5925ac42769e474adc84a1b725 is not leader of this config: current role FOLLOWER
W20260925 13:57:25.588660 21896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.195:40873, user_credentials={real_user=slave}} blocked reactor thread for 81433.3us
I20260925 13:57:25.599509 21441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ea1e11bc6d84114b7616815b6af4635" candidate_term: 5 candidate_status { last_received { term: 3 index: 195 } } ignore_live_leader: false dest_uuid: "e942d7f2e4864b5ca08e6e830085caa0" is_pre_election: true
I20260925 13:57:25.600055 21441 raft_consensus.cc:2468] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ea1e11bc6d84114b7616815b6af4635 in term 4.
I20260925 13:57:25.600360 22840 leader_election.cc:290] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e942d7f2e4864b5ca08e6e830085caa0 (127.18.182.254:43163), a9bf5a5925ac42769e474adc84a1b725 (127.18.182.252:39283)
I20260925 13:57:25.601398 22602 leader_election.cc:304] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea1e11bc6d84114b7616815b6af4635, e942d7f2e4864b5ca08e6e830085caa0; no voters:
I20260925 13:57:25.574971 22764 raft_consensus.cc:515] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:25.608418 22840 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260925 13:57:25.609462 22840 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:25.612783 22676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9bf5a5925ac42769e474adc84a1b725" candidate_term: 5 candidate_status { last_received { term: 3 index: 196 } } ignore_live_leader: false dest_uuid: "3ea1e11bc6d84114b7616815b6af4635"
I20260925 13:57:25.614668 22840 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 4 FOLLOWER]: Advancing to term 5
I20260925 13:57:25.615393 22764 leader_election.cc:290] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [CANDIDATE]: Term 5 election: Requested vote from peers e942d7f2e4864b5ca08e6e830085caa0 (127.18.182.254:43163), 3ea1e11bc6d84114b7616815b6af4635 (127.18.182.253:46177)
I20260925 13:57:25.635231 22840 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:25.640377 22676 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a9bf5a5925ac42769e474adc84a1b725 in current term 5: Already voted for candidate 3ea1e11bc6d84114b7616815b6af4635 in this term.
I20260925 13:57:25.658743 21441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ea1e11bc6d84114b7616815b6af4635" candidate_term: 5 candidate_status { last_received { term: 3 index: 195 } } ignore_live_leader: false dest_uuid: "e942d7f2e4864b5ca08e6e830085caa0"
I20260925 13:57:25.659226 21441 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 4 FOLLOWER]: Advancing to term 5
I20260925 13:57:25.666438 21586 catalog_manager.cc:2213] created HMS catalog entry for table [id=3e0dab6b1c2a48af9ada15882ebe2e9e]
I20260925 13:57:25.667587 21441 raft_consensus.cc:2468] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea1e11bc6d84114b7616815b6af4635 in term 5.
W20260925 13:57:25.668417 21586 catalog_manager.cc:2241] Illegal state: an error occurred while writing to the sys-catalog: replica a9bf5a5925ac42769e474adc84a1b725 is not leader of this config: current role FOLLOWER
I20260925 13:57:25.669363 22602 leader_election.cc:304] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea1e11bc6d84114b7616815b6af4635, e942d7f2e4864b5ca08e6e830085caa0; no voters:
I20260925 13:57:25.679147 21643 catalog_manager.cc:2811] Deleting table default.table_fce30b75d46f47089d7a31e04841dcdf [id=ec768fd401eb409aaa08df2a6cdfa2f3] in response to Hive Metastore notification log event 31
W20260925 13:57:25.683540 21643 catalog_manager.cc:2887] Illegal state: an error occurred while updating the sys-catalog: replica a9bf5a5925ac42769e474adc84a1b725 is not leader of this config: current role FOLLOWER
I20260925 13:57:25.687950 22840 leader_election.cc:290] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [CANDIDATE]: Term 5 election: Requested vote from peers e942d7f2e4864b5ca08e6e830085caa0 (127.18.182.254:43163), a9bf5a5925ac42769e474adc84a1b725 (127.18.182.252:39283)
W20260925 13:57:25.688635 21643 hms_notification_log_listener.cc:330] Failed to handle Hive Metastore notification: 31 DROP_TABLE default.table_fce30b75d46f47089d7a31e04841dcdf: Illegal state: an error occurred while updating the sys-catalog: replica a9bf5a5925ac42769e474adc84a1b725 is not leader of this config: current role FOLLOWER
W20260925 13:57:25.690879 21643 hms_notification_log_listener.cc:143] Hive Metastore notification log listener poll failed: Service unavailable: lost leadership while handling Hive Metastore notification log events: an error occurred while updating the sys-catalog: replica a9bf5a5925ac42769e474adc84a1b725 is not leader of this config: current role FOLLOWER
I20260925 13:57:25.691524 22840 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 5 FOLLOWER]: Leader election won for term 5
I20260925 13:57:25.692234 21585 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:35994 (request call id 122) took 1146 ms. Trace:
I20260925 13:57:25.692538 21585 rpcz_store.cc:276] 0925 13:57:24.545371 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:24.545508 (+ 137us) service_pool.cc:224] Handling call
0925 13:57:25.692168 (+1146660us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260925 13:57:25.692878 22840 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 5 LEADER]: Becoming Leader. State: Replica: 3ea1e11bc6d84114b7616815b6af4635, State: Running, Role: LEADER
I20260925 13:57:25.693728 22840 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 195, Committed index: 195, Last appended: 3.195, Last appended by leader: 195, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
W20260925 13:57:25.694175 22165 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.18.182.252:39283)
I20260925 13:57:25.699643 21588 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:35994 (request call id 109) took 4334 ms. Trace:
I20260925 13:57:25.700089 21588 rpcz_store.cc:276] 0925 13:57:21.365458 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:21.367902 (+ 2444us) service_pool.cc:224] Handling call
0925 13:57:24.338053 (+2970151us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 13:57:25.699559 (+1361506us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":59991,"Hive Metastore.run_cpu_time_us":315,"Hive Metastore.run_wall_time_us":706603}
I20260925 13:57:25.703159 21581 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:35994 (request call id 110) took 4337 ms. Trace:
I20260925 13:57:25.703408 21581 rpcz_store.cc:276] 0925 13:57:21.365697 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:21.413832 (+ 48135us) service_pool.cc:224] Handling call
0925 13:57:24.338903 (+2925071us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 13:57:25.701661 (+1362758us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 13:57:25.702463 (+ 802us) catalog_manager.cc:3764] Apply alter schema
0925 13:57:25.703111 (+ 648us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":765818,"Hive Metastore.run_cpu_time_us":1792,"Hive Metastore.run_wall_time_us":186640}
I20260925 13:57:25.714488 22842 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ea1e11bc6d84114b7616815b6af4635. Latest consensus state: current_term: 5 leader_uuid: "3ea1e11bc6d84114b7616815b6af4635" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:57:25.715073 22842 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [sys.catalog]: This master's current role is: LEADER
I20260925 13:57:25.728564 22843 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260925 13:57:25.728927 22785 raft_consensus.cc:493] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:25.729389 22785 raft_consensus.cc:515] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:25.731884 22785 leader_election.cc:290] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:25.733356 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ace3cd70b45451d8e9fb4b8a8d4dd0d" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:25.733889 21960 raft_consensus.cc:2468] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee02f77594864349982492862d668a36 in term 0.
I20260925 13:57:25.736852 22043 leader_election.cc:304] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36; no voters:
W20260925 13:57:25.737679 22040 leader_election.cc:343] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 8ace3cd70b45451d8e9fb4b8a8d4dd0d
I20260925 13:57:25.738540 22785 raft_consensus.cc:2804] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:25.739135 22785 raft_consensus.cc:493] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:25.739806 22785 raft_consensus.cc:3060] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:25.762998 22785 raft_consensus.cc:515] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:25.768114 22040 leader_election.cc:343] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 8ace3cd70b45451d8e9fb4b8a8d4dd0d
I20260925 13:57:25.777274 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ace3cd70b45451d8e9fb4b8a8d4dd0d" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
I20260925 13:57:25.777808 21960 raft_consensus.cc:3060] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:25.783993 21960 raft_consensus.cc:2468] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee02f77594864349982492862d668a36 in term 1.
I20260925 13:57:25.787864 22843 catalog_manager.cc:692] Loaded metadata for table default.table_2138b0cdfa7c4afda3a919dd0945a2d1 [id=012dc51aeefc492ca9aade1c75b97020]
I20260925 13:57:25.789561 22843 catalog_manager.cc:692] Loaded metadata for table default.table_f644beebc9834408bed8d57d2ac5c2ee [id=18fb001639084ecbba7437d57dfd516e]
I20260925 13:57:25.805374 22843 catalog_manager.cc:692] Loaded metadata for table default.table_e0eb6c1d49224d1cb9b4f6d06f0f7a29 [id=420cd09cc1184f6e9fb38b2cbc3f6fce]
I20260925 13:57:25.807013 22843 catalog_manager.cc:692] Loaded metadata for table default.table_b54e9b49662b4a8e892615fec1080a87 [id=722129e8c51a45f8ae31843d0d3f0a47]
I20260925 13:57:25.808708 22843 catalog_manager.cc:692] Loaded metadata for table default.table_683c0db2850941be84798e9dca3fe490 [id=7377fa3ea9664b47a930d9c5f88c60fa]
I20260925 13:57:25.810493 22843 catalog_manager.cc:692] Loaded metadata for table default.table_d6100b0f2b294ab099c179a7681cb4ed [id=74ee67384ecf4220b7d08fdf0637aefb]
I20260925 13:57:25.812064 22843 catalog_manager.cc:692] Loaded metadata for table default.table_fe2e32c0275e4744b1163ae7b22eaf51 [id=9e8b522f92404f2ba5505e6fa689ea5a]
I20260925 13:57:25.814340 22843 catalog_manager.cc:692] Loaded metadata for table default.table_d8ce7cda4ecf4422a5e74c80d7ebf4da [id=af0f962d5d5d4a17a3f576ab6a3b4df0]
I20260925 13:57:25.821565 22043 leader_election.cc:304] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36; no voters: 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:25.834766 22785 leader_election.cc:290] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 election: Requested vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:25.834851 21582 catalog_manager.cc:2223] deleted orphaned HMS catalog entry for table [id=7cba8a57ef5541a881bc69ed98657f63]
I20260925 13:57:25.837157 21582 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35994 (request call id 115) took 1845 ms. Trace:
I20260925 13:57:25.837481 21582 rpcz_store.cc:276] 0925 13:57:23.992167 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:23.992961 (+ 794us) service_pool.cc:224] Handling call
0925 13:57:24.431170 (+438209us) catalog_manager.cc:2140] Acquired catalog manager lock
0925 13:57:24.432865 (+ 1695us) catalog_manager.cc:2186] Created new table and tablet info
0925 13:57:25.394831 (+961966us) catalog_manager.cc:2212] Created new table in HMS catalog
0925 13:57:25.396500 (+ 1669us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:25.396527 (+ 27us) write_op.cc:454] Released schema lock
0925 13:57:25.397021 (+ 494us) catalog_manager.cc:2220] Rolling back HMS table creation
0925 13:57:25.837091 (+440070us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:57:25.396295 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica a9bf5a5925ac42769e474adc84a1b725 is not leader of this config: current role FOLLOWER)
0925 13:57:25.396345 (+ 50us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:25.396366 (+ 21us) write_op.cc:454] Released schema lock
0925 13:57:25.396384 (+ 18us) write_op.cc:335] FINISH: Op aborted
Metrics: {"Hive Metastore.queue_time_us":1133886,"Hive Metastore.run_cpu_time_us":968,"Hive Metastore.run_wall_time_us":263817,"spinlock_wait_cycles":12032,"child_traces":[["op",{}]]}
W20260925 13:57:25.840504 22165 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.18.182.252:39283)
I20260925 13:57:25.840600 22843 catalog_manager.cc:692] Loaded metadata for table default.table_1baae11db5a24d50bd596853233007cb [id=bb79c1d2d0094785b35250fb1b718b9a]
I20260925 13:57:25.836313 22785 raft_consensus.cc:2804] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:25.842108 22785 raft_consensus.cc:697] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P ee02f77594864349982492862d668a36 [term 1 LEADER]: Becoming Leader. State: Replica: ee02f77594864349982492862d668a36, State: Running, Role: LEADER
I20260925 13:57:25.849202 22843 catalog_manager.cc:692] Loaded metadata for table default.table_30772fdea84947818aff6f6522c73f4b [id=bcd7dc63c23c475e943a6d7395fcec10]
I20260925 13:57:25.850942 22843 catalog_manager.cc:692] Loaded metadata for table default.table_dc864d7428e5405da360d2f379eef37b [id=bf8cef2239904fd989622f9426412635]
I20260925 13:57:25.852550 22843 catalog_manager.cc:692] Loaded metadata for table default.table_05ac73cb4d774c0fbd89f99093663e76 [id=df6afcfb1e2c463493b14228e19ade26]
I20260925 13:57:25.854084 22843 catalog_manager.cc:692] Loaded metadata for table default.table_bd714c618f7a4f16a69a69d5932b2014 [id=dfd05c11872140cfa86b2628c33ef6d8]
I20260925 13:57:25.855656 22843 catalog_manager.cc:692] Loaded metadata for table default.table_fc718993756d497eb9df6af8c1651362 [id=e02d9bf4f1ff4d81bb26cddfe336f414]
I20260925 13:57:25.857214 22843 catalog_manager.cc:692] Loaded metadata for table default.table_fce30b75d46f47089d7a31e04841dcdf [id=ec768fd401eb409aaa08df2a6cdfa2f3]
W20260925 13:57:25.869827 22040 consensus_peers.cc:597] T 60ef9c152aec465eb0bb7dc286c74277 P ee02f77594864349982492862d668a36 -> Peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Couldn't send request to peer 5e2731975dbf433aade4a859df9699e2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60ef9c152aec465eb0bb7dc286c74277. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:57:25.842887 22785 consensus_queue.cc:237] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P ee02f77594864349982492862d668a36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:25.881805 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:25.907445 22165 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.18.182.252:39283)
I20260925 13:57:25.919777 22843 tablet_loader.cc:96] loaded metadata for tablet 04cd31f29357471a882157829ca700ab (table default.table_fc718993756d497eb9df6af8c1651362 [id=e02d9bf4f1ff4d81bb26cddfe336f414])
I20260925 13:57:25.920958 22843 tablet_loader.cc:96] loaded metadata for tablet 09199f2141f746a29f410907e2a5ae37 (table default.table_683c0db2850941be84798e9dca3fe490 [id=7377fa3ea9664b47a930d9c5f88c60fa])
I20260925 13:57:25.933653 22843 tablet_loader.cc:96] loaded metadata for tablet 1928234ffd3947a78e10d4c43e32aeb8 (table default.table_fce30b75d46f47089d7a31e04841dcdf [id=ec768fd401eb409aaa08df2a6cdfa2f3])
I20260925 13:57:25.935787 22843 tablet_loader.cc:96] loaded metadata for tablet 242acfcc09ca4f879e5e44a73ef16cec (table default.table_d8ce7cda4ecf4422a5e74c80d7ebf4da [id=af0f962d5d5d4a17a3f576ab6a3b4df0])
I20260925 13:57:25.937068 22843 tablet_loader.cc:96] loaded metadata for tablet 24dd39b302344f8d8f5166e9d641d13a (table default.table_e0eb6c1d49224d1cb9b4f6d06f0f7a29 [id=420cd09cc1184f6e9fb38b2cbc3f6fce])
I20260925 13:57:25.938261 22843 tablet_loader.cc:96] loaded metadata for tablet 565a7e8f453e464b8897d84e99429067 (table default.table_f644beebc9834408bed8d57d2ac5c2ee [id=18fb001639084ecbba7437d57dfd516e])
I20260925 13:57:25.939617 22843 tablet_loader.cc:96] loaded metadata for tablet 56a39663479d40be8bbd1dfd4c91040c (table default.table_05ac73cb4d774c0fbd89f99093663e76 [id=df6afcfb1e2c463493b14228e19ade26])
I20260925 13:57:25.940825 22843 tablet_loader.cc:96] loaded metadata for tablet 60ef9c152aec465eb0bb7dc286c74277 (table default.table_30772fdea84947818aff6f6522c73f4b [id=bcd7dc63c23c475e943a6d7395fcec10])
I20260925 13:57:25.944603 22843 tablet_loader.cc:96] loaded metadata for tablet 88b875abab81457481d4b98570a74ea1 (table default.table_fe2e32c0275e4744b1163ae7b22eaf51 [id=9e8b522f92404f2ba5505e6fa689ea5a])
I20260925 13:57:25.945746 22843 tablet_loader.cc:96] loaded metadata for tablet 8ace3cd70b45451d8e9fb4b8a8d4dd0d (table default.table_d6100b0f2b294ab099c179a7681cb4ed [id=74ee67384ecf4220b7d08fdf0637aefb])
I20260925 13:57:25.946913 22843 tablet_loader.cc:96] loaded metadata for tablet a06a5aac5d464d339c870df4ebd63d92 (table default.table_1baae11db5a24d50bd596853233007cb [id=bb79c1d2d0094785b35250fb1b718b9a])
I20260925 13:57:25.948192 22843 tablet_loader.cc:96] loaded metadata for tablet ba1242d8795641e7a89006d154be1216 (table default.table_bd714c618f7a4f16a69a69d5932b2014 [id=dfd05c11872140cfa86b2628c33ef6d8])
I20260925 13:57:25.949479 22843 tablet_loader.cc:96] loaded metadata for tablet c42767d8f674410188835ceefa644eb5 (table default.table_b54e9b49662b4a8e892615fec1080a87 [id=722129e8c51a45f8ae31843d0d3f0a47])
I20260925 13:57:25.952819 22843 tablet_loader.cc:96] loaded metadata for tablet d9dbc7ac9d0142c8a92262d2117ee6ec (table default.table_2138b0cdfa7c4afda3a919dd0945a2d1 [id=012dc51aeefc492ca9aade1c75b97020])
I20260925 13:57:25.954038 22843 tablet_loader.cc:96] loaded metadata for tablet ef57aa72abb2405b8187a7ed56d57f96 (table default.table_dc864d7428e5405da360d2f379eef37b [id=bf8cef2239904fd989622f9426412635])
I20260925 13:57:25.958042 22843 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260925 13:57:25.962541 21579 catalog_manager.cc:2223] deleted orphaned HMS catalog entry for table [id=e6c92379188140049e6f4d8839a700f7]
I20260925 13:57:25.963344 21579 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35994 (request call id 114) took 1980 ms. Trace:
I20260925 13:57:25.963639 21579 rpcz_store.cc:276] 0925 13:57:23.982460 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:23.982583 (+ 123us) service_pool.cc:224] Handling call
0925 13:57:24.433624 (+451041us) catalog_manager.cc:2140] Acquired catalog manager lock
0925 13:57:24.435092 (+ 1468us) catalog_manager.cc:2186] Created new table and tablet info
0925 13:57:25.458986 (+1023894us) catalog_manager.cc:2212] Created new table in HMS catalog
0925 13:57:25.460670 (+ 1684us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:25.460690 (+ 20us) write_op.cc:454] Released schema lock
0925 13:57:25.461195 (+ 505us) catalog_manager.cc:2220] Rolling back HMS table creation
0925 13:57:25.963285 (+502090us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:57:25.460482 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica a9bf5a5925ac42769e474adc84a1b725 is not leader of this config: current role FOLLOWER)
0925 13:57:25.460518 (+ 36us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:25.460539 (+ 21us) write_op.cc:454] Released schema lock
0925 13:57:25.460562 (+ 23us) write_op.cc:335] FINISH: Op aborted
Metrics: {"Hive Metastore.queue_time_us":1331515,"Hive Metastore.run_cpu_time_us":943,"Hive Metastore.run_wall_time_us":191977,"spinlock_wait_cycles":48000,"child_traces":[["op",{}]]}
I20260925 13:57:25.970009 22846 raft_consensus.cc:493] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ee02f77594864349982492862d668a36)
I20260925 13:57:25.970497 22846 raft_consensus.cc:515] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:25.973074 22846 leader_election.cc:290] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 13:57:25.979476 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
W20260925 13:57:25.985167 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
I20260925 13:57:25.985494 21732 leader_election.cc:304] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
W20260925 13:57:25.987411 22040 consensus_peers.cc:597] T a06a5aac5d464d339c870df4ebd63d92 P ee02f77594864349982492862d668a36 -> Peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Couldn't send request to peer 5e2731975dbf433aade4a859df9699e2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a06a5aac5d464d339c870df4ebd63d92. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:57:25.988181 22846 raft_consensus.cc:2749] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:57:26.002192 22785 consensus_queue.cc:1048] T 60ef9c152aec465eb0bb7dc286c74277 P ee02f77594864349982492862d668a36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 13:57:26.031975 22843 catalog_manager.cc:1308] Loaded cluster ID: d5d0bc5567a74d9cad4d92a9d1041fd9
I20260925 13:57:26.033074 22843 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260925 13:57:26.092615 22843 catalog_manager.cc:1553] Loading token signing keys...
I20260925 13:57:26.137943 21441 raft_consensus.cc:1275] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 5 FOLLOWER]: Refusing update from remote peer 3ea1e11bc6d84114b7616815b6af4635: Log matching property violated. Preceding OpId in replica: term: 3 index: 172. Preceding OpId from leader: term: 5 index: 196. (index mismatch)
I20260925 13:57:26.140005 22842 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 196, Last known committed idx: 172, Time since last communication: 0.000s
I20260925 13:57:26.140854 22857 ts_tablet_manager.cc:933] T 88b875abab81457481d4b98570a74ea1 P 5e2731975dbf433aade4a859df9699e2: Initiating tablet copy from peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:26.157594 21580 catalog_manager.cc:2223] deleted orphaned HMS catalog entry for table [id=616ee6b561b348198e3d268025078dde]
I20260925 13:57:26.158366 21580 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35994 (request call id 116) took 2098 ms. Trace:
I20260925 13:57:26.158663 21580 rpcz_store.cc:276] 0925 13:57:24.059568 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:24.060734 (+ 1166us) service_pool.cc:224] Handling call
0925 13:57:24.436705 (+375971us) catalog_manager.cc:2140] Acquired catalog manager lock
0925 13:57:24.438198 (+ 1493us) catalog_manager.cc:2186] Created new table and tablet info
0925 13:57:25.584280 (+1146082us) catalog_manager.cc:2212] Created new table in HMS catalog
0925 13:57:25.585988 (+ 1708us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:25.586007 (+ 19us) write_op.cc:454] Released schema lock
0925 13:57:25.586459 (+ 452us) catalog_manager.cc:2220] Rolling back HMS table creation
0925 13:57:26.158308 (+571849us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:57:25.585787 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica a9bf5a5925ac42769e474adc84a1b725 is not leader of this config: current role FOLLOWER)
0925 13:57:25.585837 (+ 50us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:25.585859 (+ 22us) write_op.cc:454] Released schema lock
0925 13:57:25.585881 (+ 22us) write_op.cc:335] FINISH: Op aborted
Metrics: {"Hive Metastore.queue_time_us":1395377,"Hive Metastore.run_cpu_time_us":962,"Hive Metastore.run_wall_time_us":320116,"child_traces":[["op",{}]]}
I20260925 13:57:26.175026 22857 tablet_copy_client.cc:323] T 88b875abab81457481d4b98570a74ea1 P 5e2731975dbf433aade4a859df9699e2: tablet copy: Beginning tablet copy session from remote peer at address 127.18.182.194:44291
W20260925 13:57:26.177333 21893 consensus_peers.cc:597] T ba1242d8795641e7a89006d154be1216 P 145e520e0a5a445d943a53cf8c27efee -> Peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Couldn't send request to peer 5e2731975dbf433aade4a859df9699e2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ba1242d8795641e7a89006d154be1216. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:57:26.177345 21980 tablet_copy_service.cc:140] P 145e520e0a5a445d943a53cf8c27efee: Received BeginTabletCopySession request for tablet 88b875abab81457481d4b98570a74ea1 from peer 5e2731975dbf433aade4a859df9699e2 ({username='slave'} at 127.18.182.193:58233)
I20260925 13:57:26.178375 21980 tablet_copy_service.cc:161] P 145e520e0a5a445d943a53cf8c27efee: Beginning new tablet copy session on tablet 88b875abab81457481d4b98570a74ea1 from peer 5e2731975dbf433aade4a859df9699e2 at {username='slave'} at 127.18.182.193:58233: session id = 5e2731975dbf433aade4a859df9699e2-88b875abab81457481d4b98570a74ea1
I20260925 13:57:26.179636 22843 catalog_manager.cc:6230] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635: Loaded TSK: 0
I20260925 13:57:26.187966 21980 tablet_copy_source_session.cc:215] T 88b875abab81457481d4b98570a74ea1 P 145e520e0a5a445d943a53cf8c27efee: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 13:57:26.195662 22843 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260925 13:57:26.205006 22857 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88b875abab81457481d4b98570a74ea1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:26.256683 22857 tablet_copy_client.cc:806] T 88b875abab81457481d4b98570a74ea1 P 5e2731975dbf433aade4a859df9699e2: tablet copy: Starting download of 0 data blocks...
I20260925 13:57:26.257445 22857 tablet_copy_client.cc:670] T 88b875abab81457481d4b98570a74ea1 P 5e2731975dbf433aade4a859df9699e2: tablet copy: Starting download of 1 WAL segments...
I20260925 13:57:26.263801 22857 tablet_copy_client.cc:538] T 88b875abab81457481d4b98570a74ea1 P 5e2731975dbf433aade4a859df9699e2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 13:57:26.276121 21586 catalog_manager.cc:2223] deleted orphaned HMS catalog entry for table [id=3e0dab6b1c2a48af9ada15882ebe2e9e]
I20260925 13:57:26.276981 21586 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35994 (request call id 117) took 2217 ms. Trace:
I20260925 13:57:26.277329 21586 rpcz_store.cc:276] 0925 13:57:24.059891 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:24.069245 (+ 9354us) service_pool.cc:224] Handling call
0925 13:57:24.444768 (+375523us) catalog_manager.cc:2140] Acquired catalog manager lock
0925 13:57:24.446370 (+ 1602us) catalog_manager.cc:2186] Created new table and tablet info
0925 13:57:25.666397 (+1220027us) catalog_manager.cc:2212] Created new table in HMS catalog
0925 13:57:25.668208 (+ 1811us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:25.668227 (+ 19us) write_op.cc:454] Released schema lock
0925 13:57:25.668698 (+ 471us) catalog_manager.cc:2220] Rolling back HMS table creation
0925 13:57:26.276917 (+608219us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:57:25.667995 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica a9bf5a5925ac42769e474adc84a1b725 is not leader of this config: current role FOLLOWER)
0925 13:57:25.668031 (+ 36us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:25.668054 (+ 23us) write_op.cc:454] Released schema lock
0925 13:57:25.668077 (+ 23us) write_op.cc:335] FINISH: Op aborted
Metrics: {"Hive Metastore.queue_time_us":1625196,"Hive Metastore.run_cpu_time_us":977,"Hive Metastore.run_wall_time_us":200828,"child_traces":[["op",{}]]}
I20260925 13:57:25.982828 19163 master-stress-test.cc:579] Killing test thread 1/10
I20260925 13:57:26.290271 22843 catalog_manager.cc:1576] Loading latest processed Hive Metastore notification log event ID...
I20260925 13:57:26.309052 22857 tablet_bootstrap.cc:492] T 88b875abab81457481d4b98570a74ea1 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:26.339293 22592 consensus_queue.cc:1048] T ba1242d8795641e7a89006d154be1216 P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 13:57:26.348752 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:57:26.366850 22843 catalog_manager.cc:6068] Last processed Hive Metastore notification event ID: 30
I20260925 13:57:26.372123 22847 consensus_queue.cc:1048] T a06a5aac5d464d339c870df4ebd63d92 P ee02f77594864349982492862d668a36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 13:57:26.393899 22859 sys_catalog.cc:455] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ea1e11bc6d84114b7616815b6af4635. Latest consensus state: current_term: 5 leader_uuid: "3ea1e11bc6d84114b7616815b6af4635" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:57:26.394668 22859 sys_catalog.cc:458] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:57:26.448565 22859 sys_catalog.cc:455] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "3ea1e11bc6d84114b7616815b6af4635" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:57:26.449230 22859 sys_catalog.cc:458] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:57:26.450695 22649 master_service.cc:960] ReplaceTablet: received request to replace tablet c42767d8f674410188835ceefa644eb5 from {username='slave'} at 127.0.0.1:41880
I20260925 13:57:26.454808 22648 master_service.cc:960] ReplaceTablet: received request to replace tablet 76d4d7d706824892b075ab1aa4fbaf85 from {username='slave'} at 127.0.0.1:41880
I20260925 13:57:26.483134 22857 tablet_bootstrap.cc:492] T 88b875abab81457481d4b98570a74ea1 P 5e2731975dbf433aade4a859df9699e2: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 13:57:26.484233 22857 tablet_bootstrap.cc:492] T 88b875abab81457481d4b98570a74ea1 P 5e2731975dbf433aade4a859df9699e2: Bootstrap complete.
I20260925 13:57:26.485036 22857 ts_tablet_manager.cc:1403] T 88b875abab81457481d4b98570a74ea1 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.176s user 0.016s sys 0.003s
I20260925 13:57:26.486747 22847 raft_consensus.cc:493] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:26.487290 22847 raft_consensus.cc:515] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:26.492197 22040 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
I20260925 13:57:26.493556 22766 catalog_manager.cc:2811] Deleting table default.table_fce30b75d46f47089d7a31e04841dcdf [id=ec768fd401eb409aaa08df2a6cdfa2f3] in response to Hive Metastore notification log event 31
W20260925 13:57:26.495136 22043 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
I20260925 13:57:26.495479 22847 leader_election.cc:290] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:26.495522 22043 leader_election.cc:304] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:26.496683 22847 raft_consensus.cc:2749] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:57:26.497925 22857 raft_consensus.cc:359] T 88b875abab81457481d4b98570a74ea1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:26.498591 22857 raft_consensus.cc:740] T 88b875abab81457481d4b98570a74ea1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:26.500720 21587 catalog_manager.cc:2770] deleted HMS catalog entry for table default.table_f644beebc9834408bed8d57d2ac5c2ee [id=18fb001639084ecbba7437d57dfd516e]
I20260925 13:57:26.501860 21587 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:35994 (request call id 125) took 1851 ms. Trace:
I20260925 13:57:26.502179 21587 rpcz_store.cc:276] 0925 13:57:24.650021 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:24.650149 (+ 128us) service_pool.cc:224] Handling call
0925 13:57:25.693204 (+1043055us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 13:57:26.501798 (+808594us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":581994,"Hive Metastore.run_cpu_time_us":310,"Hive Metastore.run_wall_time_us":224702}
I20260925 13:57:26.508369 22813 consensus_queue.cc:1048] T 04cd31f29357471a882157829ca700ab P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:26.499300 22857 consensus_queue.cc:260] T 88b875abab81457481d4b98570a74ea1 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:26.515676 22842 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 5 leader_uuid: "3ea1e11bc6d84114b7616815b6af4635" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:57:26.516331 22842 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [sys.catalog]: This master's current role is: LEADER
I20260925 13:57:26.517970 22857 ts_tablet_manager.cc:1434] T 88b875abab81457481d4b98570a74ea1 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.033s user 0.005s sys 0.000s
W20260925 13:57:26.523025 22773 negotiation.cc:336] Failed RPC negotiation. Trace:
0925 13:57:23.470501 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.18.182.252:39283 (local address 127.0.0.1:50808)
0925 13:57:23.516302 (+ 45801us) negotiation.cc:107] Waiting for socket to connect
0925 13:57:23.516559 (+ 257us) client_negotiation.cc:175] Beginning negotiation
0925 13:57:23.516781 (+ 222us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0925 13:57:26.521818 (+3005037us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.18.182.252:39283: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":45222,"thread_start_us":46374,"threads_started":1}
I20260925 13:57:26.523869 21843 heartbeater.cc:499] Master 127.18.182.253:46177 was elected leader, sending a full tablet report...
I20260925 13:57:26.533027 21980 tablet_copy_service.cc:342] P 145e520e0a5a445d943a53cf8c27efee: Request end of tablet copy session 5e2731975dbf433aade4a859df9699e2-88b875abab81457481d4b98570a74ea1 received from {username='slave'} at 127.18.182.193:58233
I20260925 13:57:26.533492 21980 tablet_copy_service.cc:434] P 145e520e0a5a445d943a53cf8c27efee: ending tablet copy session 5e2731975dbf433aade4a859df9699e2-88b875abab81457481d4b98570a74ea1 on tablet 88b875abab81457481d4b98570a74ea1 with peer 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:26.536446 22857 ts_tablet_manager.cc:933] T 565a7e8f453e464b8897d84e99429067 P 5e2731975dbf433aade4a859df9699e2: Initiating tablet copy from peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 13:57:26.554421 22609 leader_election.cc:336] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer a9bf5a5925ac42769e474adc84a1b725 (127.18.182.252:39283): Timed out: Client connection negotiation failed: client connection to 127.18.182.252:39283: received 0 of 4 requested bytes
I20260925 13:57:26.590667 22857 tablet_copy_client.cc:323] T 565a7e8f453e464b8897d84e99429067 P 5e2731975dbf433aade4a859df9699e2: tablet copy: Beginning tablet copy session from remote peer at address 127.18.182.194:44291
I20260925 13:57:26.592299 21980 tablet_copy_service.cc:140] P 145e520e0a5a445d943a53cf8c27efee: Received BeginTabletCopySession request for tablet 565a7e8f453e464b8897d84e99429067 from peer 5e2731975dbf433aade4a859df9699e2 ({username='slave'} at 127.18.182.193:58233)
I20260925 13:57:26.592777 21980 tablet_copy_service.cc:161] P 145e520e0a5a445d943a53cf8c27efee: Beginning new tablet copy session on tablet 565a7e8f453e464b8897d84e99429067 from peer 5e2731975dbf433aade4a859df9699e2 at {username='slave'} at 127.18.182.193:58233: session id = 5e2731975dbf433aade4a859df9699e2-565a7e8f453e464b8897d84e99429067
I20260925 13:57:26.470448 22645 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41880:
table { table_name: "default.Table_05ac73cb4d774c0fbd89f99093663e76" } modify_external_catalogs: true
I20260925 13:57:26.500201 22644 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41880:
name: "default.Table_b80b7994ff294977bd92dc8f9ddd922e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:57:26.602000 22645 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41880:
table { table_name: "default.Table_05ac73cb4d774c0fbd89f99093663e76" } modify_external_catalogs: true
I20260925 13:57:26.472188 22646 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41880:
name: "default.Table_8229f26c3e104bf1885bce04e09f8e29"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:57:26.482257 22647 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41880:
table { table_name: "default.Table_1baae11db5a24d50bd596853233007cb" } new_table_name: "default.Table_2a09482ff526433a8d028a964cfdfb0c" modify_external_catalogs: true
I20260925 13:57:26.496862 22648 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41880:
table { table_name: "default.Table_dc864d7428e5405da360d2f379eef37b" } new_table_name: "default.Table_b32d230a905d44b388dc9e436965cbc9" modify_external_catalogs: true
I20260925 13:57:26.503430 22642 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41880:
name: "default.Table_8791c6009be44efab81f5f2252a3d8ae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:57:26.517992 22641 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41880:
name: "default.Table_4586487550ec46f08eae599e1c34ef17"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:57:26.563670 22221 master_runner.cc:187] Error getting master registration for 127.18.182.252:39283: Timed out: Client connection negotiation failed: client connection to 127.18.182.252:39283: received 0 of 4 requested bytes,
I20260925 13:57:26.606500 22221 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260925 13:57:26.608376 21980 tablet_copy_source_session.cc:215] T 565a7e8f453e464b8897d84e99429067 P 145e520e0a5a445d943a53cf8c27efee: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 13:57:26.610818 22857 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 565a7e8f453e464b8897d84e99429067. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:26.662876 22649 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635: Sending DeleteTablet for 3 replicas of tablet c42767d8f674410188835ceefa644eb5
I20260925 13:57:26.666258 22649 catalog_manager.cc:6934] ReplaceTablet: tablet c42767d8f674410188835ceefa644eb5 deleted and replaced by tablet a3892ff71c0b447e86d29aed4017caf6
W20260925 13:57:26.688346 22649 catalog_manager.cc:5671] Ignoring report from unknown tablet f5bd96a2bdb842e9b3bc1f706786d6b1
W20260925 13:57:26.691905 22649 catalog_manager.cc:5671] Ignoring report from unknown tablet 60256642160a44a2bab77b97d78c27a8
W20260925 13:57:26.696389 22649 catalog_manager.cc:5671] Ignoring report from unknown tablet d267def3f0864435a0d2c40fd7052eab
I20260925 13:57:26.698387 22766 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635: Sending DeleteTablet for 3 replicas of tablet 1928234ffd3947a78e10d4c43e32aeb8
I20260925 13:57:26.709077 22857 tablet_copy_client.cc:806] T 565a7e8f453e464b8897d84e99429067 P 5e2731975dbf433aade4a859df9699e2: tablet copy: Starting download of 0 data blocks...
I20260925 13:57:26.709674 22857 tablet_copy_client.cc:670] T 565a7e8f453e464b8897d84e99429067 P 5e2731975dbf433aade4a859df9699e2: tablet copy: Starting download of 1 WAL segments...
W20260925 13:57:26.735415 21893 consensus_peers.cc:597] T 04cd31f29357471a882157829ca700ab P 145e520e0a5a445d943a53cf8c27efee -> Peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Couldn't send request to peer 5e2731975dbf433aade4a859df9699e2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 04cd31f29357471a882157829ca700ab. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:57:26.748703 22846 raft_consensus.cc:493] T 195c48349fb5410495299c9e71a72ad8 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145e520e0a5a445d943a53cf8c27efee)
I20260925 13:57:26.749217 22846 raft_consensus.cc:515] T 195c48349fb5410495299c9e71a72ad8 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:26.751062 22846 leader_election.cc:290] T 195c48349fb5410495299c9e71a72ad8 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:26.755045 22847 consensus_queue.cc:1048] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P ee02f77594864349982492862d668a36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 13:57:26.762348 21732 leader_election.cc:343] T 195c48349fb5410495299c9e71a72ad8 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 195c48349fb5410495299c9e71a72ad8
W20260925 13:57:26.763880 21732 leader_election.cc:343] T 195c48349fb5410495299c9e71a72ad8 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Not found: Tablet not found: 195c48349fb5410495299c9e71a72ad8
I20260925 13:57:26.764202 21732 leader_election.cc:304] T 195c48349fb5410495299c9e71a72ad8 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
I20260925 13:57:26.765357 22846 raft_consensus.cc:2749] T 195c48349fb5410495299c9e71a72ad8 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:57:26.768147 22651 catalog_manager.cc:5838] T a06a5aac5d464d339c870df4ebd63d92 P 145e520e0a5a445d943a53cf8c27efee reported cstate change: term changed from 0 to 1, leader changed from <none> to ee02f77594864349982492862d668a36 (127.18.182.195). New cstate: current_term: 1 leader_uuid: "ee02f77594864349982492862d668a36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } } }
I20260925 13:57:26.738274 22857 tablet_copy_client.cc:538] T 565a7e8f453e464b8897d84e99429067 P 5e2731975dbf433aade4a859df9699e2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 13:57:26.779592 21937 tablet_service.cc:1558] Processing DeleteTablet for tablet c42767d8f674410188835ceefa644eb5 with delete_type TABLET_DATA_DELETED (replaced by tablet a3892ff71c0b447e86d29aed4017caf6) from {username='slave'} at 127.0.0.1:59466
I20260925 13:57:26.780023 22874 ts_tablet_manager.cc:933] T 60ef9c152aec465eb0bb7dc286c74277 P 5e2731975dbf433aade4a859df9699e2: Initiating tablet copy from peer ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:26.780684 21940 tablet_service.cc:1558] Processing DeleteTablet for tablet 1928234ffd3947a78e10d4c43e32aeb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:57:26 UTC) from {username='slave'} at 127.0.0.1:59466
I20260925 13:57:26.780170 22650 catalog_manager.cc:5838] T ba1242d8795641e7a89006d154be1216 P ee02f77594864349982492862d668a36 reported cstate change: term changed from 0 to 1, leader changed from <none> to 145e520e0a5a445d943a53cf8c27efee (127.18.182.194). New cstate: current_term: 1 leader_uuid: "145e520e0a5a445d943a53cf8c27efee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } } }
I20260925 13:57:26.794751 21938 tablet_service.cc:1511] Processing CreateTablet for tablet 242acfcc09ca4f879e5e44a73ef16cec (DEFAULT_TABLE table=default.table_d8ce7cda4ecf4422a5e74c80d7ebf4da [id=af0f962d5d5d4a17a3f576ab6a3b4df0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:26.795766 21938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 242acfcc09ca4f879e5e44a73ef16cec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:26.800239 21940 tablet_service.cc:1511] Processing CreateTablet for tablet 09199f2141f746a29f410907e2a5ae37 (DEFAULT_TABLE table=default.table_683c0db2850941be84798e9dca3fe490 [id=7377fa3ea9664b47a930d9c5f88c60fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:26.801172 21940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09199f2141f746a29f410907e2a5ae37. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:57:26.816226 22885 log.cc:927] Time spent T 04cd31f29357471a882157829ca700ab P 145e520e0a5a445d943a53cf8c27efee: Append to log took a long time: real 0.077s user 0.001s sys 0.000s
I20260925 13:57:26.819128 22008 heartbeater.cc:499] Master 127.18.182.253:46177 was elected leader, sending a full tablet report...
I20260925 13:57:26.822847 21598 raft_consensus.cc:1275] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 5 FOLLOWER]: Refusing update from remote peer 3ea1e11bc6d84114b7616815b6af4635: Log matching property violated. Preceding OpId in replica: term: 3 index: 196. Preceding OpId from leader: term: 5 index: 196. (term mismatch)
I20260925 13:57:26.823313 21598 pending_rounds.cc:85] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Aborting all ops after (but not including) 195
I20260925 13:57:26.823665 21598 pending_rounds.cc:107] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725: Aborting uncommitted WRITE_OP operation due to leader change: 3.196
I20260925 13:57:26.823871 21597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ea1e11bc6d84114b7616815b6af4635" candidate_term: 5 candidate_status { last_received { term: 3 index: 195 } } ignore_live_leader: false dest_uuid: "a9bf5a5925ac42769e474adc84a1b725" is_pre_election: true
E20260925 13:57:26.827327 21644 catalog_manager.cc:839] Error processing pending assignments: Aborted: error persisting updated tablet metadata: Op aborted by new leader
I20260925 13:57:26.828620 21583 catalog_manager.cc:5838] T 60ef9c152aec465eb0bb7dc286c74277 P ee02f77594864349982492862d668a36 reported cstate change: term changed from 0 to 1, leader changed from <none> to ee02f77594864349982492862d668a36 (127.18.182.195). New cstate: current_term: 1 leader_uuid: "ee02f77594864349982492862d668a36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: HEALTHY } } }
W20260925 13:57:26.832691 21644 catalog_manager.cc:6644] Illegal state: an error occurred while writing to the sys-catalog: replica a9bf5a5925ac42769e474adc84a1b725 is not leader of this config: current role FOLLOWER
E20260925 13:57:26.833009 21644 catalog_manager.cc:859] Error processing tables/tablets deletions: Illegal state: an error occurred while writing to the sys-catalog: replica a9bf5a5925ac42769e474adc84a1b725 is not leader of this config: current role FOLLOWER
E20260925 13:57:26.833483 21583 catalog_manager.cc:6001] Error updating tablets from ee02f77594864349982492862d668a36: Illegal state: replica a9bf5a5925ac42769e474adc84a1b725 is not leader of this config: current role FOLLOWER. Tablet report was: is_incremental: true updated_tablets { tablet_id: "60ef9c152aec465eb0bb7dc286c74277" state: RUNNING consensus_state { current_term: 1 leader_uuid: "ee02f77594864349982492862d668a36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: HEALTHY } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY stats { } } sequence_number: 124
W20260925 13:57:26.834978 21583 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.195:53791 (request call id 125) took 1900 ms (client timeout 1000 ms). Trace:
W20260925 13:57:26.835381 21583 rpcz_store.cc:269] 0925 13:57:24.934702 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:24.942217 (+ 7515us) service_pool.cc:224] Handling call
0925 13:57:26.833285 (+1891068us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:26.833306 (+ 21us) write_op.cc:454] Released schema lock
0925 13:57:26.834908 (+ 1602us) inbound_call.cc:177] Queueing failure response
Related trace 'op':
0925 13:57:26.832763 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica a9bf5a5925ac42769e474adc84a1b725 is not leader of this config: current role FOLLOWER)
0925 13:57:26.833038 (+ 275us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:26.833059 (+ 21us) write_op.cc:454] Released schema lock
0925 13:57:26.833084 (+ 25us) write_op.cc:335] FINISH: Op aborted
Metrics: {"reported_tablets":1,"child_traces":[["op",{}]]}
I20260925 13:57:26.840067 22887 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 196, Last known committed idx: 195, Time since last communication: 0.001s
W20260925 13:57:26.848649 22651 catalog_manager.cc:5671] Ignoring report from unknown tablet 60256642160a44a2bab77b97d78c27a8
W20260925 13:57:26.849021 22651 catalog_manager.cc:5671] Ignoring report from unknown tablet d267def3f0864435a0d2c40fd7052eab
I20260925 13:57:26.854454 22889 tablet_replica.cc:333] T c42767d8f674410188835ceefa644eb5 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:26.855417 22889 raft_consensus.cc:2243] T c42767d8f674410188835ceefa644eb5 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:26.857564 22083 tablet_service.cc:1558] Processing DeleteTablet for tablet c42767d8f674410188835ceefa644eb5 with delete_type TABLET_DATA_DELETED (replaced by tablet a3892ff71c0b447e86d29aed4017caf6) from {username='slave'} at 127.0.0.1:59366
I20260925 13:57:26.860172 22084 tablet_service.cc:1558] Processing DeleteTablet for tablet 1928234ffd3947a78e10d4c43e32aeb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:57:26 UTC) from {username='slave'} at 127.0.0.1:59366
I20260925 13:57:26.863466 22894 tablet_replica.cc:333] T c42767d8f674410188835ceefa644eb5 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:26.864374 22894 raft_consensus.cc:2243] T c42767d8f674410188835ceefa644eb5 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:26.864959 22894 raft_consensus.cc:2272] T c42767d8f674410188835ceefa644eb5 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:26.867187 22083 tablet_service.cc:1511] Processing CreateTablet for tablet 09199f2141f746a29f410907e2a5ae37 (DEFAULT_TABLE table=default.table_683c0db2850941be84798e9dca3fe490 [id=7377fa3ea9664b47a930d9c5f88c60fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:26.868428 22083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09199f2141f746a29f410907e2a5ae37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:26.872720 22084 tablet_service.cc:1511] Processing CreateTablet for tablet 242acfcc09ca4f879e5e44a73ef16cec (DEFAULT_TABLE table=default.table_d8ce7cda4ecf4422a5e74c80d7ebf4da [id=af0f962d5d5d4a17a3f576ab6a3b4df0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:26.873200 22889 raft_consensus.cc:2272] T c42767d8f674410188835ceefa644eb5 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:26.873749 22084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 242acfcc09ca4f879e5e44a73ef16cec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:26.875172 22889 ts_tablet_manager.cc:1916] T c42767d8f674410188835ceefa644eb5 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:26.890272 22896 tablet_bootstrap.cc:492] T 242acfcc09ca4f879e5e44a73ef16cec P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:26.894097 22894 ts_tablet_manager.cc:1916] T c42767d8f674410188835ceefa644eb5 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:26.896344 22879 raft_consensus.cc:493] T 76d4d7d706824892b075ab1aa4fbaf85 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145e520e0a5a445d943a53cf8c27efee)
I20260925 13:57:26.896872 22879 raft_consensus.cc:515] T 76d4d7d706824892b075ab1aa4fbaf85 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:26.898744 22879 leader_election.cc:290] T 76d4d7d706824892b075ab1aa4fbaf85 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 13:57:26.901374 21732 leader_election.cc:343] T 76d4d7d706824892b075ab1aa4fbaf85 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Not found: Tablet not found: 76d4d7d706824892b075ab1aa4fbaf85
W20260925 13:57:26.902889 21732 leader_election.cc:343] T 76d4d7d706824892b075ab1aa4fbaf85 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 76d4d7d706824892b075ab1aa4fbaf85
I20260925 13:57:26.903214 21732 leader_election.cc:304] T 76d4d7d706824892b075ab1aa4fbaf85 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
I20260925 13:57:26.915503 22879 raft_consensus.cc:2749] T 76d4d7d706824892b075ab1aa4fbaf85 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:57:26.924532 22874 tablet_copy_client.cc:323] T 60ef9c152aec465eb0bb7dc286c74277 P 5e2731975dbf433aade4a859df9699e2: tablet copy: Beginning tablet copy session from remote peer at address 127.18.182.195:40873
W20260925 13:57:26.925850 22040 consensus_peers.cc:597] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P ee02f77594864349982492862d668a36 -> Peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Couldn't send request to peer 5e2731975dbf433aade4a859df9699e2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ace3cd70b45451d8e9fb4b8a8d4dd0d. This is attempt 1: this message will repeat every 5th retry.
W20260925 13:57:26.929677 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:57:26.943264 22127 tablet_copy_service.cc:140] P ee02f77594864349982492862d668a36: Received BeginTabletCopySession request for tablet 60ef9c152aec465eb0bb7dc286c74277 from peer 5e2731975dbf433aade4a859df9699e2 ({username='slave'} at 127.18.182.193:43409)
I20260925 13:57:26.943871 22127 tablet_copy_service.cc:161] P ee02f77594864349982492862d668a36: Beginning new tablet copy session on tablet 60ef9c152aec465eb0bb7dc286c74277 from peer 5e2731975dbf433aade4a859df9699e2 at {username='slave'} at 127.18.182.193:43409: session id = 5e2731975dbf433aade4a859df9699e2-60ef9c152aec465eb0bb7dc286c74277
I20260925 13:57:26.938546 22651 catalog_manager.cc:5838] T 88b875abab81457481d4b98570a74ea1 P 145e520e0a5a445d943a53cf8c27efee reported cstate change: term changed from 0 to 1, leader changed from <none> to 145e520e0a5a445d943a53cf8c27efee (127.18.182.194). New cstate: current_term: 1 leader_uuid: "145e520e0a5a445d943a53cf8c27efee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: HEALTHY } } }
I20260925 13:57:26.951143 22651 catalog_manager.cc:5838] T 565a7e8f453e464b8897d84e99429067 P 145e520e0a5a445d943a53cf8c27efee reported cstate change: term changed from 0 to 1, leader changed from <none> to 145e520e0a5a445d943a53cf8c27efee (127.18.182.194). New cstate: current_term: 1 leader_uuid: "145e520e0a5a445d943a53cf8c27efee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: HEALTHY } } }
I20260925 13:57:26.952955 22127 tablet_copy_source_session.cc:215] T 60ef9c152aec465eb0bb7dc286c74277 P ee02f77594864349982492862d668a36: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 13:57:26.956349 22868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 5 leader_uuid: "3ea1e11bc6d84114b7616815b6af4635" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:57:26.956900 22868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [sys.catalog]: This master's current role is: LEADER
I20260925 13:57:26.957839 22892 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ea1e11bc6d84114b7616815b6af4635. Latest consensus state: current_term: 5 leader_uuid: "3ea1e11bc6d84114b7616815b6af4635" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:57:26.958617 22892 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:57:26.962448 22897 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "3ea1e11bc6d84114b7616815b6af4635" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:57:26.963187 22897 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:57:26.919267 22896 tablet_bootstrap.cc:654] T 242acfcc09ca4f879e5e44a73ef16cec P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:26.967396 22901 tablet_bootstrap.cc:492] T 09199f2141f746a29f410907e2a5ae37 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:26.957986 22874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60ef9c152aec465eb0bb7dc286c74277. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:26.976051 22901 tablet_bootstrap.cc:654] T 09199f2141f746a29f410907e2a5ae37 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:26.975610 22651 catalog_manager.cc:5838] T 60ef9c152aec465eb0bb7dc286c74277 P 145e520e0a5a445d943a53cf8c27efee reported cstate change: term changed from 0 to 1, leader changed from <none> to ee02f77594864349982492862d668a36 (127.18.182.195). New cstate: current_term: 1 leader_uuid: "ee02f77594864349982492862d668a36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } } }
I20260925 13:57:26.977032 22651 catalog_manager.cc:5838] T 04cd31f29357471a882157829ca700ab P 145e520e0a5a445d943a53cf8c27efee reported cstate change: term changed from 0 to 1, leader changed from <none> to 145e520e0a5a445d943a53cf8c27efee (127.18.182.194). New cstate: current_term: 1 leader_uuid: "145e520e0a5a445d943a53cf8c27efee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: HEALTHY } } }
I20260925 13:57:26.985805 22857 tablet_bootstrap.cc:492] T 565a7e8f453e464b8897d84e99429067 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:26.987872 22901 tablet_bootstrap.cc:492] T 09199f2141f746a29f410907e2a5ae37 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:26.988281 22901 ts_tablet_manager.cc:1403] T 09199f2141f746a29f410907e2a5ae37 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.021s user 0.007s sys 0.010s
I20260925 13:57:26.991822 22901 raft_consensus.cc:359] T 09199f2141f746a29f410907e2a5ae37 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:26.992708 22901 raft_consensus.cc:385] T 09199f2141f746a29f410907e2a5ae37 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:26.993014 22901 raft_consensus.cc:740] T 09199f2141f746a29f410907e2a5ae37 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:26.994110 22901 consensus_queue.cc:260] T 09199f2141f746a29f410907e2a5ae37 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:26.978533 22651 catalog_manager.cc:5838] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 145e520e0a5a445d943a53cf8c27efee reported cstate change: term changed from 0 to 1, leader changed from <none> to ee02f77594864349982492862d668a36 (127.18.182.195). New cstate: current_term: 1 leader_uuid: "ee02f77594864349982492862d668a36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } } }
I20260925 13:57:26.996546 22651 catalog_manager.cc:5938] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291) does not have the latest schema for tablet 8ace3cd70b45451d8e9fb4b8a8d4dd0d (table default.table_d6100b0f2b294ab099c179a7681cb4ed [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 2 got 0
I20260925 13:57:27.013891 22153 heartbeater.cc:499] Master 127.18.182.253:46177 was elected leader, sending a full tablet report...
I20260925 13:57:27.022293 22896 tablet_bootstrap.cc:492] T 242acfcc09ca4f879e5e44a73ef16cec P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:27.022720 22896 ts_tablet_manager.cc:1403] T 242acfcc09ca4f879e5e44a73ef16cec P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.133s user 0.014s sys 0.003s
I20260925 13:57:27.025336 22896 raft_consensus.cc:359] T 242acfcc09ca4f879e5e44a73ef16cec P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:27.025974 22896 raft_consensus.cc:385] T 242acfcc09ca4f879e5e44a73ef16cec P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:27.026242 22896 raft_consensus.cc:740] T 242acfcc09ca4f879e5e44a73ef16cec P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:27.019486 22889 ts_tablet_manager.cc:1929] T c42767d8f674410188835ceefa644eb5 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:27.030072 22894 ts_tablet_manager.cc:1929] T c42767d8f674410188835ceefa644eb5 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:27.030478 22894 log.cc:1199] T c42767d8f674410188835ceefa644eb5 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/c42767d8f674410188835ceefa644eb5
I20260925 13:57:27.031463 22894 ts_tablet_manager.cc:1950] T c42767d8f674410188835ceefa644eb5 P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:27.032094 22889 log.cc:1199] T c42767d8f674410188835ceefa644eb5 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/c42767d8f674410188835ceefa644eb5
I20260925 13:57:27.032876 22889 ts_tablet_manager.cc:1950] T c42767d8f674410188835ceefa644eb5 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:27.026887 22896 consensus_queue.cc:260] T 242acfcc09ca4f879e5e44a73ef16cec P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:27.059095 22650 catalog_manager.cc:5671] Ignoring report from unknown tablet 71ff569d68154b61bcc14937b2e37fe5
W20260925 13:57:27.059409 22650 catalog_manager.cc:5671] Ignoring report from unknown tablet ccefd36f4e2a48f9b3d66cb147a022b5
W20260925 13:57:27.059718 22650 catalog_manager.cc:5671] Ignoring report from unknown tablet d267def3f0864435a0d2c40fd7052eab
I20260925 13:57:27.064368 21776 tablet_service.cc:1558] Processing DeleteTablet for tablet c42767d8f674410188835ceefa644eb5 with delete_type TABLET_DATA_DELETED (replaced by tablet a3892ff71c0b447e86d29aed4017caf6) from {username='slave'} at 127.0.0.1:40862
I20260925 13:57:27.065512 21775 tablet_service.cc:1511] Processing CreateTablet for tablet 09199f2141f746a29f410907e2a5ae37 (DEFAULT_TABLE table=default.table_683c0db2850941be84798e9dca3fe490 [id=7377fa3ea9664b47a930d9c5f88c60fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:27.066936 21775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09199f2141f746a29f410907e2a5ae37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:27.067457 22766 catalog_manager.cc:2811] Deleting table default.table_8229f26c3e104bf1885bce04e09f8e29 [id=7cba8a57ef5541a881bc69ed98657f63] in response to Hive Metastore notification log event 37
W20260925 13:57:27.068280 22766 hms_notification_log_listener.cc:330] Failed to handle Hive Metastore notification: 37 DROP_TABLE default.table_8229f26c3e104bf1885bce04e09f8e29: Not found: the table does not exist: table_id: "7cba8a57ef5541a881bc69ed98657f63" table_name: "default.table_8229f26c3e104bf1885bce04e09f8e29"
I20260925 13:57:27.071586 21777 tablet_service.cc:1558] Processing DeleteTablet for tablet 1928234ffd3947a78e10d4c43e32aeb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:57:26 UTC) from {username='slave'} at 127.0.0.1:40862
I20260925 13:57:27.074745 22609 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet c42767d8f674410188835ceefa644eb5 (table default.table_b54e9b49662b4a8e892615fec1080a87 [id=722129e8c51a45f8ae31843d0d3f0a47]) successfully deleted
I20260925 13:57:27.076500 22889 tablet_replica.cc:333] T 1928234ffd3947a78e10d4c43e32aeb8 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:27.077690 21777 tablet_service.cc:1511] Processing CreateTablet for tablet 242acfcc09ca4f879e5e44a73ef16cec (DEFAULT_TABLE table=default.table_d8ce7cda4ecf4422a5e74c80d7ebf4da [id=af0f962d5d5d4a17a3f576ab6a3b4df0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:27.078572 21777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 242acfcc09ca4f879e5e44a73ef16cec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:27.079725 22874 tablet_copy_client.cc:806] T 60ef9c152aec465eb0bb7dc286c74277 P 5e2731975dbf433aade4a859df9699e2: tablet copy: Starting download of 0 data blocks...
I20260925 13:57:27.080227 22874 tablet_copy_client.cc:670] T 60ef9c152aec465eb0bb7dc286c74277 P 5e2731975dbf433aade4a859df9699e2: tablet copy: Starting download of 1 WAL segments...
I20260925 13:57:27.084949 21773 tablet_service.cc:1558] Processing DeleteTablet for tablet 76d4d7d706824892b075ab1aa4fbaf85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:57:24 UTC) from {username='slave'} at 127.0.0.1:40862
I20260925 13:57:27.085958 21773 tablet_service.cc:1558] Processing DeleteTablet for tablet 794784c42d27441f829f954b6929a097 with delete_type TABLET_DATA_DELETED (replaced by tablet 09199f2141f746a29f410907e2a5ae37) from {username='slave'} at 127.0.0.1:40862
I20260925 13:57:27.086400 21774 tablet_service.cc:1558] Processing DeleteTablet for tablet 761ede873f5648ccb5216a4c0a8afa2e with delete_type TABLET_DATA_DELETED (replaced by tablet 8ace3cd70b45451d8e9fb4b8a8d4dd0d) from {username='slave'} at 127.0.0.1:40862
I20260925 13:57:27.086899 21773 tablet_service.cc:1558] Processing DeleteTablet for tablet 195c48349fb5410495299c9e71a72ad8 with delete_type TABLET_DATA_DELETED (replaced by tablet ba1242d8795641e7a89006d154be1216) from {username='slave'} at 127.0.0.1:40862
I20260925 13:57:27.088045 22891 raft_consensus.cc:493] T 09199f2141f746a29f410907e2a5ae37 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:27.088498 22891 raft_consensus.cc:515] T 09199f2141f746a29f410907e2a5ae37 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:27.090817 22891 leader_election.cc:290] T 09199f2141f746a29f410907e2a5ae37 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:27.096653 22084 tablet_service.cc:1511] Processing CreateTablet for tablet a3892ff71c0b447e86d29aed4017caf6 (DEFAULT_TABLE table=default.table_b54e9b49662b4a8e892615fec1080a87 [id=722129e8c51a45f8ae31843d0d3f0a47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:27.101670 22889 raft_consensus.cc:2243] T 1928234ffd3947a78e10d4c43e32aeb8 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:27.093184 21958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09199f2141f746a29f410907e2a5ae37" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:27.095419 21938 tablet_service.cc:1511] Processing CreateTablet for tablet a3892ff71c0b447e86d29aed4017caf6 (DEFAULT_TABLE table=default.table_b54e9b49662b4a8e892615fec1080a87 [id=722129e8c51a45f8ae31843d0d3f0a47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:27.110678 21938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3892ff71c0b447e86d29aed4017caf6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:57:27.111511 22651 catalog_manager.cc:5671] Ignoring report from unknown tablet 60256642160a44a2bab77b97d78c27a8
W20260925 13:57:27.115793 22043 leader_election.cc:343] T 09199f2141f746a29f410907e2a5ae37 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:57:27.111307 22901 ts_tablet_manager.cc:1434] T 09199f2141f746a29f410907e2a5ae37 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.123s user 0.006s sys 0.000s
I20260925 13:57:27.121447 22651 catalog_manager.cc:5938] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291) does not have the latest schema for tablet 8ace3cd70b45451d8e9fb4b8a8d4dd0d (table default.table_d6100b0f2b294ab099c179a7681cb4ed [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 2 got 0
I20260925 13:57:27.121721 21773 tablet_service.cc:1558] Processing DeleteTablet for tablet fb781a032ba649de8a214925fdb374b5 with delete_type TABLET_DATA_DELETED (replaced by tablet 242acfcc09ca4f879e5e44a73ef16cec) from {username='slave'} at 127.0.0.1:40862
I20260925 13:57:27.081697 22896 ts_tablet_manager.cc:1434] T 242acfcc09ca4f879e5e44a73ef16cec P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.059s user 0.004s sys 0.003s
I20260925 13:57:27.123065 22766 catalog_manager.cc:2811] Deleting table default.table_b80b7994ff294977bd92dc8f9ddd922e [id=e6c92379188140049e6f4d8839a700f7] in response to Hive Metastore notification log event 38
W20260925 13:57:27.128147 22766 hms_notification_log_listener.cc:330] Failed to handle Hive Metastore notification: 38 DROP_TABLE default.table_b80b7994ff294977bd92dc8f9ddd922e: Not found: the table does not exist: table_id: "e6c92379188140049e6f4d8839a700f7" table_name: "default.table_b80b7994ff294977bd92dc8f9ddd922e"
I20260925 13:57:27.123885 22874 tablet_copy_client.cc:538] T 60ef9c152aec465eb0bb7dc286c74277 P 5e2731975dbf433aade4a859df9699e2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 13:57:27.134997 22896 tablet_bootstrap.cc:492] T 09199f2141f746a29f410907e2a5ae37 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:27.133628 22889 raft_consensus.cc:2272] T 1928234ffd3947a78e10d4c43e32aeb8 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:27.144948 22889 ts_tablet_manager.cc:1916] T 1928234ffd3947a78e10d4c43e32aeb8 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:27.136036 22907 tablet_replica.cc:333] T c42767d8f674410188835ceefa644eb5 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:27.152534 22907 raft_consensus.cc:2243] T c42767d8f674410188835ceefa644eb5 P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:27.153797 22650 catalog_manager.cc:5938] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873) does not have the latest schema for tablet ba1242d8795641e7a89006d154be1216 (table default.table_97e22ed3359342caaff80295b5d37db2 [id=dfd05c11872140cfa86b2628c33ef6d8]). Expected version 1 got 0
I20260925 13:57:27.153923 22907 raft_consensus.cc:2272] T c42767d8f674410188835ceefa644eb5 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:27.160825 22907 ts_tablet_manager.cc:1916] T c42767d8f674410188835ceefa644eb5 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:27.191946 22084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3892ff71c0b447e86d29aed4017caf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:27.204694 22846 raft_consensus.cc:493] T 761ede873f5648ccb5216a4c0a8afa2e P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145e520e0a5a445d943a53cf8c27efee)
I20260925 13:57:27.205459 22846 raft_consensus.cc:515] T 761ede873f5648ccb5216a4c0a8afa2e P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:27.207739 22846 leader_election.cc:290] T 761ede873f5648ccb5216a4c0a8afa2e P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 13:57:27.211781 21732 leader_election.cc:343] T 761ede873f5648ccb5216a4c0a8afa2e P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Not found: Tablet not found: 761ede873f5648ccb5216a4c0a8afa2e
I20260925 13:57:27.214035 22901 tablet_bootstrap.cc:492] T 242acfcc09ca4f879e5e44a73ef16cec P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:27.226073 22901 tablet_bootstrap.cc:654] T 242acfcc09ca4f879e5e44a73ef16cec P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:27.235276 22846 raft_consensus.cc:493] T 794784c42d27441f829f954b6929a097 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 145e520e0a5a445d943a53cf8c27efee)
I20260925 13:57:27.236012 22846 raft_consensus.cc:515] T 794784c42d27441f829f954b6929a097 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:27.238126 22846 leader_election.cc:290] T 794784c42d27441f829f954b6929a097 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:27.239223 22908 ts_tablet_manager.cc:933] T a06a5aac5d464d339c870df4ebd63d92 P 5e2731975dbf433aade4a859df9699e2: Initiating tablet copy from peer ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:27.246120 22896 tablet_bootstrap.cc:654] T 09199f2141f746a29f410907e2a5ae37 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:27.251475 21776 tablet_service.cc:1511] Processing CreateTablet for tablet a3892ff71c0b447e86d29aed4017caf6 (DEFAULT_TABLE table=default.table_b54e9b49662b4a8e892615fec1080a87 [id=722129e8c51a45f8ae31843d0d3f0a47]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 13:57:27.253238 22040 leader_election.cc:343] T 09199f2141f746a29f410907e2a5ae37 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 09199f2141f746a29f410907e2a5ae37
I20260925 13:57:27.254016 22040 leader_election.cc:304] T 09199f2141f746a29f410907e2a5ae37 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:27.255096 22891 raft_consensus.cc:2749] T 09199f2141f746a29f410907e2a5ae37 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:57:27.261370 22894 tablet_replica.cc:333] T 1928234ffd3947a78e10d4c43e32aeb8 P ee02f77594864349982492862d668a36: stopping tablet replica
W20260925 13:57:27.263677 21732 leader_election.cc:343] T 794784c42d27441f829f954b6929a097 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Not found: Tablet not found: 794784c42d27441f829f954b6929a097
I20260925 13:57:27.274094 22908 tablet_copy_client.cc:323] T a06a5aac5d464d339c870df4ebd63d92 P 5e2731975dbf433aade4a859df9699e2: tablet copy: Beginning tablet copy session from remote peer at address 127.18.182.195:40873
I20260925 13:57:27.278393 22127 tablet_copy_service.cc:140] P ee02f77594864349982492862d668a36: Received BeginTabletCopySession request for tablet a06a5aac5d464d339c870df4ebd63d92 from peer 5e2731975dbf433aade4a859df9699e2 ({username='slave'} at 127.18.182.193:43409)
I20260925 13:57:27.278903 22127 tablet_copy_service.cc:161] P ee02f77594864349982492862d668a36: Beginning new tablet copy session on tablet a06a5aac5d464d339c870df4ebd63d92 from peer 5e2731975dbf433aade4a859df9699e2 at {username='slave'} at 127.18.182.193:43409: session id = 5e2731975dbf433aade4a859df9699e2-a06a5aac5d464d339c870df4ebd63d92
I20260925 13:57:27.278649 22912 tablet.cc:1751] T ba1242d8795641e7a89006d154be1216 P 145e520e0a5a445d943a53cf8c27efee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260925 13:57:27.281867 21896 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.195:40873, user_credentials={real_user=slave}} blocked reactor thread for 95136.4us
I20260925 13:57:27.284366 22874 tablet_bootstrap.cc:492] T 60ef9c152aec465eb0bb7dc286c74277 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:27.284864 21776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3892ff71c0b447e86d29aed4017caf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:27.287238 22127 tablet_copy_source_session.cc:215] T a06a5aac5d464d339c870df4ebd63d92 P ee02f77594864349982492862d668a36: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 13:57:27.293771 22906 tablet.cc:1751] T ba1242d8795641e7a89006d154be1216 P ee02f77594864349982492862d668a36: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:57:27.296697 22901 tablet_bootstrap.cc:492] T 242acfcc09ca4f879e5e44a73ef16cec P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:27.299129 22905 tablet.cc:1751] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 145e520e0a5a445d943a53cf8c27efee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 13:57:27.301687 22908 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a06a5aac5d464d339c870df4ebd63d92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:27.303654 22901 ts_tablet_manager.cc:1403] T 242acfcc09ca4f879e5e44a73ef16cec P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.090s user 0.009s sys 0.004s
I20260925 13:57:27.306298 22901 raft_consensus.cc:359] T 242acfcc09ca4f879e5e44a73ef16cec P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:27.311478 22901 raft_consensus.cc:385] T 242acfcc09ca4f879e5e44a73ef16cec P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:27.311825 22901 raft_consensus.cc:740] T 242acfcc09ca4f879e5e44a73ef16cec P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:27.301445 22911 tablet.cc:1751] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P ee02f77594864349982492862d668a36: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 13:57:27.313450 22901 consensus_queue.cc:260] T 242acfcc09ca4f879e5e44a73ef16cec P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:27.320559 22896 tablet_bootstrap.cc:492] T 09199f2141f746a29f410907e2a5ae37 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:27.320960 22901 ts_tablet_manager.cc:1434] T 242acfcc09ca4f879e5e44a73ef16cec P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.017s user 0.003s sys 0.002s
I20260925 13:57:27.330397 22896 ts_tablet_manager.cc:1403] T 09199f2141f746a29f410907e2a5ae37 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.196s user 0.008s sys 0.005s
W20260925 13:57:27.340638 21732 leader_election.cc:343] T 794784c42d27441f829f954b6929a097 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 794784c42d27441f829f954b6929a097
I20260925 13:57:27.340816 21797 raft_consensus.cc:1217] T 88b875abab81457481d4b98570a74ea1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 0.0->[1.1-1.1] Dedup: 1.1->[]
I20260925 13:57:27.341895 22896 raft_consensus.cc:359] T 09199f2141f746a29f410907e2a5ae37 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:27.342675 22896 raft_consensus.cc:385] T 09199f2141f746a29f410907e2a5ae37 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:27.343011 22896 raft_consensus.cc:740] T 09199f2141f746a29f410907e2a5ae37 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:27.343843 22896 consensus_queue.cc:260] T 09199f2141f746a29f410907e2a5ae37 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:27.345959 22896 ts_tablet_manager.cc:1434] T 09199f2141f746a29f410907e2a5ae37 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.015s user 0.005s sys 0.000s
I20260925 13:57:27.347420 22896 tablet_bootstrap.cc:492] T a3892ff71c0b447e86d29aed4017caf6 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:27.356115 22894 raft_consensus.cc:2243] T 1928234ffd3947a78e10d4c43e32aeb8 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:27.357208 22896 tablet_bootstrap.cc:654] T a3892ff71c0b447e86d29aed4017caf6 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:27.364511 22896 tablet_bootstrap.cc:492] T a3892ff71c0b447e86d29aed4017caf6 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:27.364980 22896 ts_tablet_manager.cc:1403] T a3892ff71c0b447e86d29aed4017caf6 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.000s
I20260925 13:57:27.367552 22896 raft_consensus.cc:359] T a3892ff71c0b447e86d29aed4017caf6 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:27.368242 22896 raft_consensus.cc:385] T a3892ff71c0b447e86d29aed4017caf6 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:27.368562 22896 raft_consensus.cc:740] T a3892ff71c0b447e86d29aed4017caf6 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:27.369282 22896 consensus_queue.cc:260] T a3892ff71c0b447e86d29aed4017caf6 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:27.373003 22896 ts_tablet_manager.cc:1434] T a3892ff71c0b447e86d29aed4017caf6 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260925 13:57:27.341264 21732 leader_election.cc:304] T 794784c42d27441f829f954b6929a097 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
I20260925 13:57:27.376001 22894 raft_consensus.cc:2272] T 1928234ffd3947a78e10d4c43e32aeb8 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:27.198539 22766 catalog_manager.cc:2811] Deleting table default.table_8791c6009be44efab81f5f2252a3d8ae [id=616ee6b561b348198e3d268025078dde] in response to Hive Metastore notification log event 39
I20260925 13:57:27.260070 22609 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet c42767d8f674410188835ceefa644eb5 (table default.table_b54e9b49662b4a8e892615fec1080a87 [id=722129e8c51a45f8ae31843d0d3f0a47]) successfully deleted
W20260925 13:57:27.377246 22766 hms_notification_log_listener.cc:330] Failed to handle Hive Metastore notification: 39 DROP_TABLE default.table_8791c6009be44efab81f5f2252a3d8ae: Not found: the table does not exist: table_id: "616ee6b561b348198e3d268025078dde" table_name: "default.table_8791c6009be44efab81f5f2252a3d8ae"
W20260925 13:57:27.376973 22609 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.18.182.195:40873, user_credentials={real_user=slave}} blocked reactor thread for 116849us
I20260925 13:57:27.379136 22650 catalog_manager.cc:5938] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873) does not have the latest schema for tablet 8ace3cd70b45451d8e9fb4b8a8d4dd0d (table default.table_d6100b0f2b294ab099c179a7681cb4ed [id=74ee67384ecf4220b7d08fdf0637aefb]). Expected version 2 got 0
W20260925 13:57:27.381495 21732 leader_election.cc:343] T 761ede873f5648ccb5216a4c0a8afa2e P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 761ede873f5648ccb5216a4c0a8afa2e
I20260925 13:57:27.388262 22889 ts_tablet_manager.cc:1929] T 1928234ffd3947a78e10d4c43e32aeb8 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:27.390369 22901 tablet_bootstrap.cc:492] T a3892ff71c0b447e86d29aed4017caf6 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:27.398057 21732 leader_election.cc:304] T 761ede873f5648ccb5216a4c0a8afa2e P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
I20260925 13:57:27.399799 22650 catalog_manager.cc:5938] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873) does not have the latest schema for tablet 09199f2141f746a29f410907e2a5ae37 (table default.table_683c0db2850941be84798e9dca3fe490 [id=7377fa3ea9664b47a930d9c5f88c60fa]). Expected version 1 got 0
W20260925 13:57:27.400156 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:57:27.400156 22901 tablet_bootstrap.cc:654] T a3892ff71c0b447e86d29aed4017caf6 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:27.404726 22889 log.cc:1199] T 1928234ffd3947a78e10d4c43e32aeb8 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/1928234ffd3947a78e10d4c43e32aeb8
I20260925 13:57:27.409617 22889 ts_tablet_manager.cc:1950] T 1928234ffd3947a78e10d4c43e32aeb8 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:27.382166 22766 catalog_manager.cc:2811] Deleting table default.table_4586487550ec46f08eae599e1c34ef17 [id=3e0dab6b1c2a48af9ada15882ebe2e9e] in response to Hive Metastore notification log event 40
W20260925 13:57:27.416785 22766 hms_notification_log_listener.cc:330] Failed to handle Hive Metastore notification: 40 DROP_TABLE default.table_4586487550ec46f08eae599e1c34ef17: Not found: the table does not exist: table_id: "3e0dab6b1c2a48af9ada15882ebe2e9e" table_name: "default.table_4586487550ec46f08eae599e1c34ef17"
I20260925 13:57:27.420078 22894 ts_tablet_manager.cc:1916] T 1928234ffd3947a78e10d4c43e32aeb8 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 13:57:27.426245 21732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.182.194:44291, user_credentials={real_user=slave}} blocked reactor thread for 44793.3us
I20260925 13:57:27.442976 22609 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet 1928234ffd3947a78e10d4c43e32aeb8 (table default.table_fce30b75d46f47089d7a31e04841dcdf [id=ec768fd401eb409aaa08df2a6cdfa2f3]) successfully deleted
I20260925 13:57:27.441759 22879 raft_consensus.cc:2749] T 761ede873f5648ccb5216a4c0a8afa2e P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:57:27.441883 22846 raft_consensus.cc:2749] T 794784c42d27441f829f954b6929a097 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:57:27.457154 22917 ts_tablet_manager.cc:933] T ba1242d8795641e7a89006d154be1216 P 5e2731975dbf433aade4a859df9699e2: Initiating tablet copy from peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:27.483386 22920 tablet_bootstrap.cc:492] T a3892ff71c0b447e86d29aed4017caf6 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:27.495759 22651 catalog_manager.cc:5938] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291) does not have the latest schema for tablet 09199f2141f746a29f410907e2a5ae37 (table default.table_683c0db2850941be84798e9dca3fe490 [id=7377fa3ea9664b47a930d9c5f88c60fa]). Expected version 1 got 0
W20260925 13:57:27.500063 22651 catalog_manager.cc:6038] Failed to send AlterTable RPC for tablet 09199f2141f746a29f410907e2a5ae37 (table default.table_683c0db2850941be84798e9dca3fe490 [id=7377fa3ea9664b47a930d9c5f88c60fa], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 09199f2141f746a29f410907e2a5ae37 consensus state has no leader
I20260925 13:57:27.504365 22907 ts_tablet_manager.cc:1929] T c42767d8f674410188835ceefa644eb5 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:27.504793 22907 log.cc:1199] T c42767d8f674410188835ceefa644eb5 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/c42767d8f674410188835ceefa644eb5
I20260925 13:57:27.501087 22766 catalog_manager.cc:2811] Deleting table default.table_f644beebc9834408bed8d57d2ac5c2ee [id=18fb001639084ecbba7437d57dfd516e] in response to Hive Metastore notification log event 41
I20260925 13:57:27.512210 22907 ts_tablet_manager.cc:1950] T c42767d8f674410188835ceefa644eb5 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:27.516039 22602 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet c42767d8f674410188835ceefa644eb5 (table default.table_b54e9b49662b4a8e892615fec1080a87 [id=722129e8c51a45f8ae31843d0d3f0a47]) successfully deleted
I20260925 13:57:27.521001 22894 ts_tablet_manager.cc:1929] T 1928234ffd3947a78e10d4c43e32aeb8 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:27.522073 22907 tablet_replica.cc:333] T 1928234ffd3947a78e10d4c43e32aeb8 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:27.522131 22917 tablet_copy_client.cc:323] T ba1242d8795641e7a89006d154be1216 P 5e2731975dbf433aade4a859df9699e2: tablet copy: Beginning tablet copy session from remote peer at address 127.18.182.194:44291
I20260925 13:57:27.523087 22907 raft_consensus.cc:2243] T 1928234ffd3947a78e10d4c43e32aeb8 P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:27.524017 22907 raft_consensus.cc:2272] T 1928234ffd3947a78e10d4c43e32aeb8 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:27.524183 21980 tablet_copy_service.cc:140] P 145e520e0a5a445d943a53cf8c27efee: Received BeginTabletCopySession request for tablet ba1242d8795641e7a89006d154be1216 from peer 5e2731975dbf433aade4a859df9699e2 ({username='slave'} at 127.18.182.193:58233)
I20260925 13:57:27.524648 21980 tablet_copy_service.cc:161] P 145e520e0a5a445d943a53cf8c27efee: Beginning new tablet copy session on tablet ba1242d8795641e7a89006d154be1216 from peer 5e2731975dbf433aade4a859df9699e2 at {username='slave'} at 127.18.182.193:58233: session id = 5e2731975dbf433aade4a859df9699e2-ba1242d8795641e7a89006d154be1216
I20260925 13:57:27.525943 22907 ts_tablet_manager.cc:1916] T 1928234ffd3947a78e10d4c43e32aeb8 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:27.529675 22874 tablet_bootstrap.cc:492] T 60ef9c152aec465eb0bb7dc286c74277 P 5e2731975dbf433aade4a859df9699e2: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 13:57:27.531755 22874 tablet_bootstrap.cc:492] T 60ef9c152aec465eb0bb7dc286c74277 P 5e2731975dbf433aade4a859df9699e2: Bootstrap complete.
I20260925 13:57:27.531958 21980 tablet_copy_source_session.cc:215] T ba1242d8795641e7a89006d154be1216 P 145e520e0a5a445d943a53cf8c27efee: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 13:57:27.532377 22874 ts_tablet_manager.cc:1403] T 60ef9c152aec465eb0bb7dc286c74277 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.248s user 0.014s sys 0.011s
I20260925 13:57:27.535159 22917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba1242d8795641e7a89006d154be1216. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:27.521230 22901 tablet_bootstrap.cc:492] T a3892ff71c0b447e86d29aed4017caf6 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:27.536890 22901 ts_tablet_manager.cc:1403] T a3892ff71c0b447e86d29aed4017caf6 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.147s user 0.022s sys 0.001s
I20260925 13:57:27.538493 22874 raft_consensus.cc:359] T 60ef9c152aec465eb0bb7dc286c74277 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:27.539520 22901 raft_consensus.cc:359] T a3892ff71c0b447e86d29aed4017caf6 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:27.543012 22901 raft_consensus.cc:385] T a3892ff71c0b447e86d29aed4017caf6 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:27.540313 22894 log.cc:1199] T 1928234ffd3947a78e10d4c43e32aeb8 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/1928234ffd3947a78e10d4c43e32aeb8
I20260925 13:57:27.547465 22894 ts_tablet_manager.cc:1950] T 1928234ffd3947a78e10d4c43e32aeb8 P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:27.548761 22874 raft_consensus.cc:740] T 60ef9c152aec465eb0bb7dc286c74277 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:27.542332 21843 heartbeater.cc:499] Master 127.18.182.253:46177 was elected leader, sending a full tablet report...
I20260925 13:57:27.546443 22901 raft_consensus.cc:740] T a3892ff71c0b447e86d29aed4017caf6 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:27.550632 22901 consensus_queue.cc:260] T a3892ff71c0b447e86d29aed4017caf6 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:27.546075 22923 ts_tablet_manager.cc:933] T 04cd31f29357471a882157829ca700ab P 5e2731975dbf433aade4a859df9699e2: Initiating tablet copy from peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:27.539975 22908 tablet_copy_client.cc:806] T a06a5aac5d464d339c870df4ebd63d92 P 5e2731975dbf433aade4a859df9699e2: tablet copy: Starting download of 0 data blocks...
I20260925 13:57:27.551683 22908 tablet_copy_client.cc:670] T a06a5aac5d464d339c870df4ebd63d92 P 5e2731975dbf433aade4a859df9699e2: tablet copy: Starting download of 1 WAL segments...
I20260925 13:57:27.555642 22908 tablet_copy_client.cc:538] T a06a5aac5d464d339c870df4ebd63d92 P 5e2731975dbf433aade4a859df9699e2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 13:57:27.568341 22901 ts_tablet_manager.cc:1434] T a3892ff71c0b447e86d29aed4017caf6 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.031s user 0.005s sys 0.000s
I20260925 13:57:27.549681 22874 consensus_queue.cc:260] T 60ef9c152aec465eb0bb7dc286c74277 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:27.575913 22857 tablet_bootstrap.cc:492] T 565a7e8f453e464b8897d84e99429067 P 5e2731975dbf433aade4a859df9699e2: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 13:57:27.556447 22609 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet 1928234ffd3947a78e10d4c43e32aeb8 (table default.table_fce30b75d46f47089d7a31e04841dcdf [id=ec768fd401eb409aaa08df2a6cdfa2f3]) successfully deleted
I20260925 13:57:27.576835 22857 tablet_bootstrap.cc:492] T 565a7e8f453e464b8897d84e99429067 P 5e2731975dbf433aade4a859df9699e2: Bootstrap complete.
W20260925 13:57:27.577267 22609 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 09199f2141f746a29f410907e2a5ae37 (table default.table_683c0db2850941be84798e9dca3fe490 [id=7377fa3ea9664b47a930d9c5f88c60fa], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 09199f2141f746a29f410907e2a5ae37 consensus state has no leader
I20260925 13:57:27.577402 22857 ts_tablet_manager.cc:1403] T 565a7e8f453e464b8897d84e99429067 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.592s user 0.011s sys 0.008s
I20260925 13:57:27.580035 22857 raft_consensus.cc:359] T 565a7e8f453e464b8897d84e99429067 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:27.584823 22874 ts_tablet_manager.cc:1434] T 60ef9c152aec465eb0bb7dc286c74277 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.052s user 0.007s sys 0.001s
I20260925 13:57:27.599632 21796 raft_consensus.cc:1217] T 60ef9c152aec465eb0bb7dc286c74277 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 0.0->[1.1-1.1] Dedup: 1.1->[]
I20260925 13:57:27.603677 22920 tablet_bootstrap.cc:654] T a3892ff71c0b447e86d29aed4017caf6 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:27.625937 22857 raft_consensus.cc:740] T 565a7e8f453e464b8897d84e99429067 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:27.605360 22908 tablet_bootstrap.cc:492] T a06a5aac5d464d339c870df4ebd63d92 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:27.626775 22857 consensus_queue.cc:260] T 565a7e8f453e464b8897d84e99429067 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:27.629464 22127 tablet_copy_service.cc:342] P ee02f77594864349982492862d668a36: Request end of tablet copy session 5e2731975dbf433aade4a859df9699e2-60ef9c152aec465eb0bb7dc286c74277 received from {username='slave'} at 127.18.182.193:43409
I20260925 13:57:27.629912 22127 tablet_copy_service.cc:434] P ee02f77594864349982492862d668a36: ending tablet copy session 5e2731975dbf433aade4a859df9699e2-60ef9c152aec465eb0bb7dc286c74277 on tablet 60ef9c152aec465eb0bb7dc286c74277 with peer 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:27.636579 22857 ts_tablet_manager.cc:1434] T 565a7e8f453e464b8897d84e99429067 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.059s user 0.001s sys 0.005s
I20260925 13:57:27.619278 22907 ts_tablet_manager.cc:1929] T 1928234ffd3947a78e10d4c43e32aeb8 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:27.638998 22907 log.cc:1199] T 1928234ffd3947a78e10d4c43e32aeb8 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/1928234ffd3947a78e10d4c43e32aeb8
I20260925 13:57:27.612950 22917 tablet_copy_client.cc:806] T ba1242d8795641e7a89006d154be1216 P 5e2731975dbf433aade4a859df9699e2: tablet copy: Starting download of 0 data blocks...
I20260925 13:57:27.615830 22923 tablet_copy_client.cc:323] T 04cd31f29357471a882157829ca700ab P 5e2731975dbf433aade4a859df9699e2: tablet copy: Beginning tablet copy session from remote peer at address 127.18.182.194:44291
I20260925 13:57:27.643072 22907 ts_tablet_manager.cc:1950] T 1928234ffd3947a78e10d4c43e32aeb8 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:27.644624 22917 tablet_copy_client.cc:670] T ba1242d8795641e7a89006d154be1216 P 5e2731975dbf433aade4a859df9699e2: tablet copy: Starting download of 1 WAL segments...
I20260925 13:57:27.644707 21980 tablet_copy_service.cc:342] P 145e520e0a5a445d943a53cf8c27efee: Request end of tablet copy session 5e2731975dbf433aade4a859df9699e2-565a7e8f453e464b8897d84e99429067 received from {username='slave'} at 127.18.182.193:58233
I20260925 13:57:27.645087 21980 tablet_copy_service.cc:434] P 145e520e0a5a445d943a53cf8c27efee: ending tablet copy session 5e2731975dbf433aade4a859df9699e2-565a7e8f453e464b8897d84e99429067 on tablet 565a7e8f453e464b8897d84e99429067 with peer 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:27.646167 21979 tablet_copy_service.cc:140] P 145e520e0a5a445d943a53cf8c27efee: Received BeginTabletCopySession request for tablet 04cd31f29357471a882157829ca700ab from peer 5e2731975dbf433aade4a859df9699e2 ({username='slave'} at 127.18.182.193:58233)
I20260925 13:57:27.646605 21979 tablet_copy_service.cc:161] P 145e520e0a5a445d943a53cf8c27efee: Beginning new tablet copy session on tablet 04cd31f29357471a882157829ca700ab from peer 5e2731975dbf433aade4a859df9699e2 at {username='slave'} at 127.18.182.193:58233: session id = 5e2731975dbf433aade4a859df9699e2-04cd31f29357471a882157829ca700ab
I20260925 13:57:27.653075 22602 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet 1928234ffd3947a78e10d4c43e32aeb8 (table default.table_fce30b75d46f47089d7a31e04841dcdf [id=ec768fd401eb409aaa08df2a6cdfa2f3]) successfully deleted
I20260925 13:57:27.655112 21979 tablet_copy_source_session.cc:215] T 04cd31f29357471a882157829ca700ab P 145e520e0a5a445d943a53cf8c27efee: Tablet Copy: opened 0 blocks and 1 log segments
W20260925 13:57:27.659461 22609 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.182.252:39283, user_credentials={real_user=slave}} blocked reactor thread for 51589.7us
I20260925 13:57:27.654457 22907 tablet_replica.cc:333] T 76d4d7d706824892b075ab1aa4fbaf85 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:27.662151 22907 raft_consensus.cc:2243] T 76d4d7d706824892b075ab1aa4fbaf85 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:27.663542 22907 raft_consensus.cc:2272] T 76d4d7d706824892b075ab1aa4fbaf85 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:27.673313 22907 ts_tablet_manager.cc:1916] T 76d4d7d706824892b075ab1aa4fbaf85 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:27.659220 22923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04cd31f29357471a882157829ca700ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:27.680559 22651 catalog_manager.cc:6391] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635: default.table_97e22ed3359342caaff80295b5d37db2 [id=dfd05c11872140cfa86b2628c33ef6d8] alter complete (version 1)
I20260925 13:57:27.684882 22917 tablet_copy_client.cc:538] T ba1242d8795641e7a89006d154be1216 P 5e2731975dbf433aade4a859df9699e2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 13:57:27.689492 22766 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635: Sending DeleteTablet for 3 replicas of tablet 565a7e8f453e464b8897d84e99429067
I20260925 13:57:27.691250 21775 tablet_service.cc:1558] Processing DeleteTablet for tablet 565a7e8f453e464b8897d84e99429067 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:57:27 UTC) from {username='slave'} at 127.0.0.1:40862
I20260925 13:57:27.693382 21938 tablet_service.cc:1558] Processing DeleteTablet for tablet 565a7e8f453e464b8897d84e99429067 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:57:27 UTC) from {username='slave'} at 127.0.0.1:59466
I20260925 13:57:27.693460 22084 tablet_service.cc:1558] Processing DeleteTablet for tablet 565a7e8f453e464b8897d84e99429067 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:57:27 UTC) from {username='slave'} at 127.0.0.1:59366
I20260925 13:57:27.694132 22889 tablet_replica.cc:333] T 565a7e8f453e464b8897d84e99429067 P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:27.694229 22894 tablet_replica.cc:333] T 565a7e8f453e464b8897d84e99429067 P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:27.694989 22889 raft_consensus.cc:2243] T 565a7e8f453e464b8897d84e99429067 P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:27.695037 22894 raft_consensus.cc:2243] T 565a7e8f453e464b8897d84e99429067 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:27.695537 22894 raft_consensus.cc:2272] T 565a7e8f453e464b8897d84e99429067 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:27.695962 22889 raft_consensus.cc:2272] T 565a7e8f453e464b8897d84e99429067 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:27.697794 22889 ts_tablet_manager.cc:1916] T 565a7e8f453e464b8897d84e99429067 P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:27.697451 22894 ts_tablet_manager.cc:1916] T 565a7e8f453e464b8897d84e99429067 P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:27.700546 22084 tablet_service.cc:1558] Processing DeleteTablet for tablet c42767d8f674410188835ceefa644eb5 with delete_type TABLET_DATA_DELETED (replaced by tablet a3892ff71c0b447e86d29aed4017caf6) from {username='slave'} at 127.0.0.1:59366
I20260925 13:57:27.701371 22084 tablet_service.cc:1558] Processing DeleteTablet for tablet 1928234ffd3947a78e10d4c43e32aeb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:57:26 UTC) from {username='slave'} at 127.0.0.1:59366
W20260925 13:57:27.708091 22644 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_b80b7994ff294977bd92dc8f9ddd922e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:57:27.718886 22908 tablet_bootstrap.cc:492] T a06a5aac5d464d339c870df4ebd63d92 P 5e2731975dbf433aade4a859df9699e2: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 13:57:27.723489 22917 tablet_bootstrap.cc:492] T ba1242d8795641e7a89006d154be1216 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:27.724436 22910 raft_consensus.cc:493] T 242acfcc09ca4f879e5e44a73ef16cec P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:27.724812 22910 raft_consensus.cc:515] T 242acfcc09ca4f879e5e44a73ef16cec P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:27.726562 22910 leader_election.cc:290] T 242acfcc09ca4f879e5e44a73ef16cec P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:27.728215 22106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "242acfcc09ca4f879e5e44a73ef16cec" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:27.728858 22106 raft_consensus.cc:2468] T 242acfcc09ca4f879e5e44a73ef16cec P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 0.
I20260925 13:57:27.730324 21896 leader_election.cc:304] T 242acfcc09ca4f879e5e44a73ef16cec P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:27.731087 22910 raft_consensus.cc:2804] T 242acfcc09ca4f879e5e44a73ef16cec P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:27.731418 22910 raft_consensus.cc:493] T 242acfcc09ca4f879e5e44a73ef16cec P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:27.731735 22910 raft_consensus.cc:3060] T 242acfcc09ca4f879e5e44a73ef16cec P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:27.736181 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "242acfcc09ca4f879e5e44a73ef16cec" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2" is_pre_election: true
W20260925 13:57:27.737531 21893 leader_election.cc:343] T 242acfcc09ca4f879e5e44a73ef16cec P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:57:27.738195 22889 ts_tablet_manager.cc:1929] T 565a7e8f453e464b8897d84e99429067 P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:27.738554 22889 log.cc:1199] T 565a7e8f453e464b8897d84e99429067 P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/565a7e8f453e464b8897d84e99429067
I20260925 13:57:27.739293 22889 ts_tablet_manager.cc:1950] T 565a7e8f453e464b8897d84e99429067 P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:27.739914 22908 tablet_bootstrap.cc:492] T a06a5aac5d464d339c870df4ebd63d92 P 5e2731975dbf433aade4a859df9699e2: Bootstrap complete.
I20260925 13:57:27.740218 22920 tablet_bootstrap.cc:492] T a3892ff71c0b447e86d29aed4017caf6 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:27.740638 22920 ts_tablet_manager.cc:1403] T a3892ff71c0b447e86d29aed4017caf6 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.258s user 0.010s sys 0.010s
I20260925 13:57:27.743304 22920 raft_consensus.cc:359] T a3892ff71c0b447e86d29aed4017caf6 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:27.743956 22920 raft_consensus.cc:385] T a3892ff71c0b447e86d29aed4017caf6 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:27.744228 22920 raft_consensus.cc:740] T a3892ff71c0b447e86d29aed4017caf6 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:27.744927 22920 consensus_queue.cc:260] T a3892ff71c0b447e86d29aed4017caf6 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:27.747318 22920 ts_tablet_manager.cc:1434] T a3892ff71c0b447e86d29aed4017caf6 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260925 13:57:27.749830 22651 catalog_manager.cc:5671] Ignoring report from unknown tablet fb781a032ba649de8a214925fdb374b5
W20260925 13:57:27.750133 22651 catalog_manager.cc:5671] Ignoring report from unknown tablet 1928234ffd3947a78e10d4c43e32aeb8
W20260925 13:57:27.750358 22651 catalog_manager.cc:5671] Ignoring report from unknown tablet 195c48349fb5410495299c9e71a72ad8
W20260925 13:57:27.750919 22651 catalog_manager.cc:5671] Ignoring report from unknown tablet f5bd96a2bdb842e9b3bc1f706786d6b1
I20260925 13:57:27.757053 22920 tablet_bootstrap.cc:492] T 242acfcc09ca4f879e5e44a73ef16cec P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:27.756045 22910 raft_consensus.cc:515] T 242acfcc09ca4f879e5e44a73ef16cec P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:27.766695 22920 tablet_bootstrap.cc:654] T 242acfcc09ca4f879e5e44a73ef16cec P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:27.740629 22908 ts_tablet_manager.cc:1403] T a06a5aac5d464d339c870df4ebd63d92 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.141s user 0.015s sys 0.003s
I20260925 13:57:27.769613 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "242acfcc09ca4f879e5e44a73ef16cec" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2"
I20260925 13:57:27.770216 22908 raft_consensus.cc:359] T a06a5aac5d464d339c870df4ebd63d92 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:27.770910 22908 raft_consensus.cc:740] T a06a5aac5d464d339c870df4ebd63d92 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
W20260925 13:57:27.771438 21893 leader_election.cc:343] T 242acfcc09ca4f879e5e44a73ef16cec P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:57:27.771695 22908 consensus_queue.cc:260] T a06a5aac5d464d339c870df4ebd63d92 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:27.754498 22642 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_8791c6009be44efab81f5f2252a3d8ae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 13:57:27.755784 22641 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_4586487550ec46f08eae599e1c34ef17 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 13:57:27.777170 22651 catalog_manager.cc:5671] Ignoring report from unknown tablet 76d4d7d706824892b075ab1aa4fbaf85
I20260925 13:57:27.779007 22106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "242acfcc09ca4f879e5e44a73ef16cec" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:27.779520 22106 raft_consensus.cc:3060] T 242acfcc09ca4f879e5e44a73ef16cec P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:27.780675 22908 ts_tablet_manager.cc:1434] T a06a5aac5d464d339c870df4ebd63d92 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.013s user 0.001s sys 0.004s
I20260925 13:57:27.782171 22127 tablet_copy_service.cc:342] P ee02f77594864349982492862d668a36: Request end of tablet copy session 5e2731975dbf433aade4a859df9699e2-a06a5aac5d464d339c870df4ebd63d92 received from {username='slave'} at 127.18.182.193:43409
I20260925 13:57:27.782536 22127 tablet_copy_service.cc:434] P ee02f77594864349982492862d668a36: ending tablet copy session 5e2731975dbf433aade4a859df9699e2-a06a5aac5d464d339c870df4ebd63d92 on tablet a06a5aac5d464d339c870df4ebd63d92 with peer 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:27.783094 22813 raft_consensus.cc:493] T 09199f2141f746a29f410907e2a5ae37 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:27.783520 22813 raft_consensus.cc:515] T 09199f2141f746a29f410907e2a5ae37 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:27.759985 22646 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_8229f26c3e104bf1885bce04e09f8e29 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:57:27.756381 22609 catalog_manager.cc:5063] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): tablet 565a7e8f453e464b8897d84e99429067 (table default.table_f644beebc9834408bed8d57d2ac5c2ee [id=18fb001639084ecbba7437d57dfd516e]) successfully deleted
I20260925 13:57:27.785256 22813 leader_election.cc:290] T 09199f2141f746a29f410907e2a5ae37 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:27.788972 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09199f2141f746a29f410907e2a5ae37" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:27.789510 22107 raft_consensus.cc:2468] T 09199f2141f746a29f410907e2a5ae37 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 0.
I20260925 13:57:27.790908 21896 leader_election.cc:304] T 09199f2141f746a29f410907e2a5ae37 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36; no voters:
W20260925 13:57:27.791638 22651 catalog_manager.cc:5671] Ignoring report from unknown tablet 761ede873f5648ccb5216a4c0a8afa2e
I20260925 13:57:27.792891 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09199f2141f746a29f410907e2a5ae37" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2" is_pre_election: true
I20260925 13:57:27.793900 22909 raft_consensus.cc:2804] T 09199f2141f746a29f410907e2a5ae37 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:27.794184 22909 raft_consensus.cc:493] T 09199f2141f746a29f410907e2a5ae37 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:27.794445 22909 raft_consensus.cc:3060] T 09199f2141f746a29f410907e2a5ae37 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
W20260925 13:57:27.805761 22651 catalog_manager.cc:5671] Ignoring report from unknown tablet 60256642160a44a2bab77b97d78c27a8
W20260925 13:57:27.806003 22651 catalog_manager.cc:5671] Ignoring report from unknown tablet 794784c42d27441f829f954b6929a097
W20260925 13:57:27.806187 22651 catalog_manager.cc:5671] Ignoring report from unknown tablet d267def3f0864435a0d2c40fd7052eab
W20260925 13:57:27.807732 21893 leader_election.cc:343] T 09199f2141f746a29f410907e2a5ae37 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:57:27.809702 22917 tablet.cc:1751] T ba1242d8795641e7a89006d154be1216 P 5e2731975dbf433aade4a859df9699e2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:57:27.822362 22106 raft_consensus.cc:2468] T 242acfcc09ca4f879e5e44a73ef16cec P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 1.
W20260925 13:57:27.823033 21575 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 5eb86772ca2e4306a5c164342514b7b0 (table default.table_d6100b0f2b294ab099c179a7681cb4ed [id=74ee67384ecf4220b7d08fdf0637aefb], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 5eb86772ca2e4306a5c164342514b7b0 consensus state has no leader
I20260925 13:57:27.823642 21896 leader_election.cc:304] T 242acfcc09ca4f879e5e44a73ef16cec P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36; no voters: 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:27.824468 22813 raft_consensus.cc:2804] T 242acfcc09ca4f879e5e44a73ef16cec P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:27.825289 22917 tablet_metadata.cc:647] Not flushing: waiting for 1 pins to be released.
I20260925 13:57:27.820160 22920 tablet_bootstrap.cc:492] T 242acfcc09ca4f879e5e44a73ef16cec P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:27.825865 22920 ts_tablet_manager.cc:1403] T 242acfcc09ca4f879e5e44a73ef16cec P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.069s user 0.009s sys 0.001s
I20260925 13:57:27.827375 22909 raft_consensus.cc:515] T 09199f2141f746a29f410907e2a5ae37 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:27.828737 22920 raft_consensus.cc:359] T 242acfcc09ca4f879e5e44a73ef16cec P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:27.829406 22920 raft_consensus.cc:385] T 242acfcc09ca4f879e5e44a73ef16cec P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:27.829715 22920 raft_consensus.cc:740] T 242acfcc09ca4f879e5e44a73ef16cec P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:27.819134 22923 tablet_copy_client.cc:806] T 04cd31f29357471a882157829ca700ab P 5e2731975dbf433aade4a859df9699e2: tablet copy: Starting download of 0 data blocks...
I20260925 13:57:27.830502 22923 tablet_copy_client.cc:670] T 04cd31f29357471a882157829ca700ab P 5e2731975dbf433aade4a859df9699e2: tablet copy: Starting download of 1 WAL segments...
I20260925 13:57:27.831936 22917 tablet_bootstrap.cc:492] T ba1242d8795641e7a89006d154be1216 P 5e2731975dbf433aade4a859df9699e2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 13:57:27.832767 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09199f2141f746a29f410907e2a5ae37" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2"
I20260925 13:57:27.836184 22650 catalog_manager.cc:5938] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873) does not have the latest schema for tablet 09199f2141f746a29f410907e2a5ae37 (table default.table_683c0db2850941be84798e9dca3fe490 [id=7377fa3ea9664b47a930d9c5f88c60fa]). Expected version 1 got 0
W20260925 13:57:27.836169 21893 leader_election.cc:343] T 09199f2141f746a29f410907e2a5ae37 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:57:27.839409 22923 tablet_copy_client.cc:538] T 04cd31f29357471a882157829ca700ab P 5e2731975dbf433aade4a859df9699e2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 13:57:27.846478 22910 leader_election.cc:290] T 242acfcc09ca4f879e5e44a73ef16cec P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Requested vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:27.848714 22106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09199f2141f746a29f410907e2a5ae37" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:27.849277 22106 raft_consensus.cc:3060] T 09199f2141f746a29f410907e2a5ae37 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:27.833177 22920 consensus_queue.cc:260] T 242acfcc09ca4f879e5e44a73ef16cec P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:27.851075 22909 leader_election.cc:290] T 09199f2141f746a29f410907e2a5ae37 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Requested vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:27.856560 22651 catalog_manager.cc:5938] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881) does not have the latest schema for tablet 09199f2141f746a29f410907e2a5ae37 (table default.table_683c0db2850941be84798e9dca3fe490 [id=7377fa3ea9664b47a930d9c5f88c60fa]). Expected version 1 got 0
I20260925 13:57:27.861635 22920 ts_tablet_manager.cc:1434] T 242acfcc09ca4f879e5e44a73ef16cec P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.035s user 0.005s sys 0.000s
I20260925 13:57:27.862926 22920 tablet_bootstrap.cc:492] T 09199f2141f746a29f410907e2a5ae37 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:27.865111 22917 tablet_bootstrap.cc:492] T ba1242d8795641e7a89006d154be1216 P 5e2731975dbf433aade4a859df9699e2: Bootstrap complete.
I20260925 13:57:27.893026 22813 raft_consensus.cc:697] T 242acfcc09ca4f879e5e44a73ef16cec P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: Becoming Leader. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Running, Role: LEADER
I20260925 13:57:27.894089 22813 consensus_queue.cc:237] T 242acfcc09ca4f879e5e44a73ef16cec P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:27.904024 22894 ts_tablet_manager.cc:1929] T 565a7e8f453e464b8897d84e99429067 P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:27.904498 22894 log.cc:1199] T 565a7e8f453e464b8897d84e99429067 P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/565a7e8f453e464b8897d84e99429067
I20260925 13:57:27.905330 22894 ts_tablet_manager.cc:1950] T 565a7e8f453e464b8897d84e99429067 P ee02f77594864349982492862d668a36: Deleting consensus metadata
W20260925 13:57:27.917084 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:57:27.917425 22923 tablet_bootstrap.cc:492] T 04cd31f29357471a882157829ca700ab P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
W20260925 13:57:27.920869 22650 catalog_manager.cc:5671] Ignoring report from unknown tablet fb781a032ba649de8a214925fdb374b5
I20260925 13:57:27.920186 22920 tablet_bootstrap.cc:654] T 09199f2141f746a29f410907e2a5ae37 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:27.910074 22106 raft_consensus.cc:2468] T 09199f2141f746a29f410907e2a5ae37 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 1.
I20260925 13:57:27.944159 21896 leader_election.cc:304] T 09199f2141f746a29f410907e2a5ae37 P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36; no voters: 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:27.945612 22933 raft_consensus.cc:2804] T 09199f2141f746a29f410907e2a5ae37 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:27.946108 22933 raft_consensus.cc:697] T 09199f2141f746a29f410907e2a5ae37 P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: Becoming Leader. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Running, Role: LEADER
I20260925 13:57:27.947324 22933 consensus_queue.cc:237] T 09199f2141f746a29f410907e2a5ae37 P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:27.934577 22609 catalog_manager.cc:5063] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): tablet 565a7e8f453e464b8897d84e99429067 (table default.table_f644beebc9834408bed8d57d2ac5c2ee [id=18fb001639084ecbba7437d57dfd516e]) successfully deleted
I20260925 13:57:27.923108 22907 ts_tablet_manager.cc:1929] T 76d4d7d706824892b075ab1aa4fbaf85 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260925 13:57:27.953579 22609 catalog_manager.cc:5030] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): delete failed for tablet c42767d8f674410188835ceefa644eb5 because the tablet was not found. No further retry: Not found: Tablet not found: c42767d8f674410188835ceefa644eb5
W20260925 13:57:27.954257 22609 catalog_manager.cc:5030] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873): delete failed for tablet 1928234ffd3947a78e10d4c43e32aeb8 because the tablet was not found. No further retry: Not found: Tablet not found: 1928234ffd3947a78e10d4c43e32aeb8
I20260925 13:57:27.958583 22908 ts_tablet_manager.cc:933] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 5e2731975dbf433aade4a859df9699e2: Initiating tablet copy from peer ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:27.958679 22917 ts_tablet_manager.cc:1403] T ba1242d8795641e7a89006d154be1216 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.244s user 0.030s sys 0.000s
I20260925 13:57:27.959077 22907 log.cc:1199] T 76d4d7d706824892b075ab1aa4fbaf85 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/76d4d7d706824892b075ab1aa4fbaf85
I20260925 13:57:27.965128 22907 ts_tablet_manager.cc:1950] T 76d4d7d706824892b075ab1aa4fbaf85 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:27.967065 22846 raft_consensus.cc:493] T 242acfcc09ca4f879e5e44a73ef16cec P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:27.967487 22846 raft_consensus.cc:515] T 242acfcc09ca4f879e5e44a73ef16cec P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:27.969435 22846 leader_election.cc:290] T 242acfcc09ca4f879e5e44a73ef16cec P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:27.971695 21958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "242acfcc09ca4f879e5e44a73ef16cec" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:27.973695 22106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "242acfcc09ca4f879e5e44a73ef16cec" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:27.974332 22106 raft_consensus.cc:2393] T 242acfcc09ca4f879e5e44a73ef16cec P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e2731975dbf433aade4a859df9699e2 in current term 1: Already voted for candidate 145e520e0a5a445d943a53cf8c27efee in this term.
I20260925 13:57:27.975499 21732 leader_election.cc:304] T 242acfcc09ca4f879e5e44a73ef16cec P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
I20260925 13:57:27.976557 22846 raft_consensus.cc:3060] T 242acfcc09ca4f879e5e44a73ef16cec P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:27.979697 22651 catalog_manager.cc:5838] T 242acfcc09ca4f879e5e44a73ef16cec P 145e520e0a5a445d943a53cf8c27efee reported cstate change: term changed from 0 to 1, leader changed from <none> to 145e520e0a5a445d943a53cf8c27efee (127.18.182.194). New cstate: current_term: 1 leader_uuid: "145e520e0a5a445d943a53cf8c27efee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: UNKNOWN } } }
I20260925 13:57:27.987170 22846 raft_consensus.cc:2749] T 242acfcc09ca4f879e5e44a73ef16cec P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:57:27.990178 22907 tablet_replica.cc:333] T 794784c42d27441f829f954b6929a097 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:27.990990 22907 raft_consensus.cc:2243] T 794784c42d27441f829f954b6929a097 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:27.991511 22907 raft_consensus.cc:2272] T 794784c42d27441f829f954b6929a097 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:27.993274 22907 ts_tablet_manager.cc:1916] T 794784c42d27441f829f954b6929a097 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:28.006536 22908 tablet_copy_client.cc:323] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 5e2731975dbf433aade4a859df9699e2: tablet copy: Beginning tablet copy session from remote peer at address 127.18.182.195:40873
I20260925 13:57:28.008242 22127 tablet_copy_service.cc:140] P ee02f77594864349982492862d668a36: Received BeginTabletCopySession request for tablet 8ace3cd70b45451d8e9fb4b8a8d4dd0d from peer 5e2731975dbf433aade4a859df9699e2 ({username='slave'} at 127.18.182.193:43409)
I20260925 13:57:28.008723 22127 tablet_copy_service.cc:161] P ee02f77594864349982492862d668a36: Beginning new tablet copy session on tablet 8ace3cd70b45451d8e9fb4b8a8d4dd0d from peer 5e2731975dbf433aade4a859df9699e2 at {username='slave'} at 127.18.182.193:43409: session id = 5e2731975dbf433aade4a859df9699e2-8ace3cd70b45451d8e9fb4b8a8d4dd0d
I20260925 13:57:28.015967 22127 tablet_copy_source_session.cc:215] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P ee02f77594864349982492862d668a36: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 13:57:28.018743 22908 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ace3cd70b45451d8e9fb4b8a8d4dd0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:28.029647 22917 raft_consensus.cc:359] T ba1242d8795641e7a89006d154be1216 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:28.030854 22917 raft_consensus.cc:740] T ba1242d8795641e7a89006d154be1216 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:28.030586 22920 tablet_bootstrap.cc:492] T 09199f2141f746a29f410907e2a5ae37 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:28.031767 22920 ts_tablet_manager.cc:1403] T 09199f2141f746a29f410907e2a5ae37 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.169s user 0.015s sys 0.000s
I20260925 13:57:28.034456 22920 raft_consensus.cc:359] T 09199f2141f746a29f410907e2a5ae37 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:28.053680 22920 raft_consensus.cc:385] T 09199f2141f746a29f410907e2a5ae37 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:28.054100 22920 raft_consensus.cc:740] T 09199f2141f746a29f410907e2a5ae37 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:28.035686 22917 consensus_queue.cc:260] T ba1242d8795641e7a89006d154be1216 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:28.058512 22917 ts_tablet_manager.cc:1434] T ba1242d8795641e7a89006d154be1216 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.100s user 0.005s sys 0.003s
I20260925 13:57:28.058956 22920 consensus_queue.cc:260] T 09199f2141f746a29f410907e2a5ae37 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:28.065380 22649 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.18.182.193:59841 (request call id 20) took 1524 ms (client timeout 1000 ms). Trace:
W20260925 13:57:28.065739 22649 rpcz_store.cc:269] 0925 13:57:26.541355 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:26.666768 (+125413us) service_pool.cc:224] Handling call
0925 13:57:26.685920 (+ 19152us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b54000104d4. stack: 00007f5e41a93323 00007f5e41a93220 00007f5e41637574 0000000000516964 00000000005168f9 00007f5e48b915c0 00007f5e48c306fa 00007f5e4a9a1d8c 00007f5e4aaa001e 00007f5e48644fb4 00007f5e48644f42 00007f5e48644e71 00007f5e48644dec 00007f5e486440a2 00007f5e468f609c 00007f5e468f54d6
0925 13:57:28.065306 (+1379386us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":15,"spinlock_wait_cycles":27128576}
I20260925 13:57:28.068336 22920 ts_tablet_manager.cc:1434] T 09199f2141f746a29f410907e2a5ae37 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.036s user 0.006s sys 0.000s
I20260925 13:57:28.078183 22923 tablet_bootstrap.cc:492] T 04cd31f29357471a882157829ca700ab P 5e2731975dbf433aade4a859df9699e2: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 13:57:28.080014 21979 tablet_copy_service.cc:342] P 145e520e0a5a445d943a53cf8c27efee: Request end of tablet copy session 5e2731975dbf433aade4a859df9699e2-ba1242d8795641e7a89006d154be1216 received from {username='slave'} at 127.18.182.193:58233
I20260925 13:57:28.080407 21979 tablet_copy_service.cc:434] P 145e520e0a5a445d943a53cf8c27efee: ending tablet copy session 5e2731975dbf433aade4a859df9699e2-ba1242d8795641e7a89006d154be1216 on tablet ba1242d8795641e7a89006d154be1216 with peer 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:28.083925 22923 tablet_bootstrap.cc:492] T 04cd31f29357471a882157829ca700ab P 5e2731975dbf433aade4a859df9699e2: Bootstrap complete.
I20260925 13:57:28.050906 22651 catalog_manager.cc:5838] T 09199f2141f746a29f410907e2a5ae37 P 145e520e0a5a445d943a53cf8c27efee reported cstate change: term changed from 0 to 1, leader changed from <none> to 145e520e0a5a445d943a53cf8c27efee (127.18.182.194). New cstate: current_term: 1 leader_uuid: "145e520e0a5a445d943a53cf8c27efee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: UNKNOWN } } }
I20260925 13:57:28.088313 22651 catalog_manager.cc:5938] TS 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291) does not have the latest schema for tablet 09199f2141f746a29f410907e2a5ae37 (table default.table_683c0db2850941be84798e9dca3fe490 [id=7377fa3ea9664b47a930d9c5f88c60fa]). Expected version 1 got 0
I20260925 13:57:28.088462 22923 ts_tablet_manager.cc:1403] T 04cd31f29357471a882157829ca700ab P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.171s user 0.011s sys 0.009s
I20260925 13:57:28.091611 22923 raft_consensus.cc:359] T 04cd31f29357471a882157829ca700ab P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:28.095548 22923 raft_consensus.cc:740] T 04cd31f29357471a882157829ca700ab P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:28.096843 22923 consensus_queue.cc:260] T 04cd31f29357471a882157829ca700ab P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:28.124397 22847 raft_consensus.cc:493] T a3892ff71c0b447e86d29aed4017caf6 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:28.125110 22847 raft_consensus.cc:515] T a3892ff71c0b447e86d29aed4017caf6 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:28.129827 22650 catalog_manager.cc:5938] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881) does not have the latest schema for tablet 09199f2141f746a29f410907e2a5ae37 (table default.table_683c0db2850941be84798e9dca3fe490 [id=7377fa3ea9664b47a930d9c5f88c60fa]). Expected version 1 got 0
I20260925 13:57:28.134766 21958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3892ff71c0b447e86d29aed4017caf6" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:28.135489 21958 raft_consensus.cc:2468] T a3892ff71c0b447e86d29aed4017caf6 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee02f77594864349982492862d668a36 in term 0.
I20260925 13:57:28.136909 22043 leader_election.cc:304] T a3892ff71c0b447e86d29aed4017caf6 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:28.137728 22941 raft_consensus.cc:2804] T a3892ff71c0b447e86d29aed4017caf6 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:28.138047 22941 raft_consensus.cc:493] T a3892ff71c0b447e86d29aed4017caf6 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:28.138337 22941 raft_consensus.cc:3060] T a3892ff71c0b447e86d29aed4017caf6 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:28.146237 22941 raft_consensus.cc:515] T a3892ff71c0b447e86d29aed4017caf6 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:28.146986 22847 leader_election.cc:290] T a3892ff71c0b447e86d29aed4017caf6 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:28.093341 22908 tablet_copy_client.cc:806] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 5e2731975dbf433aade4a859df9699e2: tablet copy: Starting download of 0 data blocks...
I20260925 13:57:28.109928 21796 raft_consensus.cc:1217] T a06a5aac5d464d339c870df4ebd63d92 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 0.0->[1.1-1.1] Dedup: 1.1->[]
I20260925 13:57:28.129504 21797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3892ff71c0b447e86d29aed4017caf6" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2" is_pre_election: true
I20260925 13:57:28.150612 21797 raft_consensus.cc:2468] T a3892ff71c0b447e86d29aed4017caf6 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee02f77594864349982492862d668a36 in term 0.
I20260925 13:57:28.152659 22908 tablet_copy_client.cc:670] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 5e2731975dbf433aade4a859df9699e2: tablet copy: Starting download of 1 WAL segments...
I20260925 13:57:28.095112 22907 ts_tablet_manager.cc:1929] T 794784c42d27441f829f954b6929a097 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:57:28.149369 21795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3892ff71c0b447e86d29aed4017caf6" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2"
I20260925 13:57:28.154048 21795 raft_consensus.cc:3060] T a3892ff71c0b447e86d29aed4017caf6 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:28.161172 21795 raft_consensus.cc:2468] T a3892ff71c0b447e86d29aed4017caf6 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee02f77594864349982492862d668a36 in term 1.
I20260925 13:57:28.168730 22040 leader_election.cc:304] T a3892ff71c0b447e86d29aed4017caf6 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:28.167747 22908 tablet_copy_client.cc:538] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 5e2731975dbf433aade4a859df9699e2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 13:57:28.177863 22907 log.cc:1199] T 794784c42d27441f829f954b6929a097 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/794784c42d27441f829f954b6929a097
I20260925 13:57:28.182615 22847 raft_consensus.cc:2804] T a3892ff71c0b447e86d29aed4017caf6 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:28.184993 22907 ts_tablet_manager.cc:1950] T 794784c42d27441f829f954b6929a097 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:28.185488 22847 raft_consensus.cc:697] T a3892ff71c0b447e86d29aed4017caf6 P ee02f77594864349982492862d668a36 [term 1 LEADER]: Becoming Leader. State: Replica: ee02f77594864349982492862d668a36, State: Running, Role: LEADER
I20260925 13:57:28.188410 22645 catalog_manager.cc:2770] deleted HMS catalog entry for table default.table_05ac73cb4d774c0fbd89f99093663e76 [id=df6afcfb1e2c463493b14228e19ade26]
I20260925 13:57:28.170308 22650 catalog_manager.cc:5938] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881) does not have the latest schema for tablet 09199f2141f746a29f410907e2a5ae37 (table default.table_683c0db2850941be84798e9dca3fe490 [id=7377fa3ea9664b47a930d9c5f88c60fa]). Expected version 1 got 0
I20260925 13:57:28.191211 21958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3892ff71c0b447e86d29aed4017caf6" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
I20260925 13:57:28.191777 21958 raft_consensus.cc:3060] T a3892ff71c0b447e86d29aed4017caf6 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:28.191965 22847 consensus_queue.cc:237] T a3892ff71c0b447e86d29aed4017caf6 P ee02f77594864349982492862d668a36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:57:28.196578 22651 catalog_manager.cc:5671] Ignoring report from unknown tablet 60256642160a44a2bab77b97d78c27a8
I20260925 13:57:28.193810 22941 leader_election.cc:290] T a3892ff71c0b447e86d29aed4017caf6 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 election: Requested vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:57:28.187273 22923 ts_tablet_manager.cc:1434] T 04cd31f29357471a882157829ca700ab P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.098s user 0.001s sys 0.005s
I20260925 13:57:28.200958 21979 tablet_copy_service.cc:342] P 145e520e0a5a445d943a53cf8c27efee: Request end of tablet copy session 5e2731975dbf433aade4a859df9699e2-04cd31f29357471a882157829ca700ab received from {username='slave'} at 127.18.182.193:58233
I20260925 13:57:28.201386 21979 tablet_copy_service.cc:434] P 145e520e0a5a445d943a53cf8c27efee: ending tablet copy session 5e2731975dbf433aade4a859df9699e2-04cd31f29357471a882157829ca700ab on tablet 04cd31f29357471a882157829ca700ab with peer 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:28.203742 22907 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40862 (request call id 6) took 1118 ms. Trace:
I20260925 13:57:28.201606 21958 raft_consensus.cc:2468] T a3892ff71c0b447e86d29aed4017caf6 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee02f77594864349982492862d668a36 in term 1.
I20260925 13:57:28.212551 22907 rpcz_store.cc:276] 0925 13:57:27.085751 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:27.085843 (+ 92us) service_pool.cc:224] Handling call
0925 13:57:27.990020 (+904177us) ts_tablet_manager.cc:1151] Deleting tablet 794784c42d27441f829f954b6929a097
0925 13:57:27.990063 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:57:28.045212 (+ 55149us) tablet_metadata.cc:665] Metadata flushed
0925 13:57:28.094984 (+ 49772us) tablet_metadata.cc:665] Metadata flushed
0925 13:57:28.203678 (+108694us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":903678}
I20260925 13:57:28.222199 22602 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet 794784c42d27441f829f954b6929a097 (table default.table_683c0db2850941be84798e9dca3fe490 [id=7377fa3ea9664b47a930d9c5f88c60fa]) successfully deleted
I20260925 13:57:28.221321 22908 tablet_bootstrap.cc:492] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:28.234555 22907 tablet_replica.cc:333] T 761ede873f5648ccb5216a4c0a8afa2e P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:28.235549 22907 raft_consensus.cc:2243] T 761ede873f5648ccb5216a4c0a8afa2e P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:28.236342 22907 raft_consensus.cc:2272] T 761ede873f5648ccb5216a4c0a8afa2e P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:28.238462 22907 ts_tablet_manager.cc:1916] T 761ede873f5648ccb5216a4c0a8afa2e P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:28.251618 22650 catalog_manager.cc:5838] T a3892ff71c0b447e86d29aed4017caf6 P ee02f77594864349982492862d668a36 reported cstate change: term changed from 0 to 1, leader changed from <none> to ee02f77594864349982492862d668a36 (127.18.182.195). New cstate: current_term: 1 leader_uuid: "ee02f77594864349982492862d668a36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: HEALTHY } } }
I20260925 13:57:28.283721 21796 raft_consensus.cc:1217] T ba1242d8795641e7a89006d154be1216 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 0.0->[1.1-1.2] Dedup: 1.2->[]
I20260925 13:57:28.306389 22106 raft_consensus.cc:1275] T 09199f2141f746a29f410907e2a5ae37 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Refusing update from remote peer 145e520e0a5a445d943a53cf8c27efee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:57:28.307860 22813 consensus_queue.cc:1048] T 09199f2141f746a29f410907e2a5ae37 P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:28.313280 21796 raft_consensus.cc:3060] T 09199f2141f746a29f410907e2a5ae37 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:28.321444 21796 raft_consensus.cc:1275] T 09199f2141f746a29f410907e2a5ae37 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Refusing update from remote peer 145e520e0a5a445d943a53cf8c27efee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:57:28.325070 22813 consensus_queue.cc:1048] T 09199f2141f746a29f410907e2a5ae37 P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 13:57:28.359639 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:57:28.386644 22651 catalog_manager.cc:5938] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873) does not have the latest schema for tablet 09199f2141f746a29f410907e2a5ae37 (table default.table_683c0db2850941be84798e9dca3fe490 [id=7377fa3ea9664b47a930d9c5f88c60fa]). Expected version 1 got 0
I20260925 13:57:28.396116 22650 catalog_manager.cc:5938] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881) does not have the latest schema for tablet 09199f2141f746a29f410907e2a5ae37 (table default.table_683c0db2850941be84798e9dca3fe490 [id=7377fa3ea9664b47a930d9c5f88c60fa]). Expected version 1 got 0
W20260925 13:57:28.411583 22651 catalog_manager.cc:5671] Ignoring report from unknown tablet fb781a032ba649de8a214925fdb374b5
I20260925 13:57:28.456028 22952 tablet.cc:1751] T 09199f2141f746a29f410907e2a5ae37 P 5e2731975dbf433aade4a859df9699e2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:57:28.477345 22650 catalog_manager.cc:5938] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881) does not have the latest schema for tablet 09199f2141f746a29f410907e2a5ae37 (table default.table_683c0db2850941be84798e9dca3fe490 [id=7377fa3ea9664b47a930d9c5f88c60fa]). Expected version 1 got 0
I20260925 13:57:28.496975 22950 tablet.cc:1751] T 09199f2141f746a29f410907e2a5ae37 P 145e520e0a5a445d943a53cf8c27efee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:57:28.500928 22951 tablet.cc:1751] T 09199f2141f746a29f410907e2a5ae37 P ee02f77594864349982492862d668a36: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:57:28.537144 22651 catalog_manager.cc:5938] TS ee02f77594864349982492862d668a36 (127.18.182.195:40873) does not have the latest schema for tablet 09199f2141f746a29f410907e2a5ae37 (table default.table_683c0db2850941be84798e9dca3fe490 [id=7377fa3ea9664b47a930d9c5f88c60fa]). Expected version 1 got 0
I20260925 13:57:28.609711 22907 ts_tablet_manager.cc:1929] T 761ede873f5648ccb5216a4c0a8afa2e P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:57:28.610422 22907 log.cc:1199] T 761ede873f5648ccb5216a4c0a8afa2e P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/761ede873f5648ccb5216a4c0a8afa2e
I20260925 13:57:28.615031 22644 catalog_manager.cc:2213] created HMS catalog entry for table [id=1f8524e67c714a18b1a8dca4d718ace1]
I20260925 13:57:28.636519 21796 raft_consensus.cc:1217] T 04cd31f29357471a882157829ca700ab P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 0.0->[1.1-1.1] Dedup: 1.1->[]
I20260925 13:57:28.651062 22644 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:41880 (request call id 37) took 2153 ms. Trace:
I20260925 13:57:28.651422 22644 rpcz_store.cc:276] 0925 13:57:26.497437 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:26.497543 (+ 106us) service_pool.cc:224] Handling call
0925 13:57:27.708681 (+1211138us) catalog_manager.cc:2140] Acquired catalog manager lock
0925 13:57:27.712447 (+ 3766us) catalog_manager.cc:2186] Created new table and tablet info
0925 13:57:28.614977 (+902530us) catalog_manager.cc:2212] Created new table in HMS catalog
0925 13:57:28.650338 (+ 35361us) catalog_manager.cc:2246] Wrote table and tablets to system table
0925 13:57:28.650650 (+ 312us) catalog_manager.cc:2280] Inserted table and tablets into CatalogManager maps
0925 13:57:28.650999 (+ 349us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:57:28.617133 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 13:57:28.617591 (+ 458us) write_op.cc:432] Acquiring schema lock in shared mode
0925 13:57:28.617617 (+ 26us) write_op.cc:435] Acquired schema lock
0925 13:57:28.617632 (+ 15us) tablet.cc:662] Decoding operations
0925 13:57:28.617953 (+ 321us) write_op.cc:620] Acquiring the partition lock for write op
0925 13:57:28.617999 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0925 13:57:28.618024 (+ 25us) tablet.cc:685] Acquiring locks for 2 operations
0925 13:57:28.618245 (+ 221us) tablet.cc:701] Row locks acquired
0925 13:57:28.618256 (+ 11us) write_op.cc:260] PREPARE: finished
0925 13:57:28.618394 (+ 138us) write_op.cc:270] Start()
0925 13:57:28.618496 (+ 102us) write_op.cc:276] Timestamp: P: 1790344648618286 usec, L: 0
0925 13:57:28.618511 (+ 15us) op_driver.cc:348] REPLICATION: starting
0925 13:57:28.619041 (+ 530us) log.cc:844] Serialized 433 byte log entry
0925 13:57:28.629712 (+ 10671us) spinlock_profiling.cc:243] Waited 8 ms on lock 0x7b180003c220. stack: 00007f5e41a93323 00007f5e41a93220 00007f5e41637574 0000000000516964 00000000005168f9 00007f5e4a9bfb81 00007f5e4a9cc571 00007f5e41ac5363 00007f5e41ac7751 00007f5e41ac7709 00007f5e41ac7699 00007f5e41ac7661 00007f5e41ac695d 00000000005bd4e4 00000000005bd319 00007f5e41aa881a
0925 13:57:28.633926 (+ 4214us) spinlock_profiling.cc:243] Waited 9.01 ms on lock 0x7b180003c220. stack: 00007f5e41a93323 00007f5e41a93220 00007f5e41637574 0000000000516964 00000000005168f9 00007f5e4a9bfb81 00007f5e4a9cc571 00007f5e41ac5363 00007f5e41ac7751 00007f5e41ac7709 00007f5e41ac7699 00007f5e41ac7661 00007f5e41ac695d 00000000005bd4e4 00000000005bd319 00007f5e41aa881a
0925 13:57:28.646539 (+ 12613us) op_driver.cc:464] REPLICATION: finished
0925 13:57:28.647239 (+ 700us) write_op.cc:301] APPLY: starting
0925 13:57:28.647295 (+ 56us) tablet.cc:1394] starting BulkCheckPresence
0925 13:57:28.647418 (+ 123us) tablet.cc:1397] finished BulkCheckPresence
0925 13:57:28.647434 (+ 16us) tablet.cc:1399] starting ApplyRowOperation cycle
0925 13:57:28.647809 (+ 375us) tablet.cc:1410] finished ApplyRowOperation cycle
0925 13:57:28.647948 (+ 139us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 13:57:28.647989 (+ 41us) write_op.cc:312] APPLY: finished
0925 13:57:28.648153 (+ 164us) log.cc:844] Serialized 29 byte log entry
0925 13:57:28.648286 (+ 133us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:28.648353 (+ 67us) write_op.cc:454] Released schema lock
0925 13:57:28.648385 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0925 13:57:28.648580 (+ 195us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:28.648605 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":474909,"Hive Metastore.run_cpu_time_us":663,"Hive Metastore.run_wall_time_us":426625,"child_traces":[["op",{"apply.queue_time_us":544,"apply.run_cpu_time_us":1322,"apply.run_wall_time_us":1322,"num_ops":2,"prepare.queue_time_us":428,"prepare.run_cpu_time_us":2239,"prepare.run_wall_time_us":2240,"raft.queue_time_us":6103,"raft.run_cpu_time_us":1532,"raft.run_wall_time_us":1531,"replication_time_us":27929,"spinlock_wait_cycles":39107200}]]}
I20260925 13:57:28.670650 19163 master-stress-test.cc:581] Killed test thread 1/10
I20260925 13:57:28.670928 19163 master-stress-test.cc:579] Killing test thread 2/10
W20260925 13:57:28.680951 22908 log.cc:927] Time spent T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 5e2731975dbf433aade4a859df9699e2: Append to log took a long time: real 0.086s user 0.000s sys 0.003s
I20260925 13:57:28.683283 22908 tablet.cc:1751] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 5e2731975dbf433aade4a859df9699e2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 13:57:28.684113 22908 tablet_metadata.cc:647] Not flushing: waiting for 1 pins to be released.
I20260925 13:57:28.611251 22907 ts_tablet_manager.cc:1950] T 761ede873f5648ccb5216a4c0a8afa2e P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:28.734027 22907 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40862 (request call id 4) took 1650 ms. Trace:
I20260925 13:57:28.734468 22907 rpcz_store.cc:276] 0925 13:57:27.083781 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:27.086177 (+ 2396us) service_pool.cc:224] Handling call
0925 13:57:28.223025 (+1136848us) ts_tablet_manager.cc:1151] Deleting tablet 761ede873f5648ccb5216a4c0a8afa2e
0925 13:57:28.231412 (+ 8387us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:57:28.385154 (+153742us) tablet_metadata.cc:665] Metadata flushed
0925 13:57:28.609586 (+224432us) tablet_metadata.cc:665] Metadata flushed
0925 13:57:28.733950 (+124364us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1136187}
I20260925 13:57:28.735993 22602 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet 761ede873f5648ccb5216a4c0a8afa2e (table default.table_d6100b0f2b294ab099c179a7681cb4ed [id=74ee67384ecf4220b7d08fdf0637aefb]) successfully deleted
I20260925 13:57:28.737360 22907 tablet_replica.cc:333] T 195c48349fb5410495299c9e71a72ad8 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:28.748323 22908 tablet_bootstrap.cc:492] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 5e2731975dbf433aade4a859df9699e2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 13:57:28.786795 22908 tablet_bootstrap.cc:492] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 5e2731975dbf433aade4a859df9699e2: Bootstrap complete.
I20260925 13:57:28.788671 22907 raft_consensus.cc:2243] T 195c48349fb5410495299c9e71a72ad8 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:28.789981 22907 raft_consensus.cc:2272] T 195c48349fb5410495299c9e71a72ad8 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 13:57:28.791023 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:57:28.793411 22907 ts_tablet_manager.cc:1916] T 195c48349fb5410495299c9e71a72ad8 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:28.820962 22908 ts_tablet_manager.cc:1403] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.600s user 0.028s sys 0.004s
I20260925 13:57:28.833830 22813 consensus_queue.cc:1048] T 242acfcc09ca4f879e5e44a73ef16cec P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:28.845798 22908 raft_consensus.cc:359] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:28.860474 22908 raft_consensus.cc:740] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:28.863719 22908 consensus_queue.cc:260] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:28.865566 22908 ts_tablet_manager.cc:1434] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.022s user 0.003s sys 0.001s
I20260925 13:57:28.866963 22127 tablet_copy_service.cc:342] P ee02f77594864349982492862d668a36: Request end of tablet copy session 5e2731975dbf433aade4a859df9699e2-8ace3cd70b45451d8e9fb4b8a8d4dd0d received from {username='slave'} at 127.18.182.193:43409
I20260925 13:57:28.867341 22127 tablet_copy_service.cc:434] P ee02f77594864349982492862d668a36: ending tablet copy session 5e2731975dbf433aade4a859df9699e2-8ace3cd70b45451d8e9fb4b8a8d4dd0d on tablet 8ace3cd70b45451d8e9fb4b8a8d4dd0d with peer 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:28.926564 22932 consensus_queue.cc:1048] T 242acfcc09ca4f879e5e44a73ef16cec P 145e520e0a5a445d943a53cf8c27efee [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:28.964972 22907 ts_tablet_manager.cc:1929] T 195c48349fb5410495299c9e71a72ad8 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:28.965471 22907 log.cc:1199] T 195c48349fb5410495299c9e71a72ad8 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/195c48349fb5410495299c9e71a72ad8
I20260925 13:57:28.966383 22907 ts_tablet_manager.cc:1950] T 195c48349fb5410495299c9e71a72ad8 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:28.974740 21796 raft_consensus.cc:1217] T 8ace3cd70b45451d8e9fb4b8a8d4dd0d P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 0.0->[1.1-1.2] Dedup: 1.2->[]
I20260925 13:57:28.991724 22847 consensus_queue.cc:1048] T a3892ff71c0b447e86d29aed4017caf6 P ee02f77594864349982492862d668a36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 13:57:29.016716 22956 consensus_queue.cc:1048] T a3892ff71c0b447e86d29aed4017caf6 P ee02f77594864349982492862d668a36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 13:57:29.053898 22907 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40862 (request call id 7) took 1967 ms. Trace:
W20260925 13:57:29.067227 22651 catalog_manager.cc:5671] Ignoring report from unknown tablet fb781a032ba649de8a214925fdb374b5
I20260925 13:57:29.069712 22907 rpcz_store.cc:276] 0925 13:57:27.086690 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:27.086784 (+ 94us) service_pool.cc:224] Handling call
0925 13:57:28.737201 (+1650417us) ts_tablet_manager.cc:1151] Deleting tablet 195c48349fb5410495299c9e71a72ad8
0925 13:57:28.737244 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:57:28.858372 (+121128us) tablet_metadata.cc:665] Metadata flushed
0925 13:57:28.964816 (+106444us) tablet_metadata.cc:665] Metadata flushed
0925 13:57:29.053817 (+ 89001us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1649909}
I20260925 13:57:29.078105 22907 tablet_replica.cc:333] T fb781a032ba649de8a214925fdb374b5 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:29.078318 22602 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet 195c48349fb5410495299c9e71a72ad8 (table default.table_97e22ed3359342caaff80295b5d37db2 [id=dfd05c11872140cfa86b2628c33ef6d8]) successfully deleted
I20260925 13:57:29.079195 22907 raft_consensus.cc:2243] T fb781a032ba649de8a214925fdb374b5 P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:29.080595 22907 raft_consensus.cc:2272] T fb781a032ba649de8a214925fdb374b5 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:29.090581 22907 ts_tablet_manager.cc:1916] T fb781a032ba649de8a214925fdb374b5 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:29.125697 22907 ts_tablet_manager.cc:1929] T fb781a032ba649de8a214925fdb374b5 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:29.131337 22907 log.cc:1199] T fb781a032ba649de8a214925fdb374b5 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/fb781a032ba649de8a214925fdb374b5
I20260925 13:57:29.132292 22907 ts_tablet_manager.cc:1950] T fb781a032ba649de8a214925fdb374b5 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:29.134351 22907 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40862 (request call id 8) took 2046 ms. Trace:
I20260925 13:57:29.136642 22907 rpcz_store.cc:276] 0925 13:57:27.088063 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:27.121570 (+ 33507us) service_pool.cc:224] Handling call
0925 13:57:29.077926 (+1956356us) ts_tablet_manager.cc:1151] Deleting tablet fb781a032ba649de8a214925fdb374b5
0925 13:57:29.077971 (+ 45us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:57:29.104493 (+ 26522us) tablet_metadata.cc:665] Metadata flushed
0925 13:57:29.125565 (+ 21072us) tablet_metadata.cc:665] Metadata flushed
0925 13:57:29.134295 (+ 8730us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1955791}
I20260925 13:57:29.139755 22602 catalog_manager.cc:5063] TS 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): tablet fb781a032ba649de8a214925fdb374b5 (table default.table_d8ce7cda4ecf4422a5e74c80d7ebf4da [id=af0f962d5d5d4a17a3f576ab6a3b4df0]) successfully deleted
I20260925 13:57:29.142056 22907 tablet_replica.cc:333] T 565a7e8f453e464b8897d84e99429067 P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:29.142861 22907 raft_consensus.cc:2243] T 565a7e8f453e464b8897d84e99429067 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:29.146802 22907 raft_consensus.cc:2272] T 565a7e8f453e464b8897d84e99429067 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:29.148747 22907 ts_tablet_manager.cc:1916] T 565a7e8f453e464b8897d84e99429067 P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:29.156289 22647 catalog_manager.cc:3541] renamed table default.table_1baae11db5a24d50bd596853233007cb [id=bb79c1d2d0094785b35250fb1b718b9a] in HMS: new name default.table_2a09482ff526433a8d028a964cfdfb0c
I20260925 13:57:29.164791 22907 ts_tablet_manager.cc:1929] T 565a7e8f453e464b8897d84e99429067 P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 13:57:29.165169 22907 log.cc:1199] T 565a7e8f453e464b8897d84e99429067 P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/565a7e8f453e464b8897d84e99429067
I20260925 13:57:29.165901 22907 ts_tablet_manager.cc:1950] T 565a7e8f453e464b8897d84e99429067 P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:29.169036 22907 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40862 (request call id 10) took 1478 ms. Trace:
I20260925 13:57:29.169345 22907 rpcz_store.cc:276] 0925 13:57:27.690978 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:27.691084 (+ 106us) service_pool.cc:224] Handling call
0925 13:57:29.141902 (+1450818us) ts_tablet_manager.cc:1151] Deleting tablet 565a7e8f453e464b8897d84e99429067
0925 13:57:29.141945 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0925 13:57:29.157103 (+ 15158us) tablet_metadata.cc:665] Metadata flushed
0925 13:57:29.164666 (+ 7563us) tablet_metadata.cc:665] Metadata flushed
0925 13:57:29.168972 (+ 4306us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1450184}
I20260925 13:57:29.170495 22966 raft_consensus.cc:493] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3ea1e11bc6d84114b7616815b6af4635)
I20260925 13:57:29.171133 22966 raft_consensus.cc:515] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:29.175135 22676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e942d7f2e4864b5ca08e6e830085caa0" candidate_term: 6 candidate_status { last_received { term: 5 index: 217 } } ignore_live_leader: false dest_uuid: "3ea1e11bc6d84114b7616815b6af4635" is_pre_election: true
I20260925 13:57:29.176782 21598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e942d7f2e4864b5ca08e6e830085caa0" candidate_term: 6 candidate_status { last_received { term: 5 index: 217 } } ignore_live_leader: false dest_uuid: "a9bf5a5925ac42769e474adc84a1b725" is_pre_election: true
I20260925 13:57:29.177341 21598 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e942d7f2e4864b5ca08e6e830085caa0 in term 5.
I20260925 13:57:29.178988 21422 leader_election.cc:304] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a9bf5a5925ac42769e474adc84a1b725, e942d7f2e4864b5ca08e6e830085caa0; no voters: 3ea1e11bc6d84114b7616815b6af4635
W20260925 13:57:29.179733 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:57:29.185626 22966 leader_election.cc:290] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3ea1e11bc6d84114b7616815b6af4635 (127.18.182.253:46177), a9bf5a5925ac42769e474adc84a1b725 (127.18.182.252:39283)
I20260925 13:57:29.186667 22966 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260925 13:57:29.187556 22966 raft_consensus.cc:493] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3ea1e11bc6d84114b7616815b6af4635)
I20260925 13:57:29.188313 22966 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 5 FOLLOWER]: Advancing to term 6
I20260925 13:57:29.204144 22966 raft_consensus.cc:515] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:29.214447 22676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e942d7f2e4864b5ca08e6e830085caa0" candidate_term: 6 candidate_status { last_received { term: 5 index: 217 } } ignore_live_leader: false dest_uuid: "3ea1e11bc6d84114b7616815b6af4635"
I20260925 13:57:29.218067 21598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e942d7f2e4864b5ca08e6e830085caa0" candidate_term: 6 candidate_status { last_received { term: 5 index: 217 } } ignore_live_leader: false dest_uuid: "a9bf5a5925ac42769e474adc84a1b725"
I20260925 13:57:29.218638 21598 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 5 FOLLOWER]: Advancing to term 6
I20260925 13:57:29.227892 21598 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e942d7f2e4864b5ca08e6e830085caa0 in term 6.
I20260925 13:57:29.215473 22966 leader_election.cc:290] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [CANDIDATE]: Term 6 election: Requested vote from peers 3ea1e11bc6d84114b7616815b6af4635 (127.18.182.253:46177), a9bf5a5925ac42769e474adc84a1b725 (127.18.182.252:39283)
I20260925 13:57:29.232232 21422 leader_election.cc:304] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a9bf5a5925ac42769e474adc84a1b725, e942d7f2e4864b5ca08e6e830085caa0; no voters: 3ea1e11bc6d84114b7616815b6af4635
I20260925 13:57:29.234609 22966 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 6 FOLLOWER]: Leader election won for term 6
I20260925 13:57:29.235395 22966 raft_consensus.cc:697] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 6 LEADER]: Becoming Leader. State: Replica: e942d7f2e4864b5ca08e6e830085caa0, State: Running, Role: LEADER
I20260925 13:57:29.236503 22966 consensus_queue.cc:237] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 217, Committed index: 217, Last appended: 5.217, Last appended by leader: 217, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:29.240814 22967 sys_catalog.cc:455] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e942d7f2e4864b5ca08e6e830085caa0. Latest consensus state: current_term: 6 leader_uuid: "e942d7f2e4864b5ca08e6e830085caa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:57:29.241550 22967 sys_catalog.cc:458] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [sys.catalog]: This master's current role is: LEADER
I20260925 13:57:29.254293 22968 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260925 13:57:29.271770 21598 raft_consensus.cc:1240] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 6 FOLLOWER]: Rejecting Update request from peer 3ea1e11bc6d84114b7616815b6af4635 for earlier term 5. Current term is 6. Ops: []
I20260925 13:57:29.279525 22969 consensus_queue.cc:1059] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 }, Status: INVALID_TERM, Last received: 5.217, Next index: 218, Last known committed idx: 217, Time since last communication: 0.000s
I20260925 13:57:29.283339 22969 raft_consensus.cc:3055] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 5 LEADER]: Stepping down as leader of term 5
I20260925 13:57:29.283860 22969 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3ea1e11bc6d84114b7616815b6af4635, State: Running, Role: LEADER
I20260925 13:57:29.285346 22969 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 217, Committed index: 217, Last appended: 5.217, Last appended by leader: 217, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:29.288472 22969 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 5 FOLLOWER]: Advancing to term 6
I20260925 13:57:29.303241 22968 catalog_manager.cc:692] Loaded metadata for table default.table_2138b0cdfa7c4afda3a919dd0945a2d1 [id=012dc51aeefc492ca9aade1c75b97020]
I20260925 13:57:29.308218 22968 catalog_manager.cc:692] Loaded metadata for table default.table_b80b7994ff294977bd92dc8f9ddd922e [id=1f8524e67c714a18b1a8dca4d718ace1]
I20260925 13:57:29.310155 22968 catalog_manager.cc:692] Loaded metadata for table default.table_e0eb6c1d49224d1cb9b4f6d06f0f7a29 [id=420cd09cc1184f6e9fb38b2cbc3f6fce]
I20260925 13:57:29.316319 22968 catalog_manager.cc:692] Loaded metadata for table default.table_b54e9b49662b4a8e892615fec1080a87 [id=722129e8c51a45f8ae31843d0d3f0a47]
I20260925 13:57:29.318008 22968 catalog_manager.cc:692] Loaded metadata for table default.table_683c0db2850941be84798e9dca3fe490 [id=7377fa3ea9664b47a930d9c5f88c60fa]
I20260925 13:57:29.319715 22968 catalog_manager.cc:692] Loaded metadata for table default.table_d6100b0f2b294ab099c179a7681cb4ed [id=74ee67384ecf4220b7d08fdf0637aefb]
I20260925 13:57:29.321436 22968 catalog_manager.cc:692] Loaded metadata for table default.table_fe2e32c0275e4744b1163ae7b22eaf51 [id=9e8b522f92404f2ba5505e6fa689ea5a]
I20260925 13:57:29.330793 22968 catalog_manager.cc:692] Loaded metadata for table default.table_d8ce7cda4ecf4422a5e74c80d7ebf4da [id=af0f962d5d5d4a17a3f576ab6a3b4df0]
I20260925 13:57:29.349666 22968 catalog_manager.cc:692] Loaded metadata for table default.table_1baae11db5a24d50bd596853233007cb [id=bb79c1d2d0094785b35250fb1b718b9a]
I20260925 13:57:29.369060 22968 catalog_manager.cc:692] Loaded metadata for table default.table_30772fdea84947818aff6f6522c73f4b [id=bcd7dc63c23c475e943a6d7395fcec10]
I20260925 13:57:29.388116 22968 catalog_manager.cc:692] Loaded metadata for table default.table_dc864d7428e5405da360d2f379eef37b [id=bf8cef2239904fd989622f9426412635]
I20260925 13:57:29.405664 22968 catalog_manager.cc:692] Loaded metadata for table default.table_05ac73cb4d774c0fbd89f99093663e76 [id=df6afcfb1e2c463493b14228e19ade26]
I20260925 13:57:29.416555 22968 catalog_manager.cc:692] Loaded metadata for table default.table_97e22ed3359342caaff80295b5d37db2 [id=dfd05c11872140cfa86b2628c33ef6d8]
I20260925 13:57:29.451768 22968 catalog_manager.cc:692] Loaded metadata for table default.table_fc718993756d497eb9df6af8c1651362 [id=e02d9bf4f1ff4d81bb26cddfe336f414]
W20260925 13:57:29.600026 21619 debug-util.cc:398] Leaking SignalData structure 0x7b08000996c0 after lost signal to thread 21550
I20260925 13:57:29.600488 22965 raft_consensus.cc:493] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ee02f77594864349982492862d668a36)
W20260925 13:57:29.600709 21619 debug-util.cc:398] Leaking SignalData structure 0x7b080004d400 after lost signal to thread 21622
I20260925 13:57:29.600931 22965 raft_consensus.cc:515] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:29.602758 22965 leader_election.cc:290] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 13:57:29.606890 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
W20260925 13:57:29.607517 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
I20260925 13:57:29.607828 21732 leader_election.cc:304] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
I20260925 13:57:29.608780 22965 raft_consensus.cc:2749] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:57:29.648772 22968 tablet_loader.cc:96] loaded metadata for tablet 04cd31f29357471a882157829ca700ab (table default.table_fc718993756d497eb9df6af8c1651362 [id=e02d9bf4f1ff4d81bb26cddfe336f414])
I20260925 13:57:29.663029 22968 tablet_loader.cc:96] loaded metadata for tablet 09199f2141f746a29f410907e2a5ae37 (table default.table_683c0db2850941be84798e9dca3fe490 [id=7377fa3ea9664b47a930d9c5f88c60fa])
I20260925 13:57:29.664273 22968 tablet_loader.cc:96] loaded metadata for tablet 1d47ea37ee1e4abfbc0f47001c811748 (table default.table_b80b7994ff294977bd92dc8f9ddd922e [id=1f8524e67c714a18b1a8dca4d718ace1])
I20260925 13:57:29.696900 22968 tablet_loader.cc:96] loaded metadata for tablet 242acfcc09ca4f879e5e44a73ef16cec (table default.table_d8ce7cda4ecf4422a5e74c80d7ebf4da [id=af0f962d5d5d4a17a3f576ab6a3b4df0])
I20260925 13:57:29.705741 22968 tablet_loader.cc:96] loaded metadata for tablet 24dd39b302344f8d8f5166e9d641d13a (table default.table_e0eb6c1d49224d1cb9b4f6d06f0f7a29 [id=420cd09cc1184f6e9fb38b2cbc3f6fce])
I20260925 13:57:29.723945 22676 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 6 FOLLOWER]: Refusing update from remote peer e942d7f2e4864b5ca08e6e830085caa0: Log matching property violated. Preceding OpId in replica: term: 5 index: 217. Preceding OpId from leader: term: 6 index: 218. (index mismatch)
I20260925 13:57:29.722348 22968 tablet_loader.cc:96] loaded metadata for tablet 56a39663479d40be8bbd1dfd4c91040c (table default.table_05ac73cb4d774c0fbd89f99093663e76 [id=df6afcfb1e2c463493b14228e19ade26])
I20260925 13:57:29.726634 22967 consensus_queue.cc:1048] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 218, Last known committed idx: 217, Time since last communication: 0.000s
I20260925 13:57:29.743885 22969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e942d7f2e4864b5ca08e6e830085caa0. Latest consensus state: current_term: 6 leader_uuid: "e942d7f2e4864b5ca08e6e830085caa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:57:29.744582 22969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 13:57:29.768446 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:29.771886 22603 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:57:29.772897 22968 tablet_loader.cc:96] loaded metadata for tablet 60ef9c152aec465eb0bb7dc286c74277 (table default.table_30772fdea84947818aff6f6522c73f4b [id=bcd7dc63c23c475e943a6d7395fcec10])
I20260925 13:57:29.786680 22968 tablet_loader.cc:96] loaded metadata for tablet 88b875abab81457481d4b98570a74ea1 (table default.table_fe2e32c0275e4744b1163ae7b22eaf51 [id=9e8b522f92404f2ba5505e6fa689ea5a])
I20260925 13:57:29.804970 22968 tablet_loader.cc:96] loaded metadata for tablet 8ace3cd70b45451d8e9fb4b8a8d4dd0d (table default.table_d6100b0f2b294ab099c179a7681cb4ed [id=74ee67384ecf4220b7d08fdf0637aefb])
I20260925 13:57:29.815752 22969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "e942d7f2e4864b5ca08e6e830085caa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:57:29.816803 22969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:57:29.817934 21598 raft_consensus.cc:1275] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 6 FOLLOWER]: Refusing update from remote peer e942d7f2e4864b5ca08e6e830085caa0: Log matching property violated. Preceding OpId in replica: term: 5 index: 217. Preceding OpId from leader: term: 6 index: 218. (index mismatch)
I20260925 13:57:29.822201 22968 tablet_loader.cc:96] loaded metadata for tablet a06a5aac5d464d339c870df4ebd63d92 (table default.table_1baae11db5a24d50bd596853233007cb [id=bb79c1d2d0094785b35250fb1b718b9a])
I20260925 13:57:29.824266 22974 consensus_queue.cc:1048] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 218, Last known committed idx: 217, Time since last communication: 0.000s
I20260925 13:57:29.843312 22968 tablet_loader.cc:96] loaded metadata for tablet a3892ff71c0b447e86d29aed4017caf6 (table default.table_b54e9b49662b4a8e892615fec1080a87 [id=722129e8c51a45f8ae31843d0d3f0a47])
I20260925 13:57:29.852892 22967 sys_catalog.cc:455] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 6 leader_uuid: "e942d7f2e4864b5ca08e6e830085caa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:57:29.854537 22967 sys_catalog.cc:458] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [sys.catalog]: This master's current role is: LEADER
I20260925 13:57:29.855844 22968 tablet_loader.cc:96] loaded metadata for tablet ba1242d8795641e7a89006d154be1216 (table default.table_97e22ed3359342caaff80295b5d37db2 [id=dfd05c11872140cfa86b2628c33ef6d8])
I20260925 13:57:29.858937 22968 tablet_loader.cc:96] loaded metadata for tablet d9dbc7ac9d0142c8a92262d2117ee6ec (table default.table_2138b0cdfa7c4afda3a919dd0945a2d1 [id=012dc51aeefc492ca9aade1c75b97020])
I20260925 13:57:29.861466 22968 tablet_loader.cc:96] loaded metadata for tablet ef57aa72abb2405b8187a7ed56d57f96 (table default.table_dc864d7428e5405da360d2f379eef37b [id=bf8cef2239904fd989622f9426412635])
I20260925 13:57:29.868018 21441 raft_consensus.cc:1240] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [term 6 LEADER]: Rejecting Update request from peer 3ea1e11bc6d84114b7616815b6af4635 for earlier term 5. Current term is 6. Ops: []
I20260925 13:57:29.869663 22968 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260925 13:57:29.883191 22968 catalog_manager.cc:1308] Loaded cluster ID: d5d0bc5567a74d9cad4d92a9d1041fd9
I20260925 13:57:29.883606 22968 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260925 13:57:29.898449 22968 catalog_manager.cc:1553] Loading token signing keys...
I20260925 13:57:29.911638 22968 catalog_manager.cc:6230] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0: Loaded TSK: 0
I20260925 13:57:29.913156 22968 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260925 13:57:29.927652 22968 catalog_manager.cc:1576] Loading latest processed Hive Metastore notification log event ID...
I20260925 13:57:29.940711 22968 catalog_manager.cc:6068] Last processed Hive Metastore notification event ID: 41
I20260925 13:57:30.005836 22648 catalog_manager.cc:3541] renamed table default.table_dc864d7428e5405da360d2f379eef37b [id=bf8cef2239904fd989622f9426412635] in HMS: new name default.table_b32d230a905d44b388dc9e436965cbc9
I20260925 13:57:30.006435 22980 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e942d7f2e4864b5ca08e6e830085caa0. Latest consensus state: current_term: 6 leader_uuid: "e942d7f2e4864b5ca08e6e830085caa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:57:30.007107 22980 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:57:30.011751 22983 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "e942d7f2e4864b5ca08e6e830085caa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:57:30.014760 22983 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:57:30.031569 22975 sys_catalog.cc:455] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 6 leader_uuid: "e942d7f2e4864b5ca08e6e830085caa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:57:30.032279 22975 sys_catalog.cc:458] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0 [sys.catalog]: This master's current role is: LEADER
I20260925 13:57:30.145949 21775 tablet_service.cc:1511] Processing CreateTablet for tablet 1d47ea37ee1e4abfbc0f47001c811748 (DEFAULT_TABLE table=default.table_b80b7994ff294977bd92dc8f9ddd922e [id=1f8524e67c714a18b1a8dca4d718ace1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:30.147403 21775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d47ea37ee1e4abfbc0f47001c811748. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:30.151544 21938 tablet_service.cc:1511] Processing CreateTablet for tablet 1d47ea37ee1e4abfbc0f47001c811748 (DEFAULT_TABLE table=default.table_b80b7994ff294977bd92dc8f9ddd922e [id=1f8524e67c714a18b1a8dca4d718ace1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:30.152714 21938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d47ea37ee1e4abfbc0f47001c811748. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:30.164952 22084 tablet_service.cc:1511] Processing CreateTablet for tablet 1d47ea37ee1e4abfbc0f47001c811748 (DEFAULT_TABLE table=default.table_b80b7994ff294977bd92dc8f9ddd922e [id=1f8524e67c714a18b1a8dca4d718ace1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:30.166106 22084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d47ea37ee1e4abfbc0f47001c811748. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:57:30.173488 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:57:30.222065 22997 tablet_bootstrap.cc:492] T 1d47ea37ee1e4abfbc0f47001c811748 P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:30.229770 22998 tablet_bootstrap.cc:492] T 1d47ea37ee1e4abfbc0f47001c811748 P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:30.255445 22154 heartbeater.cc:499] Master 127.18.182.254:43163 was elected leader, sending a full tablet report...
I20260925 13:57:30.270925 22997 tablet_bootstrap.cc:654] T 1d47ea37ee1e4abfbc0f47001c811748 P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:30.280488 22996 tablet_bootstrap.cc:492] T 1d47ea37ee1e4abfbc0f47001c811748 P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
W20260925 13:57:30.284763 21435 catalog_manager.cc:5671] Ignoring report from unknown tablet 71ff569d68154b61bcc14937b2e37fe5
I20260925 13:57:30.287655 22998 tablet_bootstrap.cc:654] T 1d47ea37ee1e4abfbc0f47001c811748 P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
W20260925 13:57:30.290457 21435 catalog_manager.cc:5671] Ignoring report from unknown tablet ccefd36f4e2a48f9b3d66cb147a022b5
W20260925 13:57:30.290755 21435 catalog_manager.cc:5671] Ignoring report from unknown tablet d267def3f0864435a0d2c40fd7052eab
I20260925 13:57:30.313194 22998 tablet_bootstrap.cc:492] T 1d47ea37ee1e4abfbc0f47001c811748 P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:30.313730 22998 ts_tablet_manager.cc:1403] T 1d47ea37ee1e4abfbc0f47001c811748 P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.084s user 0.005s sys 0.006s
I20260925 13:57:30.318977 22997 tablet_bootstrap.cc:492] T 1d47ea37ee1e4abfbc0f47001c811748 P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:30.319518 22997 ts_tablet_manager.cc:1403] T 1d47ea37ee1e4abfbc0f47001c811748 P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.098s user 0.005s sys 0.008s
I20260925 13:57:30.316773 22998 raft_consensus.cc:359] T 1d47ea37ee1e4abfbc0f47001c811748 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:30.322952 22998 raft_consensus.cc:385] T 1d47ea37ee1e4abfbc0f47001c811748 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:30.329229 22997 raft_consensus.cc:359] T 1d47ea37ee1e4abfbc0f47001c811748 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:30.328362 22998 raft_consensus.cc:740] T 1d47ea37ee1e4abfbc0f47001c811748 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:30.332849 22996 tablet_bootstrap.cc:654] T 1d47ea37ee1e4abfbc0f47001c811748 P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:30.348953 22998 consensus_queue.cc:260] T 1d47ea37ee1e4abfbc0f47001c811748 P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:30.354522 22997 raft_consensus.cc:385] T 1d47ea37ee1e4abfbc0f47001c811748 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:30.356274 22997 raft_consensus.cc:740] T 1d47ea37ee1e4abfbc0f47001c811748 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:30.362619 22009 heartbeater.cc:499] Master 127.18.182.254:43163 was elected leader, sending a full tablet report...
W20260925 13:57:30.384765 21434 catalog_manager.cc:5671] Ignoring report from unknown tablet 60256642160a44a2bab77b97d78c27a8
W20260925 13:57:30.385123 21434 catalog_manager.cc:5671] Ignoring report from unknown tablet d267def3f0864435a0d2c40fd7052eab
I20260925 13:57:30.386783 22998 ts_tablet_manager.cc:1434] T 1d47ea37ee1e4abfbc0f47001c811748 P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.073s user 0.005s sys 0.000s
I20260925 13:57:30.388164 22997 consensus_queue.cc:260] T 1d47ea37ee1e4abfbc0f47001c811748 P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:30.392709 22997 ts_tablet_manager.cc:1434] T 1d47ea37ee1e4abfbc0f47001c811748 P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.073s user 0.005s sys 0.000s
I20260925 13:57:30.421432 22996 tablet_bootstrap.cc:492] T 1d47ea37ee1e4abfbc0f47001c811748 P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:30.421958 22996 ts_tablet_manager.cc:1403] T 1d47ea37ee1e4abfbc0f47001c811748 P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.142s user 0.012s sys 0.004s
I20260925 13:57:30.424775 22996 raft_consensus.cc:359] T 1d47ea37ee1e4abfbc0f47001c811748 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:30.425518 22996 raft_consensus.cc:385] T 1d47ea37ee1e4abfbc0f47001c811748 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:30.425882 22996 raft_consensus.cc:740] T 1d47ea37ee1e4abfbc0f47001c811748 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:30.426651 22996 consensus_queue.cc:260] T 1d47ea37ee1e4abfbc0f47001c811748 P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:30.476397 21848 heartbeater.cc:499] Master 127.18.182.254:43163 was elected leader, sending a full tablet report...
I20260925 13:57:30.483292 22642 catalog_manager.cc:2213] created HMS catalog entry for table [id=c77d37849e4340aa82ba49866f99e7c2]
W20260925 13:57:30.485462 22642 catalog_manager.cc:2241] Illegal state: an error occurred while writing to the sys-catalog: replica 3ea1e11bc6d84114b7616815b6af4635 is not leader of this config: current role FOLLOWER
I20260925 13:57:30.486418 22996 ts_tablet_manager.cc:1434] T 1d47ea37ee1e4abfbc0f47001c811748 P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.064s user 0.003s sys 0.003s
W20260925 13:57:30.488910 21435 catalog_manager.cc:5671] Ignoring report from unknown tablet 60256642160a44a2bab77b97d78c27a8
I20260925 13:57:30.569846 23006 raft_consensus.cc:493] T 1d47ea37ee1e4abfbc0f47001c811748 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:30.570626 23006 raft_consensus.cc:515] T 1d47ea37ee1e4abfbc0f47001c811748 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:30.574210 21958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d47ea37ee1e4abfbc0f47001c811748" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:30.574816 21958 raft_consensus.cc:2468] T 1d47ea37ee1e4abfbc0f47001c811748 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:30.576860 21732 leader_election.cc:304] T 1d47ea37ee1e4abfbc0f47001c811748 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:30.577932 23007 raft_consensus.cc:2804] T 1d47ea37ee1e4abfbc0f47001c811748 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:30.578248 23007 raft_consensus.cc:493] T 1d47ea37ee1e4abfbc0f47001c811748 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:30.578500 23007 raft_consensus.cc:3060] T 1d47ea37ee1e4abfbc0f47001c811748 P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:30.588927 23006 leader_election.cc:290] T 1d47ea37ee1e4abfbc0f47001c811748 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:30.590312 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d47ea37ee1e4abfbc0f47001c811748" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:30.590826 22107 raft_consensus.cc:2468] T 1d47ea37ee1e4abfbc0f47001c811748 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
W20260925 13:57:30.607908 21435 catalog_manager.cc:5671] Ignoring report from unknown tablet f5bd96a2bdb842e9b3bc1f706786d6b1
W20260925 13:57:30.609853 21435 catalog_manager.cc:5671] Ignoring report from unknown tablet 60256642160a44a2bab77b97d78c27a8
W20260925 13:57:30.610138 21435 catalog_manager.cc:5671] Ignoring report from unknown tablet d267def3f0864435a0d2c40fd7052eab
I20260925 13:57:30.598071 23007 raft_consensus.cc:515] T 1d47ea37ee1e4abfbc0f47001c811748 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:30.623657 23007 leader_election.cc:290] T 1d47ea37ee1e4abfbc0f47001c811748 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Requested vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:30.624280 21958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d47ea37ee1e4abfbc0f47001c811748" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
I20260925 13:57:30.624811 21958 raft_consensus.cc:3060] T 1d47ea37ee1e4abfbc0f47001c811748 P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:30.629285 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d47ea37ee1e4abfbc0f47001c811748" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:30.630082 22107 raft_consensus.cc:3060] T 1d47ea37ee1e4abfbc0f47001c811748 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:30.646389 22107 raft_consensus.cc:2468] T 1d47ea37ee1e4abfbc0f47001c811748 P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:30.653436 21732 leader_election.cc:304] T 1d47ea37ee1e4abfbc0f47001c811748 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:30.648857 21958 raft_consensus.cc:2468] T 1d47ea37ee1e4abfbc0f47001c811748 P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:30.671043 23007 raft_consensus.cc:2804] T 1d47ea37ee1e4abfbc0f47001c811748 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:30.671875 23007 raft_consensus.cc:697] T 1d47ea37ee1e4abfbc0f47001c811748 P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Becoming Leader. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Running, Role: LEADER
I20260925 13:57:30.674930 23007 consensus_queue.cc:237] T 1d47ea37ee1e4abfbc0f47001c811748 P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:30.741873 21435 catalog_manager.cc:5838] T 1d47ea37ee1e4abfbc0f47001c811748 P 5e2731975dbf433aade4a859df9699e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e2731975dbf433aade4a859df9699e2 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "5e2731975dbf433aade4a859df9699e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: UNKNOWN } } }
I20260925 13:57:30.754411 22641 catalog_manager.cc:2213] created HMS catalog entry for table [id=888e76bafce3489db762f38ccf9e13eb]
W20260925 13:57:30.756400 22641 catalog_manager.cc:2241] Illegal state: an error occurred while writing to the sys-catalog: replica 3ea1e11bc6d84114b7616815b6af4635 is not leader of this config: current role FOLLOWER
W20260925 13:57:30.767412 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:57:30.814625 21704 catalog_manager.cc:2811] Deleting table default.table_05ac73cb4d774c0fbd89f99093663e76 [id=df6afcfb1e2c463493b14228e19ade26] in response to Hive Metastore notification log event 42
I20260925 13:57:30.901000 21704 catalog_manager.cc:6133] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0: Sending DeleteTablet for 3 replicas of tablet 56a39663479d40be8bbd1dfd4c91040c
I20260925 13:57:30.903244 21775 tablet_service.cc:1558] Processing DeleteTablet for tablet 56a39663479d40be8bbd1dfd4c91040c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:57:30 UTC) from {username='slave'} at 127.0.0.1:46710
I20260925 13:57:30.905993 21938 tablet_service.cc:1558] Processing DeleteTablet for tablet 56a39663479d40be8bbd1dfd4c91040c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:57:30 UTC) from {username='slave'} at 127.0.0.1:44958
I20260925 13:57:30.907337 22084 tablet_service.cc:1558] Processing DeleteTablet for tablet 56a39663479d40be8bbd1dfd4c91040c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:57:30 UTC) from {username='slave'} at 127.0.0.1:41660
I20260925 13:57:30.921239 22646 catalog_manager.cc:2213] created HMS catalog entry for table [id=fcde453f78884574a9f2aeba9492ae02]
W20260925 13:57:30.943858 22646 catalog_manager.cc:2241] Illegal state: an error occurred while writing to the sys-catalog: replica 3ea1e11bc6d84114b7616815b6af4635 is not leader of this config: current role FOLLOWER
I20260925 13:57:30.948029 22766 catalog_manager.cc:2811] Deleting table default.table_05ac73cb4d774c0fbd89f99093663e76 [id=df6afcfb1e2c463493b14228e19ade26] in response to Hive Metastore notification log event 42
I20260925 13:57:30.957367 23021 tablet_replica.cc:333] T 56a39663479d40be8bbd1dfd4c91040c P 145e520e0a5a445d943a53cf8c27efee: stopping tablet replica
I20260925 13:57:30.958442 23021 raft_consensus.cc:2243] T 56a39663479d40be8bbd1dfd4c91040c P 145e520e0a5a445d943a53cf8c27efee [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:57:30.959440 23021 raft_consensus.cc:2272] T 56a39663479d40be8bbd1dfd4c91040c P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:30.960683 23020 tablet_replica.cc:333] T 56a39663479d40be8bbd1dfd4c91040c P 5e2731975dbf433aade4a859df9699e2: stopping tablet replica
I20260925 13:57:30.961665 23020 raft_consensus.cc:2243] T 56a39663479d40be8bbd1dfd4c91040c P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:30.962464 23020 raft_consensus.cc:2272] T 56a39663479d40be8bbd1dfd4c91040c P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 13:57:30.963003 22766 catalog_manager.cc:2887] Illegal state: an error occurred while updating the sys-catalog: replica 3ea1e11bc6d84114b7616815b6af4635 is not leader of this config: current role FOLLOWER
W20260925 13:57:30.963562 22766 hms_notification_log_listener.cc:330] Failed to handle Hive Metastore notification: 42 DROP_TABLE default.table_05ac73cb4d774c0fbd89f99093663e76: Illegal state: an error occurred while updating the sys-catalog: replica 3ea1e11bc6d84114b7616815b6af4635 is not leader of this config: current role FOLLOWER
W20260925 13:57:30.964046 22766 hms_notification_log_listener.cc:143] Hive Metastore notification log listener poll failed: Service unavailable: lost leadership while handling Hive Metastore notification log events: an error occurred while updating the sys-catalog: replica 3ea1e11bc6d84114b7616815b6af4635 is not leader of this config: current role FOLLOWER
I20260925 13:57:30.964391 23022 tablet_replica.cc:333] T 56a39663479d40be8bbd1dfd4c91040c P ee02f77594864349982492862d668a36: stopping tablet replica
I20260925 13:57:30.965235 23022 raft_consensus.cc:2243] T 56a39663479d40be8bbd1dfd4c91040c P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:57:30.965783 23022 raft_consensus.cc:2272] T 56a39663479d40be8bbd1dfd4c91040c P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:57:30.971894 22645 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:41880 (request call id 35) took 4511 ms. Trace:
I20260925 13:57:30.972275 22645 rpcz_store.cc:276] 0925 13:57:26.460208 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:26.460306 (+ 98us) service_pool.cc:224] Handling call
0925 13:57:26.470208 (+ 9902us) spinlock_profiling.cc:243] Waited 7.8 ms on lock 0x7b54000104d4. stack: 00007f5e41a93323 00007f5e41a93220 00007f5e41637574 0000000000516964 00000000005168f9 00007f5e48b915c0 00007f5e48c306fa 00007f5e4a9a1d8c 00007f5e4aaa1876 00007f5e48650b34 00007f5e48650ac2 00007f5e486509f1 00007f5e4865096c 00007f5e4864fc22 00007f5e468f609c 00007f5e468f54d6
0925 13:57:27.706524 (+1236316us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 13:57:30.971681 (+3265157us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":146,"Hive Metastore.run_cpu_time_us":429,"Hive Metastore.run_wall_time_us":480058,"spinlock_wait_cycles":17932672}
W20260925 13:57:30.976984 22163 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.18.182.253:46177)
I20260925 13:57:30.992041 23021 ts_tablet_manager.cc:1916] T 56a39663479d40be8bbd1dfd4c91040c P 145e520e0a5a445d943a53cf8c27efee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:30.994585 22647 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:41880 (request call id 33) took 4537 ms. Trace:
I20260925 13:57:30.995199 23020 ts_tablet_manager.cc:1916] T 56a39663479d40be8bbd1dfd4c91040c P 5e2731975dbf433aade4a859df9699e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:30.995015 22647 rpcz_store.cc:276] 0925 13:57:26.456804 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:26.456894 (+ 90us) service_pool.cc:224] Handling call
0925 13:57:26.482043 (+ 25149us) spinlock_profiling.cc:243] Waited 23.4 ms on lock 0x7b54000104d4. stack: 00007f5e41a93323 00007f5e41a93220 00007f5e41637574 0000000000516964 00000000005168f9 00007f5e48b915c0 00007f5e48c306fa 00007f5e4a9a1d8c 00007f5e4aaa1be6 00007f5e48655634 00007f5e486555c2 00007f5e486554f1 00007f5e4865546c 00007f5e48654722 00007f5e468f609c 00007f5e468f54d6
0925 13:57:27.751550 (+1269507us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 13:57:30.987997 (+3236447us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 13:57:30.993372 (+ 5375us) catalog_manager.cc:3764] Apply alter schema
0925 13:57:30.994370 (+ 998us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":862140,"Hive Metastore.run_cpu_time_us":1889,"Hive Metastore.run_wall_time_us":541645,"spinlock_wait_cycles":53929728}
I20260925 13:57:31.000226 22648 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:41880 (request call id 36) took 4503 ms. Trace:
I20260925 13:57:31.001118 23022 ts_tablet_manager.cc:1916] T 56a39663479d40be8bbd1dfd4c91040c P ee02f77594864349982492862d668a36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:57:31.000600 22648 rpcz_store.cc:276] 0925 13:57:26.496426 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:26.496538 (+ 112us) service_pool.cc:224] Handling call
0925 13:57:27.752604 (+1256066us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 13:57:30.998189 (+3245585us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0925 13:57:30.999174 (+ 985us) catalog_manager.cc:3764] Apply alter schema
0925 13:57:31.000070 (+ 896us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":1402855,"Hive Metastore.run_cpu_time_us":1726,"Hive Metastore.run_wall_time_us":848857}
I20260925 13:57:31.019942 21435 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37566:
table { table_name: "default.Table_05ac73cb4d774c0fbd89f99093663e76" } modify_external_catalogs: true
I20260925 13:57:31.020681 21435 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37566:
table { table_name: "default.Table_05ac73cb4d774c0fbd89f99093663e76" } modify_external_catalogs: true
I20260925 13:57:31.186882 23021 ts_tablet_manager.cc:1929] T 56a39663479d40be8bbd1dfd4c91040c P 145e520e0a5a445d943a53cf8c27efee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:57:31.189949 23022 ts_tablet_manager.cc:1929] T 56a39663479d40be8bbd1dfd4c91040c P ee02f77594864349982492862d668a36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:57:31.190469 23022 log.cc:1199] T 56a39663479d40be8bbd1dfd4c91040c P ee02f77594864349982492862d668a36: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-2/wal/wals/56a39663479d40be8bbd1dfd4c91040c
I20260925 13:57:31.191464 23022 ts_tablet_manager.cc:1950] T 56a39663479d40be8bbd1dfd4c91040c P ee02f77594864349982492862d668a36: Deleting consensus metadata
I20260925 13:57:31.191783 23021 log.cc:1199] T 56a39663479d40be8bbd1dfd4c91040c P 145e520e0a5a445d943a53cf8c27efee: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-1/wal/wals/56a39663479d40be8bbd1dfd4c91040c
I20260925 13:57:31.193204 23021 ts_tablet_manager.cc:1950] T 56a39663479d40be8bbd1dfd4c91040c P 145e520e0a5a445d943a53cf8c27efee: Deleting consensus metadata
I20260925 13:57:31.194404 23020 ts_tablet_manager.cc:1929] T 56a39663479d40be8bbd1dfd4c91040c P 5e2731975dbf433aade4a859df9699e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:57:31.213802 23020 log.cc:1199] T 56a39663479d40be8bbd1dfd4c91040c P 5e2731975dbf433aade4a859df9699e2: Deleting WAL directory at /tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/ts-0/wal/wals/56a39663479d40be8bbd1dfd4c91040c
I20260925 13:57:31.215054 23020 ts_tablet_manager.cc:1950] T 56a39663479d40be8bbd1dfd4c91040c P 5e2731975dbf433aade4a859df9699e2: Deleting consensus metadata
I20260925 13:57:31.272195 23043 tablet.cc:1751] T ef57aa72abb2405b8187a7ed56d57f96 P ee02f77594864349982492862d668a36: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:57:31.274322 23042 tablet.cc:1751] T a06a5aac5d464d339c870df4ebd63d92 P ee02f77594864349982492862d668a36: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260925 13:57:31.278635 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:57:31.288287 23044 tablet.cc:1751] T a06a5aac5d464d339c870df4ebd63d92 P 5e2731975dbf433aade4a859df9699e2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:57:31.324664 23045 tablet.cc:1751] T ef57aa72abb2405b8187a7ed56d57f96 P 145e520e0a5a445d943a53cf8c27efee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:57:31.312994 23046 tablet.cc:1751] T a06a5aac5d464d339c870df4ebd63d92 P 145e520e0a5a445d943a53cf8c27efee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:57:31.354638 21435 catalog_manager.cc:6391] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0: default.table_b32d230a905d44b388dc9e436965cbc9 [id=bf8cef2239904fd989622f9426412635] alter complete (version 1)
I20260925 13:57:31.365640 23047 tablet.cc:1751] T ef57aa72abb2405b8187a7ed56d57f96 P 5e2731975dbf433aade4a859df9699e2: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 13:57:31.377895 21435 catalog_manager.cc:6391] T 00000000000000000000000000000000 P e942d7f2e4864b5ca08e6e830085caa0: default.table_2a09482ff526433a8d028a964cfdfb0c [id=bb79c1d2d0094785b35250fb1b718b9a] alter complete (version 1)
W20260925 13:57:31.385581 21435 catalog_manager.cc:5671] Ignoring report from unknown tablet 60256642160a44a2bab77b97d78c27a8
I20260925 13:57:31.393540 22642 catalog_manager.cc:2223] deleted orphaned HMS catalog entry for table [id=c77d37849e4340aa82ba49866f99e7c2]
I20260925 13:57:31.394675 22642 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:41880 (request call id 38) took 4894 ms. Trace:
I20260925 13:57:31.394977 22642 rpcz_store.cc:276] 0925 13:57:26.500600 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:26.500742 (+ 142us) service_pool.cc:224] Handling call
0925 13:57:27.806482 (+1305740us) catalog_manager.cc:2140] Acquired catalog manager lock
0925 13:57:27.808414 (+ 1932us) catalog_manager.cc:2186] Created new table and tablet info
0925 13:57:30.483253 (+2674839us) catalog_manager.cc:2212] Created new table in HMS catalog
0925 13:57:30.485197 (+ 1944us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:30.485228 (+ 31us) write_op.cc:454] Released schema lock
0925 13:57:30.485758 (+ 530us) catalog_manager.cc:2220] Rolling back HMS table creation
0925 13:57:31.394574 (+908816us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:57:30.484974 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica 3ea1e11bc6d84114b7616815b6af4635 is not leader of this config: current role FOLLOWER)
0925 13:57:30.485019 (+ 45us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:30.485043 (+ 24us) write_op.cc:454] Released schema lock
0925 13:57:30.485066 (+ 23us) write_op.cc:335] FINISH: Op aborted
Metrics: {"Hive Metastore.queue_time_us":2651374,"Hive Metastore.run_cpu_time_us":1067,"Hive Metastore.run_wall_time_us":929791,"child_traces":[["op",{}]]}
W20260925 13:57:31.399529 22163 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.18.182.254:43163)
I20260925 13:57:31.408789 21435 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37566:
name: "default.Table_8791c6009be44efab81f5f2252a3d8ae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:57:31.420380 21704 catalog_manager.cc:2811] Deleting table default.table_8791c6009be44efab81f5f2252a3d8ae [id=c77d37849e4340aa82ba49866f99e7c2] in response to Hive Metastore notification log event 49
W20260925 13:57:31.421267 21704 hms_notification_log_listener.cc:330] Failed to handle Hive Metastore notification: 49 DROP_TABLE default.table_8791c6009be44efab81f5f2252a3d8ae: Not found: the table does not exist: table_id: "c77d37849e4340aa82ba49866f99e7c2" table_name: "default.table_8791c6009be44efab81f5f2252a3d8ae"
W20260925 13:57:31.438292 21435 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_8791c6009be44efab81f5f2252a3d8ae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:57:31.549726 21435 catalog_manager.cc:2213] created HMS catalog entry for table [id=038891ef1fdb4cee98081f6d56bb1c58]
I20260925 13:57:31.569235 23052 consensus_queue.cc:1048] T 1d47ea37ee1e4abfbc0f47001c811748 P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 13:57:31.598956 22641 catalog_manager.cc:2223] deleted orphaned HMS catalog entry for table [id=888e76bafce3489db762f38ccf9e13eb]
I20260925 13:57:31.599933 22641 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:41880 (request call id 39) took 5082 ms. Trace:
I20260925 13:57:31.600246 22641 rpcz_store.cc:276] 0925 13:57:26.517210 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:26.517311 (+ 101us) service_pool.cc:224] Handling call
0925 13:57:27.808976 (+1291665us) catalog_manager.cc:2140] Acquired catalog manager lock
0925 13:57:27.810741 (+ 1765us) catalog_manager.cc:2186] Created new table and tablet info
0925 13:57:30.754363 (+2943622us) catalog_manager.cc:2212] Created new table in HMS catalog
0925 13:57:30.756177 (+ 1814us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:30.756203 (+ 26us) write_op.cc:454] Released schema lock
0925 13:57:30.756689 (+ 486us) catalog_manager.cc:2220] Rolling back HMS table creation
0925 13:57:31.599862 (+843173us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:57:30.755985 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica 3ea1e11bc6d84114b7616815b6af4635 is not leader of this config: current role FOLLOWER)
0925 13:57:30.756020 (+ 35us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:30.756045 (+ 25us) write_op.cc:454] Released schema lock
0925 13:57:30.756069 (+ 24us) write_op.cc:335] FINISH: Op aborted
Metrics: {"Hive Metastore.queue_time_us":3308158,"Hive Metastore.run_cpu_time_us":966,"Hive Metastore.run_wall_time_us":476090,"child_traces":[["op",{}]]}
I20260925 13:57:31.613176 21434 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37566:
name: "default.Table_4586487550ec46f08eae599e1c34ef17"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:57:31.621392 21938 tablet_service.cc:1511] Processing CreateTablet for tablet 5dfb93f602fe447baf4972a39c54394f (DEFAULT_TABLE table=default.table_8791c6009be44efab81f5f2252a3d8ae [id=038891ef1fdb4cee98081f6d56bb1c58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:31.622393 21938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dfb93f602fe447baf4972a39c54394f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:31.624168 21775 tablet_service.cc:1511] Processing CreateTablet for tablet 5dfb93f602fe447baf4972a39c54394f (DEFAULT_TABLE table=default.table_8791c6009be44efab81f5f2252a3d8ae [id=038891ef1fdb4cee98081f6d56bb1c58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:31.625397 21775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dfb93f602fe447baf4972a39c54394f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:31.630167 22084 tablet_service.cc:1511] Processing CreateTablet for tablet 5dfb93f602fe447baf4972a39c54394f (DEFAULT_TABLE table=default.table_8791c6009be44efab81f5f2252a3d8ae [id=038891ef1fdb4cee98081f6d56bb1c58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:31.641010 21704 catalog_manager.cc:2811] Deleting table default.table_4586487550ec46f08eae599e1c34ef17 [id=888e76bafce3489db762f38ccf9e13eb] in response to Hive Metastore notification log event 51
I20260925 13:57:31.631217 22084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dfb93f602fe447baf4972a39c54394f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 13:57:31.660698 21704 hms_notification_log_listener.cc:330] Failed to handle Hive Metastore notification: 51 DROP_TABLE default.table_4586487550ec46f08eae599e1c34ef17: Not found: the table does not exist: table_id: "888e76bafce3489db762f38ccf9e13eb" table_name: "default.table_4586487550ec46f08eae599e1c34ef17"
I20260925 13:57:31.681352 23051 consensus_queue.cc:1048] T 1d47ea37ee1e4abfbc0f47001c811748 P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260925 13:57:31.719990 21434 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_4586487550ec46f08eae599e1c34ef17 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:57:31.759604 23056 tablet_bootstrap.cc:492] T 5dfb93f602fe447baf4972a39c54394f P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:31.760371 23057 tablet_bootstrap.cc:492] T 5dfb93f602fe447baf4972a39c54394f P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
W20260925 13:57:31.777212 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:57:31.782389 23056 tablet_bootstrap.cc:654] T 5dfb93f602fe447baf4972a39c54394f P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:31.787943 23057 tablet_bootstrap.cc:654] T 5dfb93f602fe447baf4972a39c54394f P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:31.795083 21434 catalog_manager.cc:2213] created HMS catalog entry for table [id=ad8f47ab1edd4ae1a1141afe4f07625a]
I20260925 13:57:31.793993 23056 tablet_bootstrap.cc:492] T 5dfb93f602fe447baf4972a39c54394f P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:31.796741 23056 ts_tablet_manager.cc:1403] T 5dfb93f602fe447baf4972a39c54394f P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.037s user 0.008s sys 0.003s
I20260925 13:57:31.804567 23056 raft_consensus.cc:359] T 5dfb93f602fe447baf4972a39c54394f P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:31.808384 23057 tablet_bootstrap.cc:492] T 5dfb93f602fe447baf4972a39c54394f P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:31.808826 23057 ts_tablet_manager.cc:1403] T 5dfb93f602fe447baf4972a39c54394f P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.052s user 0.007s sys 0.005s
I20260925 13:57:31.811524 22646 catalog_manager.cc:2223] deleted orphaned HMS catalog entry for table [id=fcde453f78884574a9f2aeba9492ae02]
I20260925 13:57:31.811915 23056 raft_consensus.cc:385] T 5dfb93f602fe447baf4972a39c54394f P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:31.812211 23056 raft_consensus.cc:740] T 5dfb93f602fe447baf4972a39c54394f P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:31.812304 22646 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:41880 (request call id 34) took 5354 ms. Trace:
I20260925 13:57:31.813388 23057 raft_consensus.cc:359] T 5dfb93f602fe447baf4972a39c54394f P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:31.812592 22646 rpcz_store.cc:276] 0925 13:57:26.458351 (+ 0us) service_pool.cc:167] Inserting onto call queue
0925 13:57:26.458454 (+ 103us) service_pool.cc:224] Handling call
0925 13:57:26.471940 (+ 13486us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b54000104d4. stack: 00007f5e41a93323 00007f5e41a93220 00007f5e41637574 0000000000516964 00000000005168f9 00007f5e48b915c0 00007f5e48c306fa 00007f5e4a9a1d8c 00007f5e4aaa1566 00007f5e4864c034 00007f5e4864bfc2 00007f5e4864bef1 00007f5e4864be6c 00007f5e4864b122 00007f5e468f609c 00007f5e468f54d6
0925 13:57:27.811243 (+1339303us) catalog_manager.cc:2140] Acquired catalog manager lock
0925 13:57:27.812976 (+ 1733us) catalog_manager.cc:2186] Created new table and tablet info
0925 13:57:30.920992 (+3108016us) catalog_manager.cc:2212] Created new table in HMS catalog
0925 13:57:30.943557 (+ 22565us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:30.943583 (+ 26us) write_op.cc:454] Released schema lock
0925 13:57:30.944371 (+ 788us) catalog_manager.cc:2220] Rolling back HMS table creation
0925 13:57:31.812248 (+867877us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0925 13:57:30.943247 (+ 0us) op_driver.cc:389] HandleFailure(Illegal state: replica 3ea1e11bc6d84114b7616815b6af4635 is not leader of this config: current role FOLLOWER)
0925 13:57:30.943300 (+ 53us) write_op.cc:489] Releasing partition, row and schema locks
0925 13:57:30.943338 (+ 38us) write_op.cc:454] Released schema lock
0925 13:57:30.943400 (+ 62us) write_op.cc:335] FINISH: Op aborted
Metrics: {"Hive Metastore.queue_time_us":3594482,"Hive Metastore.run_cpu_time_us":1223,"Hive Metastore.run_wall_time_us":379305,"spinlock_wait_cycles":27504000,"child_traces":[["op",{}]]}
I20260925 13:57:31.813993 23057 raft_consensus.cc:385] T 5dfb93f602fe447baf4972a39c54394f P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:31.814244 23057 raft_consensus.cc:740] T 5dfb93f602fe447baf4972a39c54394f P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:31.814872 23057 consensus_queue.cc:260] T 5dfb93f602fe447baf4972a39c54394f P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:31.817183 23056 consensus_queue.cc:260] T 5dfb93f602fe447baf4972a39c54394f P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:31.819248 23056 ts_tablet_manager.cc:1434] T 5dfb93f602fe447baf4972a39c54394f P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.022s user 0.004s sys 0.001s
I20260925 13:57:31.826145 23057 ts_tablet_manager.cc:1434] T 5dfb93f602fe447baf4972a39c54394f P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.017s user 0.006s sys 0.000s
I20260925 13:57:31.832315 21432 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37566:
name: "default.Table_8229f26c3e104bf1885bce04e09f8e29"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:57:31.855546 19163 master-stress-test.cc:581] Killed test thread 2/10
I20260925 13:57:31.855810 19163 master-stress-test.cc:579] Killing test thread 3/10
F20260925 13:57:31.858423 21704 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 52, previous 52; dumping first 2 out of 2 received events: NotificationEvent(eventId=52, eventTime=1790344651, eventType=DROP_TABLE, dbName=default, tableName=table_8229f26c3e104bf1885bce04e09f8e29, message={"server":"","servicePrincipal":"","db":"default","table":"table_8229f26c3e104bf1885bce04e09f8e29","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_8229f26c3e104bf1885bce04e09f8e29\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790344650},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/warehouse/table_8229f26c3e104bf1885bce04e09f8e29\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_8229f26c3e104bf1885bce04e09f8e29\",\"kudu.table_id\":\"fcde453f78884574a9f2aeba9492ae02\",\"transient_lastDdlTime\":\"1790344650\",\"comment\":\"\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.master_addresses\":\"127.18.182.254:43163,127.18.182.253:46177,127.18.182.252:39283\",\"kudu.cluster_id\":\"d5d0bc5567a74d9cad4d92a9d1041fd9\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790344651}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=52, eventTime=1790344651, eventType=CREATE_TABLE, dbName=default, tableName=table_4586487550ec46f08eae599e1c34ef17, message={"server":"","servicePrincipal":"","db":"default","table":"table_4586487550ec46f08eae599e1c34ef17","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_4586487550ec46f08eae599e1c34ef17\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790344651},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/warehouse/table_4586487550ec46f08eae599e1c34ef17\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_4586487550ec46f08eae599e1c34ef17\",\"kudu.table_id\":\"ad8f47ab1edd4ae1a1141afe4f07625a\",\"transient_lastDdlTime\":\"1790344651\",\"comment\":\"\",\"kudu.master_addresses\":\"127.18.182.254:43163,127.18.182.253:46177,127.18.182.252:39283\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"d5d0bc5567a74d9cad4d92a9d1041fd9\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790344651,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
@ 0x7fd4d80bf920 google::LogMessage::Flush() at ??:0
@ 0x7fd4d80bf68f google::LogMessage::~LogMessage() at ??:0
I20260925 13:57:31.938581 23061 tablet_bootstrap.cc:492] T 5dfb93f602fe447baf4972a39c54394f P ee02f77594864349982492862d668a36: Bootstrap starting.
@ 0x7fd4e325235c kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7fd4e3251b1c kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7fd4e3255b92 kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260925 13:57:32.019402 21938 tablet_service.cc:1511] Processing CreateTablet for tablet 4fccfe2595134d1ba43f742a771ce4fd (DEFAULT_TABLE table=default.table_4586487550ec46f08eae599e1c34ef17 [id=ad8f47ab1edd4ae1a1141afe4f07625a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:32.020413 21938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fccfe2595134d1ba43f742a771ce4fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:32.021478 22084 tablet_service.cc:1511] Processing CreateTablet for tablet 4fccfe2595134d1ba43f742a771ce4fd (DEFAULT_TABLE table=default.table_4586487550ec46f08eae599e1c34ef17 [id=ad8f47ab1edd4ae1a1141afe4f07625a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:32.022554 22084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fccfe2595134d1ba43f742a771ce4fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:32.033855 23056 tablet_bootstrap.cc:492] T 4fccfe2595134d1ba43f742a771ce4fd P 145e520e0a5a445d943a53cf8c27efee: Bootstrap starting.
I20260925 13:57:32.039341 23056 tablet_bootstrap.cc:654] T 4fccfe2595134d1ba43f742a771ce4fd P 145e520e0a5a445d943a53cf8c27efee: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:32.046510 21775 tablet_service.cc:1511] Processing CreateTablet for tablet 4fccfe2595134d1ba43f742a771ce4fd (DEFAULT_TABLE table=default.table_4586487550ec46f08eae599e1c34ef17 [id=ad8f47ab1edd4ae1a1141afe4f07625a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:57:32.047474 21775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fccfe2595134d1ba43f742a771ce4fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:57:32.050261 23061 tablet_bootstrap.cc:654] T 5dfb93f602fe447baf4972a39c54394f P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:32.065481 23057 tablet_bootstrap.cc:492] T 4fccfe2595134d1ba43f742a771ce4fd P 5e2731975dbf433aade4a859df9699e2: Bootstrap starting.
I20260925 13:57:32.066082 23056 tablet_bootstrap.cc:492] T 4fccfe2595134d1ba43f742a771ce4fd P 145e520e0a5a445d943a53cf8c27efee: No bootstrap required, opened a new log
I20260925 13:57:32.066502 23056 ts_tablet_manager.cc:1403] T 4fccfe2595134d1ba43f742a771ce4fd P 145e520e0a5a445d943a53cf8c27efee: Time spent bootstrapping tablet: real 0.033s user 0.001s sys 0.011s
I20260925 13:57:32.068960 23056 raft_consensus.cc:359] T 4fccfe2595134d1ba43f742a771ce4fd P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } }
I20260925 13:57:32.069571 23056 raft_consensus.cc:385] T 4fccfe2595134d1ba43f742a771ce4fd P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:32.069794 23056 raft_consensus.cc:740] T 4fccfe2595134d1ba43f742a771ce4fd P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145e520e0a5a445d943a53cf8c27efee, State: Initialized, Role: FOLLOWER
I20260925 13:57:32.070600 23056 consensus_queue.cc:260] T 4fccfe2595134d1ba43f742a771ce4fd P 145e520e0a5a445d943a53cf8c27efee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } }
I20260925 13:57:32.071947 23057 tablet_bootstrap.cc:654] T 4fccfe2595134d1ba43f742a771ce4fd P 5e2731975dbf433aade4a859df9699e2: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:32.078969 23056 ts_tablet_manager.cc:1434] T 4fccfe2595134d1ba43f742a771ce4fd P 145e520e0a5a445d943a53cf8c27efee: Time spent starting tablet: real 0.012s user 0.005s sys 0.001s
I20260925 13:57:32.080917 23057 tablet_bootstrap.cc:492] T 4fccfe2595134d1ba43f742a771ce4fd P 5e2731975dbf433aade4a859df9699e2: No bootstrap required, opened a new log
I20260925 13:57:32.081415 23057 ts_tablet_manager.cc:1403] T 4fccfe2595134d1ba43f742a771ce4fd P 5e2731975dbf433aade4a859df9699e2: Time spent bootstrapping tablet: real 0.016s user 0.011s sys 0.000s
I20260925 13:57:32.084208 23057 raft_consensus.cc:359] T 4fccfe2595134d1ba43f742a771ce4fd P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } }
I20260925 13:57:32.087503 23057 raft_consensus.cc:385] T 4fccfe2595134d1ba43f742a771ce4fd P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:32.087819 23057 raft_consensus.cc:740] T 4fccfe2595134d1ba43f742a771ce4fd P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Initialized, Role: FOLLOWER
I20260925 13:57:32.088622 23057 consensus_queue.cc:260] T 4fccfe2595134d1ba43f742a771ce4fd P 5e2731975dbf433aade4a859df9699e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } }
I20260925 13:57:32.108630 23061 tablet_bootstrap.cc:492] T 5dfb93f602fe447baf4972a39c54394f P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
I20260925 13:57:32.109112 23061 ts_tablet_manager.cc:1403] T 5dfb93f602fe447baf4972a39c54394f P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.172s user 0.005s sys 0.008s
I20260925 13:57:32.112741 23057 ts_tablet_manager.cc:1434] T 4fccfe2595134d1ba43f742a771ce4fd P 5e2731975dbf433aade4a859df9699e2: Time spent starting tablet: real 0.031s user 0.005s sys 0.000s
@ 0x7fd4e3255b4a _ZNSt3__18__invokeIRZN4kudu6master30HmsNotificationLogListenerTask4InitEvE3$_0JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOS6_DpOS7_ at ??:0
I20260925 13:57:32.114773 23061 raft_consensus.cc:359] T 5dfb93f602fe447baf4972a39c54394f P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:32.132002 23061 raft_consensus.cc:385] T 5dfb93f602fe447baf4972a39c54394f P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:32.132572 23061 raft_consensus.cc:740] T 5dfb93f602fe447baf4972a39c54394f P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:32.133875 23061 consensus_queue.cc:260] T 5dfb93f602fe447baf4972a39c54394f P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:32.140933 23051 raft_consensus.cc:493] T 4fccfe2595134d1ba43f742a771ce4fd P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:32.141723 23061 ts_tablet_manager.cc:1434] T 5dfb93f602fe447baf4972a39c54394f P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.032s user 0.007s sys 0.000s
I20260925 13:57:32.141417 23051 raft_consensus.cc:515] T 4fccfe2595134d1ba43f742a771ce4fd P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } }
I20260925 13:57:32.143890 23061 tablet_bootstrap.cc:492] T 4fccfe2595134d1ba43f742a771ce4fd P ee02f77594864349982492862d668a36: Bootstrap starting.
I20260925 13:57:32.144517 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fccfe2595134d1ba43f742a771ce4fd" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:32.145102 21960 raft_consensus.cc:2468] T 4fccfe2595134d1ba43f742a771ce4fd P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:32.146247 21732 leader_election.cc:304] T 4fccfe2595134d1ba43f742a771ce4fd P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:32.147104 23052 raft_consensus.cc:2804] T 4fccfe2595134d1ba43f742a771ce4fd P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:32.147444 23052 raft_consensus.cc:493] T 4fccfe2595134d1ba43f742a771ce4fd P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:32.147818 23052 raft_consensus.cc:3060] T 4fccfe2595134d1ba43f742a771ce4fd P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:32.153906 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fccfe2595134d1ba43f742a771ce4fd" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
W20260925 13:57:32.155527 21732 leader_election.cc:343] T 4fccfe2595134d1ba43f742a771ce4fd P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:57:32.151281 23051 leader_election.cc:290] T 4fccfe2595134d1ba43f742a771ce4fd P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:32.170050 23061 tablet_bootstrap.cc:654] T 4fccfe2595134d1ba43f742a771ce4fd P ee02f77594864349982492862d668a36: Neither blocks nor log segments found. Creating new log.
I20260925 13:57:32.175597 23052 raft_consensus.cc:515] T 4fccfe2595134d1ba43f742a771ce4fd P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } }
I20260925 13:57:32.182281 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fccfe2595134d1ba43f742a771ce4fd" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
I20260925 13:57:32.182807 21960 raft_consensus.cc:3060] T 4fccfe2595134d1ba43f742a771ce4fd P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:32.189105 21960 raft_consensus.cc:2468] T 4fccfe2595134d1ba43f742a771ce4fd P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:32.190095 21732 leader_election.cc:304] T 4fccfe2595134d1ba43f742a771ce4fd P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2; no voters:
I20260925 13:57:32.190899 23051 raft_consensus.cc:2804] T 4fccfe2595134d1ba43f742a771ce4fd P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:32.198136 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fccfe2595134d1ba43f742a771ce4fd" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
W20260925 13:57:32.199420 21732 leader_election.cc:343] T 4fccfe2595134d1ba43f742a771ce4fd P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Illegal state: must be running to vote when last-logged opid is not known
I20260925 13:57:32.200585 23052 leader_election.cc:290] T 4fccfe2595134d1ba43f742a771ce4fd P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Requested vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:32.203267 23051 raft_consensus.cc:697] T 4fccfe2595134d1ba43f742a771ce4fd P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Becoming Leader. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Running, Role: LEADER
I20260925 13:57:32.204190 23051 consensus_queue.cc:237] T 4fccfe2595134d1ba43f742a771ce4fd P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } }
I20260925 13:57:32.214622 23061 tablet_bootstrap.cc:492] T 4fccfe2595134d1ba43f742a771ce4fd P ee02f77594864349982492862d668a36: No bootstrap required, opened a new log
@ 0x7fd4e3255ada std::__1::__invoke_void_return_wrapper<>::__call<>() at ??:0
I20260925 13:57:32.218988 23061 ts_tablet_manager.cc:1403] T 4fccfe2595134d1ba43f742a771ce4fd P ee02f77594864349982492862d668a36: Time spent bootstrapping tablet: real 0.075s user 0.010s sys 0.009s
I20260925 13:57:32.244000 23061 raft_consensus.cc:359] T 4fccfe2595134d1ba43f742a771ce4fd P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } }
I20260925 13:57:32.248634 23061 raft_consensus.cc:385] T 4fccfe2595134d1ba43f742a771ce4fd P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:57:32.249552 21431 catalog_manager.cc:5838] T 4fccfe2595134d1ba43f742a771ce4fd P 5e2731975dbf433aade4a859df9699e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e2731975dbf433aade4a859df9699e2 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "5e2731975dbf433aade4a859df9699e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: HEALTHY } } }
I20260925 13:57:32.250955 23061 raft_consensus.cc:740] T 4fccfe2595134d1ba43f742a771ce4fd P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee02f77594864349982492862d668a36, State: Initialized, Role: FOLLOWER
I20260925 13:57:32.255839 23061 consensus_queue.cc:260] T 4fccfe2595134d1ba43f742a771ce4fd P ee02f77594864349982492862d668a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } }
I20260925 13:57:32.271523 23061 ts_tablet_manager.cc:1434] T 4fccfe2595134d1ba43f742a771ce4fd P ee02f77594864349982492862d668a36: Time spent starting tablet: real 0.050s user 0.005s sys 0.005s
@ 0x7fd4e3255aa2 std::__1::__function::__alloc_func<>::operator()() at ??:0
W20260925 13:57:32.334360 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
@ 0x7fd4e3254d9e std::__1::__function::__func<>::operator()() at ??:0
@ 0x5bd4e5 std::__1::__function::__value_func<>::operator()() at ??:?
W20260925 13:57:32.381210 21896 consensus_peers.cc:597] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee -> Peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Couldn't send request to peer ee02f77594864349982492862d668a36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60256642160a44a2bab77b97d78c27a8. This is attempt 6: this message will repeat every 5th retry.
@ 0x5bd31a std::__1::function<>::operator()() at ??:?
W20260925 13:57:32.395749 21431 catalog_manager.cc:5671] Ignoring report from unknown tablet 60256642160a44a2bab77b97d78c27a8
@ 0x7fd4da2b581b kudu::Thread::SuperviseThread() at ??:0
@ 0x4569a0 __tsan_thread_start_func at ??:?
I20260925 13:57:32.447719 23075 raft_consensus.cc:493] T 5dfb93f602fe447baf4972a39c54394f P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:32.448163 23075 raft_consensus.cc:515] T 5dfb93f602fe447baf4972a39c54394f P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:32.450156 23075 leader_election.cc:290] T 5dfb93f602fe447baf4972a39c54394f P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:32.455320 23052 raft_consensus.cc:493] T 5dfb93f602fe447baf4972a39c54394f P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:32.455799 23052 raft_consensus.cc:515] T 5dfb93f602fe447baf4972a39c54394f P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:32.457757 23052 leader_election.cc:290] T 5dfb93f602fe447baf4972a39c54394f P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:32.457729 22106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dfb93f602fe447baf4972a39c54394f" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:32.458303 22106 raft_consensus.cc:2468] T 5dfb93f602fe447baf4972a39c54394f P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 145e520e0a5a445d943a53cf8c27efee in term 0.
@ 0x7fd4dba166db start_thread at ??:0
I20260925 13:57:32.471678 22106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dfb93f602fe447baf4972a39c54394f" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36" is_pre_election: true
I20260925 13:57:32.472164 22106 raft_consensus.cc:2468] T 5dfb93f602fe447baf4972a39c54394f P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
I20260925 13:57:32.472470 21896 leader_election.cc:304] T 5dfb93f602fe447baf4972a39c54394f P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:32.473793 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dfb93f602fe447baf4972a39c54394f" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:32.474326 21960 raft_consensus.cc:2468] T 5dfb93f602fe447baf4972a39c54394f P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 0.
@ 0x7fd4d6d6161f clone at ??:0
I20260925 13:57:32.477569 21732 leader_election.cc:304] T 5dfb93f602fe447baf4972a39c54394f P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2, ee02f77594864349982492862d668a36; no voters:
I20260925 13:57:32.478200 23071 raft_consensus.cc:2804] T 5dfb93f602fe447baf4972a39c54394f P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:32.478500 23071 raft_consensus.cc:493] T 5dfb93f602fe447baf4972a39c54394f P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:32.478791 23071 raft_consensus.cc:3060] T 5dfb93f602fe447baf4972a39c54394f P 5e2731975dbf433aade4a859df9699e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:32.478799 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dfb93f602fe447baf4972a39c54394f" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2" is_pre_election: true
I20260925 13:57:32.479889 23075 raft_consensus.cc:2804] T 5dfb93f602fe447baf4972a39c54394f P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:57:32.480209 23075 raft_consensus.cc:493] T 5dfb93f602fe447baf4972a39c54394f P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:57:32.480502 23075 raft_consensus.cc:3060] T 5dfb93f602fe447baf4972a39c54394f P 145e520e0a5a445d943a53cf8c27efee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:32.485281 23071 raft_consensus.cc:515] T 5dfb93f602fe447baf4972a39c54394f P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:32.486193 21796 raft_consensus.cc:2393] T 5dfb93f602fe447baf4972a39c54394f P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 145e520e0a5a445d943a53cf8c27efee in current term 1: Already voted for candidate 5e2731975dbf433aade4a859df9699e2 in this term.
I20260925 13:57:32.490360 23075 raft_consensus.cc:515] T 5dfb93f602fe447baf4972a39c54394f P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:32.492201 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dfb93f602fe447baf4972a39c54394f" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee"
I20260925 13:57:32.494374 21960 raft_consensus.cc:2393] T 5dfb93f602fe447baf4972a39c54394f P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e2731975dbf433aade4a859df9699e2 in current term 1: Already voted for candidate 145e520e0a5a445d943a53cf8c27efee in this term.
I20260925 13:57:32.494366 22106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dfb93f602fe447baf4972a39c54394f" candidate_uuid: "5e2731975dbf433aade4a859df9699e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:32.494823 22106 raft_consensus.cc:3060] T 5dfb93f602fe447baf4972a39c54394f P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:32.494791 23075 leader_election.cc:290] T 5dfb93f602fe447baf4972a39c54394f P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Requested vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:32.496905 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dfb93f602fe447baf4972a39c54394f" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2"
I20260925 13:57:32.497473 21796 raft_consensus.cc:2393] T 5dfb93f602fe447baf4972a39c54394f P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 145e520e0a5a445d943a53cf8c27efee in current term 1: Already voted for candidate 5e2731975dbf433aade4a859df9699e2 in this term.
I20260925 13:57:32.498817 23071 leader_election.cc:290] T 5dfb93f602fe447baf4972a39c54394f P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Requested vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
I20260925 13:57:32.499722 22107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dfb93f602fe447baf4972a39c54394f" candidate_uuid: "145e520e0a5a445d943a53cf8c27efee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee02f77594864349982492862d668a36"
I20260925 13:57:32.501067 21896 leader_election.cc:304] T 5dfb93f602fe447baf4972a39c54394f P 145e520e0a5a445d943a53cf8c27efee [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 145e520e0a5a445d943a53cf8c27efee; no voters: 5e2731975dbf433aade4a859df9699e2, ee02f77594864349982492862d668a36
I20260925 13:57:32.503460 22106 raft_consensus.cc:2468] T 5dfb93f602fe447baf4972a39c54394f P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e2731975dbf433aade4a859df9699e2 in term 1.
I20260925 13:57:32.504333 23075 raft_consensus.cc:2749] T 5dfb93f602fe447baf4972a39c54394f P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260925 13:57:32.505716 21732 leader_election.cc:304] T 5dfb93f602fe447baf4972a39c54394f P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2, ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee
I20260925 13:57:32.507203 23071 raft_consensus.cc:2804] T 5dfb93f602fe447baf4972a39c54394f P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:57:32.510557 23071 raft_consensus.cc:697] T 5dfb93f602fe447baf4972a39c54394f P 5e2731975dbf433aade4a859df9699e2 [term 1 LEADER]: Becoming Leader. State: Replica: 5e2731975dbf433aade4a859df9699e2, State: Running, Role: LEADER
I20260925 13:57:32.513146 23071 consensus_queue.cc:237] T 5dfb93f602fe447baf4972a39c54394f P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:32.650588 23039 raft_consensus.cc:493] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:32.651057 23039 raft_consensus.cc:515] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:32.652980 23039 leader_election.cc:290] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 13:57:32.655900 22043 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
W20260925 13:57:32.656731 22040 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
I20260925 13:57:32.657080 22040 leader_election.cc:304] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:32.657922 23039 raft_consensus.cc:2749] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:57:32.730041 23039 raft_consensus.cc:493] T 4fccfe2595134d1ba43f742a771ce4fd P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:32.730494 23039 raft_consensus.cc:515] T 4fccfe2595134d1ba43f742a771ce4fd P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } }
I20260925 13:57:32.732493 23039 leader_election.cc:290] T 4fccfe2595134d1ba43f742a771ce4fd P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881)
I20260925 13:57:32.733282 21796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fccfe2595134d1ba43f742a771ce4fd" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e2731975dbf433aade4a859df9699e2" is_pre_election: true
I20260925 13:57:32.734441 21960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fccfe2595134d1ba43f742a771ce4fd" candidate_uuid: "ee02f77594864349982492862d668a36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145e520e0a5a445d943a53cf8c27efee" is_pre_election: true
I20260925 13:57:32.735100 21960 raft_consensus.cc:2393] T 4fccfe2595134d1ba43f742a771ce4fd P 145e520e0a5a445d943a53cf8c27efee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ee02f77594864349982492862d668a36 in current term 1: Already voted for candidate 5e2731975dbf433aade4a859df9699e2 in this term.
I20260925 13:57:32.736258 22043 leader_election.cc:304] T 4fccfe2595134d1ba43f742a771ce4fd P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:32.737314 23039 raft_consensus.cc:3060] T 4fccfe2595134d1ba43f742a771ce4fd P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:57:32.747174 23039 raft_consensus.cc:2749] T 4fccfe2595134d1ba43f742a771ce4fd P ee02f77594864349982492862d668a36 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 13:57:32.791678 23086 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e942d7f2e4864b5ca08e6e830085caa0)
I20260925 13:57:32.792323 23086 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:32.795756 23086 leader_election.cc:290] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e942d7f2e4864b5ca08e6e830085caa0 (127.18.182.254:43163), a9bf5a5925ac42769e474adc84a1b725 (127.18.182.252:39283)
I20260925 13:57:32.796859 21598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ea1e11bc6d84114b7616815b6af4635" candidate_term: 7 candidate_status { last_received { term: 6 index: 236 } } ignore_live_leader: false dest_uuid: "a9bf5a5925ac42769e474adc84a1b725" is_pre_election: true
I20260925 13:57:32.797429 21598 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ea1e11bc6d84114b7616815b6af4635 in term 6.
I20260925 13:57:32.798745 22609 leader_election.cc:304] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea1e11bc6d84114b7616815b6af4635, a9bf5a5925ac42769e474adc84a1b725; no voters:
I20260925 13:57:32.799499 23086 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260925 13:57:32.799878 23086 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e942d7f2e4864b5ca08e6e830085caa0)
I20260925 13:57:32.800285 23086 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 6 FOLLOWER]: Advancing to term 7
I20260925 13:57:32.809752 23086 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:32.821697 23086 leader_election.cc:290] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [CANDIDATE]: Term 7 election: Requested vote from peers e942d7f2e4864b5ca08e6e830085caa0 (127.18.182.254:43163), a9bf5a5925ac42769e474adc84a1b725 (127.18.182.252:39283)
I20260925 13:57:32.822629 21598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ea1e11bc6d84114b7616815b6af4635" candidate_term: 7 candidate_status { last_received { term: 6 index: 236 } } ignore_live_leader: false dest_uuid: "a9bf5a5925ac42769e474adc84a1b725"
I20260925 13:57:32.823112 21598 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 6 FOLLOWER]: Advancing to term 7
I20260925 13:57:32.829334 21598 raft_consensus.cc:2468] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ea1e11bc6d84114b7616815b6af4635 in term 7.
I20260925 13:57:32.830755 22609 leader_election.cc:304] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea1e11bc6d84114b7616815b6af4635, a9bf5a5925ac42769e474adc84a1b725; no voters:
I20260925 13:57:32.833179 23086 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 7 FOLLOWER]: Leader election won for term 7
I20260925 13:57:32.847985 23086 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [term 7 LEADER]: Becoming Leader. State: Replica: 3ea1e11bc6d84114b7616815b6af4635, State: Running, Role: LEADER
I20260925 13:57:32.848941 23086 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 236, Committed index: 236, Last appended: 6.236, Last appended by leader: 236, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } }
I20260925 13:57:32.853283 23087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ea1e11bc6d84114b7616815b6af4635. Latest consensus state: current_term: 7 leader_uuid: "3ea1e11bc6d84114b7616815b6af4635" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:57:32.854028 23087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [sys.catalog]: This master's current role is: LEADER
I20260925 13:57:32.864271 23088 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260925 13:57:32.886982 23088 catalog_manager.cc:692] Loaded metadata for table default.table_2138b0cdfa7c4afda3a919dd0945a2d1 [id=012dc51aeefc492ca9aade1c75b97020]
I20260925 13:57:32.888584 23088 catalog_manager.cc:692] Loaded metadata for table default.table_8791c6009be44efab81f5f2252a3d8ae [id=038891ef1fdb4cee98081f6d56bb1c58]
I20260925 13:57:32.890472 23088 catalog_manager.cc:692] Loaded metadata for table default.table_b80b7994ff294977bd92dc8f9ddd922e [id=1f8524e67c714a18b1a8dca4d718ace1]
W20260925 13:57:32.891743 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:57:32.891916 23088 catalog_manager.cc:692] Loaded metadata for table default.table_e0eb6c1d49224d1cb9b4f6d06f0f7a29 [id=420cd09cc1184f6e9fb38b2cbc3f6fce]
I20260925 13:57:32.893268 23088 catalog_manager.cc:692] Loaded metadata for table default.table_b54e9b49662b4a8e892615fec1080a87 [id=722129e8c51a45f8ae31843d0d3f0a47]
I20260925 13:57:32.894654 23088 catalog_manager.cc:692] Loaded metadata for table default.table_683c0db2850941be84798e9dca3fe490 [id=7377fa3ea9664b47a930d9c5f88c60fa]
I20260925 13:57:32.896056 23088 catalog_manager.cc:692] Loaded metadata for table default.table_d6100b0f2b294ab099c179a7681cb4ed [id=74ee67384ecf4220b7d08fdf0637aefb]
I20260925 13:57:32.897414 23088 catalog_manager.cc:692] Loaded metadata for table default.table_fe2e32c0275e4744b1163ae7b22eaf51 [id=9e8b522f92404f2ba5505e6fa689ea5a]
I20260925 13:57:32.898887 23088 catalog_manager.cc:692] Loaded metadata for table default.table_4586487550ec46f08eae599e1c34ef17 [id=ad8f47ab1edd4ae1a1141afe4f07625a]
I20260925 13:57:32.900288 23088 catalog_manager.cc:692] Loaded metadata for table default.table_d8ce7cda4ecf4422a5e74c80d7ebf4da [id=af0f962d5d5d4a17a3f576ab6a3b4df0]
I20260925 13:57:32.901652 23088 catalog_manager.cc:692] Loaded metadata for table default.table_2a09482ff526433a8d028a964cfdfb0c [id=bb79c1d2d0094785b35250fb1b718b9a]
I20260925 13:57:32.902999 23088 catalog_manager.cc:692] Loaded metadata for table default.table_30772fdea84947818aff6f6522c73f4b [id=bcd7dc63c23c475e943a6d7395fcec10]
I20260925 13:57:32.904395 23088 catalog_manager.cc:692] Loaded metadata for table default.table_b32d230a905d44b388dc9e436965cbc9 [id=bf8cef2239904fd989622f9426412635]
I20260925 13:57:32.905747 23088 catalog_manager.cc:692] Loaded metadata for table default.table_97e22ed3359342caaff80295b5d37db2 [id=dfd05c11872140cfa86b2628c33ef6d8]
I20260925 13:57:32.907234 23088 catalog_manager.cc:692] Loaded metadata for table default.table_fc718993756d497eb9df6af8c1651362 [id=e02d9bf4f1ff4d81bb26cddfe336f414]
I20260925 13:57:32.921094 23088 tablet_loader.cc:96] loaded metadata for tablet 04cd31f29357471a882157829ca700ab (table default.table_fc718993756d497eb9df6af8c1651362 [id=e02d9bf4f1ff4d81bb26cddfe336f414])
I20260925 13:57:32.922590 23088 tablet_loader.cc:96] loaded metadata for tablet 09199f2141f746a29f410907e2a5ae37 (table default.table_683c0db2850941be84798e9dca3fe490 [id=7377fa3ea9664b47a930d9c5f88c60fa])
I20260925 13:57:32.923789 23088 tablet_loader.cc:96] loaded metadata for tablet 1d47ea37ee1e4abfbc0f47001c811748 (table default.table_b80b7994ff294977bd92dc8f9ddd922e [id=1f8524e67c714a18b1a8dca4d718ace1])
I20260925 13:57:32.924968 23088 tablet_loader.cc:96] loaded metadata for tablet 242acfcc09ca4f879e5e44a73ef16cec (table default.table_d8ce7cda4ecf4422a5e74c80d7ebf4da [id=af0f962d5d5d4a17a3f576ab6a3b4df0])
I20260925 13:57:32.926096 23088 tablet_loader.cc:96] loaded metadata for tablet 24dd39b302344f8d8f5166e9d641d13a (table default.table_e0eb6c1d49224d1cb9b4f6d06f0f7a29 [id=420cd09cc1184f6e9fb38b2cbc3f6fce])
I20260925 13:57:32.927284 23088 tablet_loader.cc:96] loaded metadata for tablet 4fccfe2595134d1ba43f742a771ce4fd (table default.table_4586487550ec46f08eae599e1c34ef17 [id=ad8f47ab1edd4ae1a1141afe4f07625a])
I20260925 13:57:32.928529 23088 tablet_loader.cc:96] loaded metadata for tablet 5dfb93f602fe447baf4972a39c54394f (table default.table_8791c6009be44efab81f5f2252a3d8ae [id=038891ef1fdb4cee98081f6d56bb1c58])
I20260925 13:57:32.929693 23088 tablet_loader.cc:96] loaded metadata for tablet 60ef9c152aec465eb0bb7dc286c74277 (table default.table_30772fdea84947818aff6f6522c73f4b [id=bcd7dc63c23c475e943a6d7395fcec10])
I20260925 13:57:32.930846 23088 tablet_loader.cc:96] loaded metadata for tablet 88b875abab81457481d4b98570a74ea1 (table default.table_fe2e32c0275e4744b1163ae7b22eaf51 [id=9e8b522f92404f2ba5505e6fa689ea5a])
I20260925 13:57:32.932039 23088 tablet_loader.cc:96] loaded metadata for tablet 8ace3cd70b45451d8e9fb4b8a8d4dd0d (table default.table_d6100b0f2b294ab099c179a7681cb4ed [id=74ee67384ecf4220b7d08fdf0637aefb])
I20260925 13:57:32.933183 23088 tablet_loader.cc:96] loaded metadata for tablet a06a5aac5d464d339c870df4ebd63d92 (table default.table_2a09482ff526433a8d028a964cfdfb0c [id=bb79c1d2d0094785b35250fb1b718b9a])
I20260925 13:57:32.934602 23088 tablet_loader.cc:96] loaded metadata for tablet a3892ff71c0b447e86d29aed4017caf6 (table default.table_b54e9b49662b4a8e892615fec1080a87 [id=722129e8c51a45f8ae31843d0d3f0a47])
I20260925 13:57:32.935902 23088 tablet_loader.cc:96] loaded metadata for tablet ba1242d8795641e7a89006d154be1216 (table default.table_97e22ed3359342caaff80295b5d37db2 [id=dfd05c11872140cfa86b2628c33ef6d8])
I20260925 13:57:32.937090 23088 tablet_loader.cc:96] loaded metadata for tablet d9dbc7ac9d0142c8a92262d2117ee6ec (table default.table_2138b0cdfa7c4afda3a919dd0945a2d1 [id=012dc51aeefc492ca9aade1c75b97020])
I20260925 13:57:32.938266 23088 tablet_loader.cc:96] loaded metadata for tablet ef57aa72abb2405b8187a7ed56d57f96 (table default.table_b32d230a905d44b388dc9e436965cbc9 [id=bf8cef2239904fd989622f9426412635])
I20260925 13:57:32.942353 23088 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260925 13:57:32.955490 23088 catalog_manager.cc:1308] Loaded cluster ID: d5d0bc5567a74d9cad4d92a9d1041fd9
I20260925 13:57:32.955744 23088 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260925 13:57:32.968345 23088 catalog_manager.cc:1553] Loading token signing keys...
I20260925 13:57:32.985105 23088 catalog_manager.cc:6230] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635: Loaded TSK: 0
I20260925 13:57:32.986176 23088 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260925 13:57:32.998963 23088 catalog_manager.cc:1576] Loading latest processed Hive Metastore notification log event ID...
I20260925 13:57:33.011039 23088 catalog_manager.cc:6068] Last processed Hive Metastore notification event ID: 51
W20260925 13:57:33.085822 22040 connection.cc:573] client connection to 127.18.182.254:43163 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 13:57:33.086748 21893 connection.cc:573] client connection to 127.18.182.254:43163 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 13:57:33.089078 22162 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.18.182.254:43163)
W20260925 13:57:33.093037 21729 connection.cc:573] client connection to 127.18.182.254:43163 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 13:57:33.095762 22602 connection.cc:573] client connection to 127.18.182.254:43163 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 13:57:33.096201 22602 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107) [suppressed 3 similar messages]
I20260925 13:57:33.104358 22648 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41880:
name: "default.Table_8229f26c3e104bf1885bce04e09f8e29"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20260925 13:57:33.111738 22766 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 52, previous 52; dumping first 2 out of 2 received events: NotificationEvent(eventId=52, eventTime=1790344651, eventType=DROP_TABLE, dbName=default, tableName=table_8229f26c3e104bf1885bce04e09f8e29, message={"server":"","servicePrincipal":"","db":"default","table":"table_8229f26c3e104bf1885bce04e09f8e29","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_8229f26c3e104bf1885bce04e09f8e29\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790344650},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/warehouse/table_8229f26c3e104bf1885bce04e09f8e29\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_8229f26c3e104bf1885bce04e09f8e29\",\"kudu.table_id\":\"fcde453f78884574a9f2aeba9492ae02\",\"transient_lastDdlTime\":\"1790344650\",\"comment\":\"\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.master_addresses\":\"127.18.182.254:43163,127.18.182.253:46177,127.18.182.252:39283\",\"kudu.cluster_id\":\"d5d0bc5567a74d9cad4d92a9d1041fd9\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790344651}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=52, eventTime=1790344651, eventType=CREATE_TABLE, dbName=default, tableName=table_4586487550ec46f08eae599e1c34ef17, message={"server":"","servicePrincipal":"","db":"default","table":"table_4586487550ec46f08eae599e1c34ef17","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_4586487550ec46f08eae599e1c34ef17\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790344651},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskYmJttM/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790344553314151-19163-0/minicluster-data/warehouse/table_4586487550ec46f08eae599e1c34ef17\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_4586487550ec46f08eae599e1c34ef17\",\"kudu.table_id\":\"ad8f47ab1edd4ae1a1141afe4f07625a\",\"transient_lastDdlTime\":\"1790344651\",\"comment\":\"\",\"kudu.master_addresses\":\"127.18.182.254:43163,127.18.182.253:46177,127.18.182.252:39283\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"d5d0bc5567a74d9cad4d92a9d1041fd9\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790344651,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
W20260925 13:57:33.137537 23089 negotiation.cc:336] Failed RPC negotiation. Trace:
0925 13:57:33.090169 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.18.182.254:43163 (local address 127.0.0.1:55396)
0925 13:57:33.134699 (+ 44530us) negotiation.cc:107] Waiting for socket to connect
0925 13:57:33.135871 (+ 1172us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.182.254:43163: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":43827,"threads_started":1}
I20260925 13:57:33.145749 23071 consensus_queue.cc:1048] T 4fccfe2595134d1ba43f742a771ce4fd P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:33.147207 23071 consensus_queue.cc:1048] T 4fccfe2595134d1ba43f742a771ce4fd P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 13:57:33.152096 22602 leader_election.cc:336] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e942d7f2e4864b5ca08e6e830085caa0 (127.18.182.254:43163): Network error: Client connection negotiation failed: client connection to 127.18.182.254:43163: connect: Connection refused (error 111)
W20260925 13:57:33.152947 22602 leader_election.cc:336] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer e942d7f2e4864b5ca08e6e830085caa0 (127.18.182.254:43163): Network error: Client connection negotiation failed: client connection to 127.18.182.254:43163: connect: Connection refused (error 111)
@ 0x7f5e3f8b2920 google::LogMessage::Flush() at ??:0
@ 0x7f5e3f8b268f google::LogMessage::~LogMessage() at ??:0
W20260925 13:57:33.274284 22602 consensus_peers.cc:597] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 -> Peer e942d7f2e4864b5ca08e6e830085caa0 (127.18.182.254:43163): Couldn't send request to peer e942d7f2e4864b5ca08e6e830085caa0. Status: Network error: Client connection negotiation failed: client connection to 127.18.182.254:43163: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 13:57:33.295737 23092 consensus_queue.cc:1048] T 5dfb93f602fe447baf4972a39c54394f P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:57:33.298643 22153 heartbeater.cc:499] Master 127.18.182.253:46177 was elected leader, sending a full tablet report...
I20260925 13:57:33.309636 22008 heartbeater.cc:499] Master 127.18.182.253:46177 was elected leader, sending a full tablet report...
I20260925 13:57:33.327741 21843 heartbeater.cc:499] Master 127.18.182.253:46177 was elected leader, sending a full tablet report...
W20260925 13:57:33.341652 22642 catalog_manager.cc:5671] Ignoring report from unknown tablet 71ff569d68154b61bcc14937b2e37fe5
W20260925 13:57:33.342022 22642 catalog_manager.cc:5671] Ignoring report from unknown tablet ccefd36f4e2a48f9b3d66cb147a022b5
W20260925 13:57:33.342262 22642 catalog_manager.cc:5671] Ignoring report from unknown tablet d267def3f0864435a0d2c40fd7052eab
W20260925 13:57:33.351936 22646 catalog_manager.cc:5671] Ignoring report from unknown tablet f5bd96a2bdb842e9b3bc1f706786d6b1
W20260925 13:57:33.360373 22646 catalog_manager.cc:5671] Ignoring report from unknown tablet 60256642160a44a2bab77b97d78c27a8
W20260925 13:57:33.360689 22646 catalog_manager.cc:5671] Ignoring report from unknown tablet d267def3f0864435a0d2c40fd7052eab
I20260925 13:57:33.392159 21598 raft_consensus.cc:1275] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [term 7 FOLLOWER]: Refusing update from remote peer 3ea1e11bc6d84114b7616815b6af4635: Log matching property violated. Preceding OpId in replica: term: 6 index: 236. Preceding OpId from leader: term: 7 index: 237. (index mismatch)
I20260925 13:57:33.393965 23087 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 237, Last known committed idx: 236, Time since last communication: 0.000s
I20260925 13:57:33.387794 22646 catalog_manager.cc:5838] T 5dfb93f602fe447baf4972a39c54394f P 5e2731975dbf433aade4a859df9699e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e2731975dbf433aade4a859df9699e2 (127.18.182.193). New cstate: current_term: 1 leader_uuid: "5e2731975dbf433aade4a859df9699e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } health_report { overall_health: UNKNOWN } } }
@ 0x7f5e4aa4535c kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260925 13:57:33.397156 23071 consensus_queue.cc:1048] T 5dfb93f602fe447baf4972a39c54394f P 5e2731975dbf433aade4a859df9699e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.033s
W20260925 13:57:33.440335 22642 catalog_manager.cc:5671] Ignoring report from unknown tablet 60256642160a44a2bab77b97d78c27a8
W20260925 13:57:33.440635 22642 catalog_manager.cc:5671] Ignoring report from unknown tablet d267def3f0864435a0d2c40fd7052eab
W20260925 13:57:33.441141 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:57:33.454883 23108 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ea1e11bc6d84114b7616815b6af4635. Latest consensus state: current_term: 7 leader_uuid: "3ea1e11bc6d84114b7616815b6af4635" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:57:33.455575 23108 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [sys.catalog]: This master's current role is: FOLLOWER
@ 0x7f5e4aa44b1c kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260925 13:57:33.537395 23116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 7 leader_uuid: "3ea1e11bc6d84114b7616815b6af4635" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:57:33.538100 23116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ea1e11bc6d84114b7616815b6af4635 [sys.catalog]: This master's current role is: LEADER
I20260925 13:57:33.553995 23108 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "3ea1e11bc6d84114b7616815b6af4635" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e942d7f2e4864b5ca08e6e830085caa0" member_type: VOTER last_known_addr { host: "127.18.182.254" port: 43163 } } peers { permanent_uuid: "3ea1e11bc6d84114b7616815b6af4635" member_type: VOTER last_known_addr { host: "127.18.182.253" port: 46177 } } peers { permanent_uuid: "a9bf5a5925ac42769e474adc84a1b725" member_type: VOTER last_known_addr { host: "127.18.182.252" port: 39283 } } }
I20260925 13:57:33.555018 23108 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9bf5a5925ac42769e474adc84a1b725 [sys.catalog]: This master's current role is: FOLLOWER
@ 0x7f5e4aa48b92 kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
W20260925 13:57:33.643314 22642 catalog_manager.cc:5671] Ignoring report from unknown tablet 60256642160a44a2bab77b97d78c27a8
@ 0x7f5e4aa48b4a _ZNSt3__18__invokeIRZN4kudu6master30HmsNotificationLogListenerTask4InitEvE3$_0JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOS6_DpOS7_ at ??:0
@ 0x7f5e4aa48ada std::__1::__invoke_void_return_wrapper<>::__call<>() at ??:0
@ 0x7f5e4aa48aa2 std::__1::__function::__alloc_func<>::operator()() at ??:0
@ 0x7f5e4aa47d9e std::__1::__function::__func<>::operator()() at ??:0
@ 0x5bd4e5 std::__1::__function::__value_func<>::operator()() at ??:?
@ 0x5bd31a std::__1::function<>::operator()() at ??:?
@ 0x7f5e41aa881b kudu::Thread::SuperviseThread() at ??:0
@ 0x4569a0 __tsan_thread_start_func at ??:?
@ 0x7f5e432096db start_thread at ??:0
@ 0x7f5e3e55461f clone at ??:0
W20260925 13:57:33.929957 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:34.236961 22163 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.18.182.253:46177)
W20260925 13:57:34.245085 22041 connection.cc:573] client connection to 127.18.182.253:46177 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 13:57:34.245178 21894 connection.cc:573] client connection to 127.18.182.253:46177 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 13:57:34.246084 21730 connection.cc:573] client connection to 127.18.182.253:46177 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 13:57:34.254261 23137 negotiation.cc:336] Failed RPC negotiation. Trace:
0925 13:57:34.238081 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.18.182.253:46177 (local address 127.0.0.1:33466)
0925 13:57:34.242221 (+ 4140us) negotiation.cc:107] Waiting for socket to connect
0925 13:57:34.242361 (+ 140us) client_negotiation.cc:175] Beginning negotiation
0925 13:57:34.243816 (+ 1455us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0925 13:57:34.252360 (+ 8544us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.182.253:46177: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":2365,"thread_start_us":12723,"threads_started":1}
W20260925 13:57:34.320789 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:34.901836 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:35.293951 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:35.831193 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:57:36.190826 23142 raft_consensus.cc:493] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ee02f77594864349982492862d668a36)
I20260925 13:57:36.191255 23142 raft_consensus.cc:515] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:36.192816 23142 leader_election.cc:290] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 13:57:36.194386 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
W20260925 13:57:36.194998 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
I20260925 13:57:36.195247 21732 leader_election.cc:304] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
I20260925 13:57:36.195962 23142 raft_consensus.cc:2749] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 13:57:36.272166 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:36.699792 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:37.280083 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:37.859144 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:38.319619 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:38.866482 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:39.291999 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:57:39.747375 23132 raft_consensus.cc:493] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:39.748080 23132 raft_consensus.cc:515] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:39.749754 23132 leader_election.cc:290] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 13:57:39.752014 22043 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
W20260925 13:57:39.752523 22040 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
I20260925 13:57:39.752888 22040 leader_election.cc:304] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:39.753597 23132 raft_consensus.cc:2749] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:57:39.794379 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:40.277065 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:40.880177 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:41.309479 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:41.319094 22148 debug-util.cc:398] Leaking SignalData structure 0x7b0800121920 after lost signal to thread 22017
W20260925 13:57:41.319994 22148 debug-util.cc:398] Leaking SignalData structure 0x7b080012b780 after lost signal to thread 22151
W20260925 13:57:41.768862 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:42.239528 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:42.308178 21619 debug-util.cc:398] Leaking SignalData structure 0x7b080004e200 after lost signal to thread 21550
W20260925 13:57:42.308796 21619 debug-util.cc:398] Leaking SignalData structure 0x7b08000bb4e0 after lost signal to thread 21622
W20260925 13:57:42.720638 21896 consensus_peers.cc:597] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee -> Peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Couldn't send request to peer ee02f77594864349982492862d668a36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60256642160a44a2bab77b97d78c27a8. This is attempt 11: this message will repeat every 5th retry.
W20260925 13:57:42.826785 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:43.376085 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:43.807773 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:44.189065 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:44.236939 21575 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 5eb86772ca2e4306a5c164342514b7b0 (table default.table_d6100b0f2b294ab099c179a7681cb4ed [id=74ee67384ecf4220b7d08fdf0637aefb], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 5eb86772ca2e4306a5c164342514b7b0 consensus state has no leader
W20260925 13:57:44.641021 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:45.234323 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:45.713281 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:57:45.881911 23202 raft_consensus.cc:493] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ee02f77594864349982492862d668a36)
I20260925 13:57:45.882555 23202 raft_consensus.cc:515] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:45.885196 23202 leader_election.cc:290] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 13:57:45.886758 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
W20260925 13:57:45.888216 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
I20260925 13:57:45.888643 21732 leader_election.cc:304] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
I20260925 13:57:45.889315 23202 raft_consensus.cc:2749] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 13:57:46.291397 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:46.890096 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:47.318372 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:47.791950 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:48.187479 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:48.797073 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:49.366750 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:49.907126 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:57:50.218824 23208 raft_consensus.cc:493] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:50.219462 23208 raft_consensus.cc:515] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:50.221637 23208 leader_election.cc:290] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 13:57:50.222980 22040 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
W20260925 13:57:50.223591 22043 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
I20260925 13:57:50.223878 22043 leader_election.cc:304] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:50.224536 23208 raft_consensus.cc:2749] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:57:50.402134 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:50.950841 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:51.526139 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:51.911231 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:52.337170 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:52.832731 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:53.291613 21896 consensus_peers.cc:597] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee -> Peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Couldn't send request to peer ee02f77594864349982492862d668a36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60256642160a44a2bab77b97d78c27a8. This is attempt 16: this message will repeat every 5th retry.
W20260925 13:57:53.355011 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:57:53.362233 23208 raft_consensus.cc:493] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:53.362656 23208 raft_consensus.cc:515] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:53.364526 23208 leader_election.cc:290] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 13:57:53.365864 22040 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
W20260925 13:57:53.367010 22043 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
I20260925 13:57:53.367298 22043 leader_election.cc:304] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:53.367913 23208 raft_consensus.cc:2749] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:57:53.973889 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:54.514858 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:55.041848 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:55.441888 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:56.061131 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:57:56.072557 23208 raft_consensus.cc:493] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:57:56.073272 23208 raft_consensus.cc:515] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:56.075197 23208 leader_election.cc:290] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 13:57:56.077319 22040 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
W20260925 13:57:56.077832 22043 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
I20260925 13:57:56.078207 22043 leader_election.cc:304] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:57:56.078941 23208 raft_consensus.cc:2749] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:57:56.622182 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:57.124153 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:57.543861 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:57.992620 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:58.584062 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:57:59.083024 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:57:59.108745 23328 raft_consensus.cc:493] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ee02f77594864349982492862d668a36)
I20260925 13:57:59.109524 23328 raft_consensus.cc:515] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:57:59.111465 23328 leader_election.cc:290] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 13:57:59.113332 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
W20260925 13:57:59.114009 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
I20260925 13:57:59.114401 21732 leader_election.cc:304] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
I20260925 13:57:59.115010 23328 raft_consensus.cc:2749] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 13:57:59.612466 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:00.108991 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:00.564018 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:00.988140 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:01.558205 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:02.056216 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:02.459656 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:03.036011 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:03.606236 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:04.022531 21896 consensus_peers.cc:597] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee -> Peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Couldn't send request to peer ee02f77594864349982492862d668a36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60256642160a44a2bab77b97d78c27a8. This is attempt 21: this message will repeat every 5th retry.
W20260925 13:58:04.165393 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:04.742588 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:05.232266 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:05.700770 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:06.213059 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:06.769976 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:58:07.131668 23409 raft_consensus.cc:493] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:58:07.132380 23409 raft_consensus.cc:515] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:58:07.134572 23409 leader_election.cc:290] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 13:58:07.136425 22040 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
W20260925 13:58:07.137022 22043 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
I20260925 13:58:07.137257 22043 leader_election.cc:304] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:58:07.137791 23409 raft_consensus.cc:2749] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:58:07.338279 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:07.887979 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:08.236002 21843 heartbeater.cc:646] Failed to heartbeat to 127.18.182.253:46177 (69 consecutive failures): Network error: Failed to ping master at 127.18.182.253:46177: Client connection negotiation failed: client connection to 127.18.182.253:46177: connect: Connection refused (error 111)
W20260925 13:58:08.286755 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:08.479266 22005 heartbeater.cc:646] Failed to heartbeat to 127.18.182.252:39283 (44 consecutive failures): Timed out: Failed to ping master at 127.18.182.252:39283: Ping RPC to 127.18.182.252:39283 timed out after 1.000s (SENT)
W20260925 13:58:08.690688 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:08.715212 22154 heartbeater.cc:646] Failed to heartbeat to 127.18.182.254:43163 (75 consecutive failures): Network error: Failed to ping master at 127.18.182.254:43163: Client connection negotiation failed: client connection to 127.18.182.254:43163: connect: Connection refused (error 111)
W20260925 13:58:09.246173 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:09.639163 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:10.086234 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:58:10.466740 23409 raft_consensus.cc:493] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:58:10.467200 23409 raft_consensus.cc:515] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:58:10.469106 23409 leader_election.cc:290] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 13:58:10.470932 22040 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
W20260925 13:58:10.471493 22043 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
I20260925 13:58:10.471829 22043 leader_election.cc:304] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:58:10.472723 23440 raft_consensus.cc:2749] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:58:10.616690 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:11.142514 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:11.637651 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:12.154170 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:12.766333 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:13.324301 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:58:13.769801 23448 raft_consensus.cc:493] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:58:13.770279 23448 raft_consensus.cc:515] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:58:13.772205 23448 leader_election.cc:290] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 13:58:13.774549 22040 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
W20260925 13:58:13.776984 22043 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
I20260925 13:58:13.777340 22043 leader_election.cc:304] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:58:13.778270 23448 raft_consensus.cc:2749] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:58:13.880968 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:14.489812 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:14.984181 21896 consensus_peers.cc:597] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee -> Peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Couldn't send request to peer ee02f77594864349982492862d668a36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60256642160a44a2bab77b97d78c27a8. This is attempt 26: this message will repeat every 5th retry.
W20260925 13:58:15.101938 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:15.682770 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:16.103206 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:16.632953 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:58:16.902355 23474 raft_consensus.cc:493] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ee02f77594864349982492862d668a36)
I20260925 13:58:16.903007 23474 raft_consensus.cc:515] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:58:16.905422 23474 leader_election.cc:290] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 13:58:16.907162 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
W20260925 13:58:16.908285 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
I20260925 13:58:16.908599 21732 leader_election.cc:304] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
I20260925 13:58:16.909544 23474 raft_consensus.cc:2749] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 13:58:17.023847 21575 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 5eb86772ca2e4306a5c164342514b7b0 (table default.table_d6100b0f2b294ab099c179a7681cb4ed [id=74ee67384ecf4220b7d08fdf0637aefb], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 5eb86772ca2e4306a5c164342514b7b0 consensus state has no leader
W20260925 13:58:17.171530 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:17.605126 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:18.170620 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:18.764576 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:19.179683 22001 debug-util.cc:398] Leaking SignalData structure 0x7b0800121d40 after lost signal to thread 21854
W20260925 13:58:19.180759 22001 debug-util.cc:398] Leaking SignalData structure 0x7b080012a520 after lost signal to thread 22004
W20260925 13:58:19.253999 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:19.788257 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:20.274220 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:20.885042 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:21.449703 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:21.873831 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:22.301970 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:22.871428 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:58:23.119488 23474 raft_consensus.cc:493] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ee02f77594864349982492862d668a36)
I20260925 13:58:23.120203 23474 raft_consensus.cc:515] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:58:23.122315 23474 leader_election.cc:290] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 13:58:23.123910 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
W20260925 13:58:23.124490 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
I20260925 13:58:23.124769 21732 leader_election.cc:304] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
I20260925 13:58:23.125298 23474 raft_consensus.cc:2749] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:58:23.285327 23502 raft_consensus.cc:493] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:58:23.285974 23502 raft_consensus.cc:515] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:58:23.288681 23502 leader_election.cc:290] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 13:58:23.290611 22043 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
W20260925 13:58:23.291635 22040 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
I20260925 13:58:23.292016 22040 leader_election.cc:304] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:58:23.292676 23502 raft_consensus.cc:2749] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:58:23.451531 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:24.039861 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:24.597748 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:25.013142 21896 consensus_peers.cc:597] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee -> Peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Couldn't send request to peer ee02f77594864349982492862d668a36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60256642160a44a2bab77b97d78c27a8. This is attempt 31: this message will repeat every 5th retry.
W20260925 13:58:25.211606 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:25.766042 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:26.202572 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:26.603490 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:27.026504 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:58:27.389778 23533 raft_consensus.cc:493] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:58:27.390440 23533 raft_consensus.cc:515] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:58:27.392740 23533 leader_election.cc:290] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 13:58:27.395483 22040 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
W20260925 13:58:27.399127 22043 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
I20260925 13:58:27.399520 22043 leader_election.cc:304] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:58:27.400151 23533 raft_consensus.cc:2749] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:58:27.647711 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:28.084580 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:28.699307 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:58:29.049185 23552 raft_consensus.cc:493] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:58:29.049669 23552 raft_consensus.cc:515] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:58:29.051720 23552 leader_election.cc:290] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 13:58:29.052932 22040 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
W20260925 13:58:29.053364 22043 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
I20260925 13:58:29.053759 22043 leader_election.cc:304] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:58:29.055125 23552 raft_consensus.cc:2749] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:58:29.088968 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:29.484997 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:29.869055 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:30.485056 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:31.076593 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:31.466168 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:58:31.650461 23566 raft_consensus.cc:493] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ee02f77594864349982492862d668a36)
I20260925 13:58:31.651000 23566 raft_consensus.cc:515] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:58:31.652722 23566 leader_election.cc:290] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 13:58:31.654312 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
W20260925 13:58:31.654834 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
I20260925 13:58:31.655098 21732 leader_election.cc:304] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
I20260925 13:58:31.655658 23566 raft_consensus.cc:2749] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 13:58:31.911464 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:32.334815 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:32.718255 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:33.204126 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:33.755517 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:34.304728 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:34.619539 21896 consensus_peers.cc:597] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee -> Peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Couldn't send request to peer ee02f77594864349982492862d668a36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60256642160a44a2bab77b97d78c27a8. This is attempt 36: this message will repeat every 5th retry.
W20260925 13:58:34.766072 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:35.298611 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:35.797040 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:36.328760 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:36.862419 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:37.281466 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:37.786670 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:38.200045 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:38.593070 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:58:39.033704 23566 raft_consensus.cc:493] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ee02f77594864349982492862d668a36)
I20260925 13:58:39.034165 23566 raft_consensus.cc:515] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:58:39.036772 23566 leader_election.cc:290] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 13:58:39.038451 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
W20260925 13:58:39.039043 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
I20260925 13:58:39.039407 21732 leader_election.cc:304] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
I20260925 13:58:39.040030 23566 raft_consensus.cc:2749] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 13:58:39.086133 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:39.585495 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:39.630367 21619 debug-util.cc:398] Leaking SignalData structure 0x7b08000b1440 after lost signal to thread 21550
W20260925 13:58:39.631238 21619 debug-util.cc:398] Leaking SignalData structure 0x7b0800048fe0 after lost signal to thread 21622
W20260925 13:58:40.029408 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:40.523028 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:41.112663 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:41.674330 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:42.154450 21838 debug-util.cc:398] Leaking SignalData structure 0x7b08000d9660 after lost signal to thread 21681
W20260925 13:58:42.155637 21838 debug-util.cc:398] Leaking SignalData structure 0x7b0800042540 after lost signal to thread 21841
W20260925 13:58:42.284089 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:42.786890 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:43.272886 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:58:43.464636 23622 raft_consensus.cc:493] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:58:43.465323 23622 raft_consensus.cc:515] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:58:43.470888 22040 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
I20260925 13:58:43.472627 23622 leader_election.cc:290] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 13:58:43.473728 22043 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
I20260925 13:58:43.474071 22043 leader_election.cc:304] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:58:43.475030 23622 raft_consensus.cc:2749] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:58:43.795841 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:44.323722 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:44.895201 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:45.285825 21896 consensus_peers.cc:597] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee -> Peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Couldn't send request to peer ee02f77594864349982492862d668a36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60256642160a44a2bab77b97d78c27a8. This is attempt 41: this message will repeat every 5th retry.
W20260925 13:58:45.288080 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:45.763923 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:46.140151 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:46.529547 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:58:46.873026 23666 raft_consensus.cc:493] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:58:46.873741 23666 raft_consensus.cc:515] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:58:46.875818 23666 leader_election.cc:290] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 13:58:46.877215 22040 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
W20260925 13:58:46.877853 22043 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
I20260925 13:58:46.878149 22043 leader_election.cc:304] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:58:46.878675 23666 raft_consensus.cc:2749] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:58:47.018275 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:47.619383 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:48.028182 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:48.554632 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:48.956158 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:49.471258 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:49.997114 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:50.588264 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:50.636338 21838 debug-util.cc:398] Leaking SignalData structure 0x7b080011f740 after lost signal to thread 21681
W20260925 13:58:50.637547 21838 debug-util.cc:398] Leaking SignalData structure 0x7b080006e080 after lost signal to thread 21841
W20260925 13:58:51.046821 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:51.435640 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:51.847052 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:52.319121 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:58:52.383437 23686 raft_consensus.cc:493] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ee02f77594864349982492862d668a36)
I20260925 13:58:52.384063 23686 raft_consensus.cc:515] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:58:52.385947 23686 leader_election.cc:290] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 13:58:52.387270 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
W20260925 13:58:52.388269 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
I20260925 13:58:52.388530 21732 leader_election.cc:304] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
I20260925 13:58:52.389129 23686 raft_consensus.cc:2749] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 13:58:52.815472 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:53.342072 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:53.727571 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:54.212525 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:54.692124 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:54.905977 21896 consensus_peers.cc:597] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee -> Peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Couldn't send request to peer ee02f77594864349982492862d668a36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60256642160a44a2bab77b97d78c27a8. This is attempt 46: this message will repeat every 5th retry.
W20260925 13:58:55.294888 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:55.676316 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:56.139046 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:56.657104 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:57.127586 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:57.638145 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:58.143079 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:58:58.616117 21577 maintenance_manager.cc:419] P a9bf5a5925ac42769e474adc84a1b725: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033386
W20260925 13:58:58.626296 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:58:58.681607 23686 raft_consensus.cc:493] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ee02f77594864349982492862d668a36)
I20260925 13:58:58.682106 23686 raft_consensus.cc:515] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:58:58.685052 23686 leader_election.cc:290] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 13:58:58.687501 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
W20260925 13:58:58.694835 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
I20260925 13:58:58.695212 21732 leader_election.cc:304] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
I20260925 13:58:58.696321 23686 raft_consensus.cc:2749] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:58:58.781682 21576 maintenance_manager.cc:643] P a9bf5a5925ac42769e474adc84a1b725: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.164s user 0.129s sys 0.011s Metrics: {"bytes_written":39205,"cfile_init":1,"drs_written":1,"lbm_read_time_us":283,"lbm_reads_lt_1ms":4,"lbm_write_time_us":4024,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"rows_written":101,"thread_start_us":3136,"threads_started":1,"wal-append.queue_time_us":3536}
I20260925 13:58:58.783898 21577 maintenance_manager.cc:419] P a9bf5a5925ac42769e474adc84a1b725: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7659 bytes on disk
I20260925 13:58:58.785461 21576 maintenance_manager.cc:643] P a9bf5a5925ac42769e474adc84a1b725: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":200,"lbm_reads_lt_1ms":4}
W20260925 13:58:59.206715 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:58:59.355549 22148 debug-util.cc:398] Leaking SignalData structure 0x7b08000fbba0 after lost signal to thread 22017
W20260925 13:58:59.356576 22148 debug-util.cc:398] Leaking SignalData structure 0x7b0800128460 after lost signal to thread 22151
W20260925 13:58:59.692965 22001 debug-util.cc:398] Leaking SignalData structure 0x7b0800118860 after lost signal to thread 21854
W20260925 13:58:59.694094 22001 debug-util.cc:398] Leaking SignalData structure 0x7b08000e1d40 after lost signal to thread 22004
W20260925 13:58:59.744903 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:00.227978 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:00.847882 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:01.291807 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:01.776186 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:02.309578 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:59:02.320573 23772 raft_consensus.cc:493] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:59:02.321234 23772 raft_consensus.cc:515] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 13:59:02.325956 22040 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
W20260925 13:59:02.326629 22043 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
I20260925 13:59:02.325973 23772 leader_election.cc:290] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
I20260925 13:59:02.327139 22043 leader_election.cc:304] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:59:02.327965 23772 raft_consensus.cc:2749] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:59:02.895139 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:03.301910 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:03.826028 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:04.265357 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:04.726889 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:59:05.123348 23772 raft_consensus.cc:493] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:59:05.123943 23772 raft_consensus.cc:515] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:59:05.125676 23772 leader_election.cc:290] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 13:59:05.127004 22040 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
W20260925 13:59:05.127897 22043 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
I20260925 13:59:05.128257 22043 leader_election.cc:304] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:59:05.128867 23772 raft_consensus.cc:2749] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:59:05.298228 21896 consensus_peers.cc:597] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee -> Peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Couldn't send request to peer ee02f77594864349982492862d668a36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60256642160a44a2bab77b97d78c27a8. This is attempt 51: this message will repeat every 5th retry.
W20260925 13:59:05.298861 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:05.873227 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:06.363205 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:06.911010 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:07.493090 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:07.979882 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:08.264699 21848 heartbeater.cc:646] Failed to heartbeat to 127.18.182.254:43163 (192 consecutive failures): Network error: Failed to ping master at 127.18.182.254:43163: Client connection negotiation failed: client connection to 127.18.182.254:43163: connect: Connection refused (error 111)
I20260925 13:59:08.378140 23798 raft_consensus.cc:493] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ee02f77594864349982492862d668a36)
I20260925 13:59:08.378859 23798 raft_consensus.cc:515] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:59:08.380784 23798 leader_election.cc:290] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 13:59:08.382544 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
W20260925 13:59:08.383213 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
I20260925 13:59:08.383565 21732 leader_election.cc:304] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
I20260925 13:59:08.384246 23798 raft_consensus.cc:2749] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 13:59:08.390930 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:08.680027 22005 heartbeater.cc:646] Failed to heartbeat to 127.18.182.252:39283 (103 consecutive failures): Timed out: Failed to ping master at 127.18.182.252:39283: Ping RPC to 127.18.182.252:39283 timed out after 1.000s (SENT)
W20260925 13:59:08.729238 22153 heartbeater.cc:646] Failed to heartbeat to 127.18.182.253:46177 (188 consecutive failures): Network error: Failed to ping master at 127.18.182.253:46177: Client connection negotiation failed: client connection to 127.18.182.253:46177: connect: Connection refused (error 111)
W20260925 13:59:08.926864 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:09.307951 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:09.732667 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:10.133842 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:10.647804 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:11.240295 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:11.802661 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:12.391068 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:12.807135 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:13.375030 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:13.991844 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:14.454591 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:14.894268 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:15.190387 21896 consensus_peers.cc:597] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee -> Peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Couldn't send request to peer ee02f77594864349982492862d668a36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60256642160a44a2bab77b97d78c27a8. This is attempt 56: this message will repeat every 5th retry.
W20260925 13:59:15.459879 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:59:15.825358 23798 raft_consensus.cc:493] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ee02f77594864349982492862d668a36)
I20260925 13:59:15.825968 23798 raft_consensus.cc:515] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:59:15.828615 23798 leader_election.cc:290] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 13:59:15.830302 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
W20260925 13:59:15.830896 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
I20260925 13:59:15.831228 21732 leader_election.cc:304] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
I20260925 13:59:15.831856 23798 raft_consensus.cc:2749] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 13:59:15.848260 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:59:15.869827 23828 raft_consensus.cc:493] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:59:15.870422 23828 raft_consensus.cc:515] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:59:15.872364 23828 leader_election.cc:290] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 13:59:15.873868 22040 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
W20260925 13:59:15.874387 22043 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
I20260925 13:59:15.874780 22043 leader_election.cc:304] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:59:15.875468 23828 raft_consensus.cc:2749] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:59:16.415638 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:17.026358 21575 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 5eb86772ca2e4306a5c164342514b7b0 (table default.table_d6100b0f2b294ab099c179a7681cb4ed [id=74ee67384ecf4220b7d08fdf0637aefb], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 5eb86772ca2e4306a5c164342514b7b0 consensus state has no leader
W20260925 13:59:17.041357 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:17.447547 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:17.999609 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:18.565994 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:19.047220 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:19.468029 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:20.093317 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:20.605718 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:59:20.966181 23828 raft_consensus.cc:493] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:59:20.966759 23828 raft_consensus.cc:515] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:59:20.969144 23828 leader_election.cc:290] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 13:59:20.970697 22040 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
W20260925 13:59:20.971164 22043 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
I20260925 13:59:20.971539 22043 leader_election.cc:304] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:59:20.972198 23828 raft_consensus.cc:2749] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:59:21.171123 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:21.792441 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:22.351218 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:22.799490 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:23.313158 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:23.741755 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:24.297255 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:24.837203 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:24.899340 21896 consensus_peers.cc:597] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee -> Peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Couldn't send request to peer ee02f77594864349982492862d668a36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60256642160a44a2bab77b97d78c27a8. This is attempt 61: this message will repeat every 5th retry.
W20260925 13:59:25.273701 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:25.857645 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:26.437923 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:27.061453 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:27.531685 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:27.979943 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:28.381480 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:28.977098 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:29.466758 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:59:29.466468 23798 raft_consensus.cc:493] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ee02f77594864349982492862d668a36)
I20260925 13:59:29.467127 23798 raft_consensus.cc:515] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:59:29.468966 23798 leader_election.cc:290] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 13:59:29.470737 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
W20260925 13:59:29.471238 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
I20260925 13:59:29.471542 21732 leader_election.cc:304] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
I20260925 13:59:29.472100 23798 raft_consensus.cc:2749] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:59:29.969503 23828 raft_consensus.cc:493] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:59:29.970132 23828 raft_consensus.cc:515] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:59:29.972051 23828 leader_election.cc:290] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 13:59:29.973546 22040 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
W20260925 13:59:29.974067 22043 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
I20260925 13:59:29.974380 22043 leader_election.cc:304] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:59:29.975003 23828 raft_consensus.cc:2749] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:59:30.008683 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:30.596066 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:31.058470 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:31.595865 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:32.068830 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:59:32.071625 23963 raft_consensus.cc:493] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:59:32.072059 23963 raft_consensus.cc:515] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:59:32.074064 23963 leader_election.cc:290] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 13:59:32.075556 22040 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
W20260925 13:59:32.075955 22043 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
I20260925 13:59:32.076325 22043 leader_election.cc:304] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:59:32.076959 23963 raft_consensus.cc:2749] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:59:32.560602 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:32.982993 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:33.376679 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:33.986900 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:34.588444 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:35.037809 21896 consensus_peers.cc:597] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee -> Peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Couldn't send request to peer ee02f77594864349982492862d668a36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60256642160a44a2bab77b97d78c27a8. This is attempt 66: this message will repeat every 5th retry.
W20260925 13:59:35.129663 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:35.709015 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:36.183522 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:59:36.382797 23995 raft_consensus.cc:493] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:59:36.383754 23995 raft_consensus.cc:515] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:59:36.386631 23995 leader_election.cc:290] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 13:59:36.387781 22040 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
W20260925 13:59:36.388420 22043 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
I20260925 13:59:36.388803 22043 leader_election.cc:304] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:59:36.389626 23995 raft_consensus.cc:2749] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:59:36.640149 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:37.036173 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:37.528403 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:38.104697 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:38.553095 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:38.935034 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:39.453370 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:39.968478 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:59:40.447237 23984 raft_consensus.cc:493] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ee02f77594864349982492862d668a36)
I20260925 13:59:40.448061 23984 raft_consensus.cc:515] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:59:40.450243 23984 leader_election.cc:290] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 13:59:40.451993 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
W20260925 13:59:40.452629 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
I20260925 13:59:40.452936 21732 leader_election.cc:304] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
I20260925 13:59:40.453598 23984 raft_consensus.cc:2749] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 13:59:40.593127 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:41.031901 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:59:41.324334 24002 raft_consensus.cc:493] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:59:41.324816 24002 raft_consensus.cc:515] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:59:41.326607 24002 leader_election.cc:290] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 13:59:41.328449 22043 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
W20260925 13:59:41.328856 22040 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
I20260925 13:59:41.329185 22040 leader_election.cc:304] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:59:41.329730 24002 raft_consensus.cc:2749] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:59:41.438144 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:42.030751 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:42.551862 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:43.077001 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:43.496037 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:43.950663 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:44.554019 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:44.953245 21896 consensus_peers.cc:597] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee -> Peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Couldn't send request to peer ee02f77594864349982492862d668a36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60256642160a44a2bab77b97d78c27a8. This is attempt 71: this message will repeat every 5th retry.
W20260925 13:59:44.957165 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:45.334456 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:45.918462 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:46.323987 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:59:46.560573 24002 raft_consensus.cc:493] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:59:46.561231 24002 raft_consensus.cc:515] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:59:46.563141 24002 leader_election.cc:290] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 13:59:46.564520 22040 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
W20260925 13:59:46.565084 22043 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
I20260925 13:59:46.565397 22043 leader_election.cc:304] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:59:46.565977 24002 raft_consensus.cc:2749] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:59:46.724727 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:47.174052 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:47.674134 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:48.188094 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:48.695492 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:49.300760 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:49.758416 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:50.377978 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:50.956308 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:51.543309 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:51.974136 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:52.379112 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:59:52.893429 24084 raft_consensus.cc:493] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ee02f77594864349982492862d668a36)
I20260925 13:59:52.894112 24084 raft_consensus.cc:515] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:59:52.896009 24084 leader_election.cc:290] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 13:59:52.897645 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
W20260925 13:59:52.898279 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
I20260925 13:59:52.898571 21732 leader_election.cc:304] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
I20260925 13:59:52.899154 24084 raft_consensus.cc:2749] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 13:59:52.999718 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:53.332418 21619 debug-util.cc:398] Leaking SignalData structure 0x7b080008a480 after lost signal to thread 21550
W20260925 13:59:53.333105 21619 debug-util.cc:398] Leaking SignalData structure 0x7b0800099720 after lost signal to thread 21622
W20260925 13:59:53.570406 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:53.948235 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:54.407893 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:54.909205 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:59:55.342046 24002 raft_consensus.cc:493] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:59:55.342474 24002 raft_consensus.cc:515] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:59:55.344717 24002 leader_election.cc:290] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 13:59:55.346122 22040 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
W20260925 13:59:55.346576 22043 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
I20260925 13:59:55.347002 22043 leader_election.cc:304] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:59:55.347571 24002 raft_consensus.cc:2749] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:59:55.378459 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 13:59:55.499446 24002 raft_consensus.cc:493] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:59:55.499831 24002 raft_consensus.cc:515] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 13:59:55.501612 24002 leader_election.cc:290] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 13:59:55.502861 22040 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
W20260925 13:59:55.503296 22043 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
I20260925 13:59:55.503676 22043 leader_election.cc:304] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 13:59:55.504251 24002 raft_consensus.cc:2749] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 13:59:55.590806 21896 consensus_peers.cc:597] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee -> Peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Couldn't send request to peer ee02f77594864349982492862d668a36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60256642160a44a2bab77b97d78c27a8. This is attempt 76: this message will repeat every 5th retry.
W20260925 13:59:55.968901 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:56.543242 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:57.156840 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:57.731305 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:58.301174 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:58.740955 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:59.208500 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 13:59:59.632192 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:00.008584 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:00.479226 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:00.892922 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:01.431715 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:01.887828 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:02.314049 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:02.907285 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:03.459589 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:03.901357 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:04.317329 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:04.833062 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 14:00:05.255501 24253 raft_consensus.cc:493] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:00:05.256040 24253 raft_consensus.cc:515] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:00:05.257737 24253 leader_election.cc:290] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 14:00:05.259466 22040 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
W20260925 14:00:05.259943 22043 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
I20260925 14:00:05.260215 22043 leader_election.cc:304] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 14:00:05.260759 24253 raft_consensus.cc:2749] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 14:00:05.306386 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:05.830480 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:06.230969 21896 consensus_peers.cc:597] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee -> Peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Couldn't send request to peer ee02f77594864349982492862d668a36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60256642160a44a2bab77b97d78c27a8. This is attempt 81: this message will repeat every 5th retry.
W20260925 14:00:06.304170 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:06.686434 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:07.256740 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:07.323326 21838 debug-util.cc:398] Leaking SignalData structure 0x7b08000e7e80 after lost signal to thread 21681
W20260925 14:00:07.324371 21838 debug-util.cc:398] Leaking SignalData structure 0x7b08001133e0 after lost signal to thread 21841
W20260925 14:00:07.771451 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:08.364424 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:08.543067 21848 heartbeater.cc:646] Failed to heartbeat to 127.18.182.254:43163 (312 consecutive failures): Network error: Failed to ping master at 127.18.182.254:43163: Client connection negotiation failed: client connection to 127.18.182.254:43163: connect: Connection refused (error 111)
W20260925 14:00:08.715113 22009 heartbeater.cc:646] Failed to heartbeat to 127.18.182.254:43163 (393 consecutive failures): Network error: Failed to ping master at 127.18.182.254:43163: Client connection negotiation failed: client connection to 127.18.182.254:43163: connect: Connection refused (error 111)
W20260925 14:00:08.783102 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:08.800357 22154 heartbeater.cc:646] Failed to heartbeat to 127.18.182.254:43163 (314 consecutive failures): Network error: Failed to ping master at 127.18.182.254:43163: Client connection negotiation failed: client connection to 127.18.182.254:43163: connect: Connection refused (error 111)
W20260925 14:00:09.240974 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:09.716571 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 14:00:09.766824 24277 raft_consensus.cc:493] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:00:09.767442 24277 raft_consensus.cc:515] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:00:09.769552 24277 leader_election.cc:290] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 14:00:09.770936 22040 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
W20260925 14:00:09.771814 22043 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
I20260925 14:00:09.772099 22043 leader_election.cc:304] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 14:00:09.772657 24277 raft_consensus.cc:2749] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 14:00:10.206514 24084 raft_consensus.cc:493] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ee02f77594864349982492862d668a36)
I20260925 14:00:10.207155 24084 raft_consensus.cc:515] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:00:10.208935 24084 leader_election.cc:290] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 14:00:10.210568 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
W20260925 14:00:10.211225 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
I20260925 14:00:10.211576 21732 leader_election.cc:304] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
I20260925 14:00:10.212178 24084 raft_consensus.cc:2749] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 14:00:10.312845 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:10.744527 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:11.362844 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:11.782622 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:12.178777 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:12.560674 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:13.093674 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:13.528117 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 14:00:13.675642 24338 raft_consensus.cc:493] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:00:13.676087 24338 raft_consensus.cc:515] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:00:13.677883 24338 leader_election.cc:290] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 14:00:13.679463 22043 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
W20260925 14:00:13.679829 22040 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
I20260925 14:00:13.680084 22040 leader_election.cc:304] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 14:00:13.680611 24338 raft_consensus.cc:2749] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 14:00:14.051192 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:14.628305 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:15.129225 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:15.679313 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:16.115094 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:16.456279 21896 consensus_peers.cc:597] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee -> Peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Couldn't send request to peer ee02f77594864349982492862d668a36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60256642160a44a2bab77b97d78c27a8. This is attempt 86: this message will repeat every 5th retry.
W20260925 14:00:16.561319 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:16.971874 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:17.075331 21575 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 5eb86772ca2e4306a5c164342514b7b0 (table default.table_d6100b0f2b294ab099c179a7681cb4ed [id=74ee67384ecf4220b7d08fdf0637aefb], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 5eb86772ca2e4306a5c164342514b7b0 consensus state has no leader
W20260925 14:00:17.596402 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:18.085789 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:18.590284 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:18.988605 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:19.470734 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:19.900305 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:20.378332 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:20.872640 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 14:00:21.104208 24402 raft_consensus.cc:493] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:00:21.104866 24402 raft_consensus.cc:515] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:00:21.106842 24402 leader_election.cc:290] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 14:00:21.108199 22040 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
W20260925 14:00:21.108732 22043 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
I20260925 14:00:21.109035 22043 leader_election.cc:304] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 14:00:21.109560 24402 raft_consensus.cc:2749] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 14:00:21.399924 24381 raft_consensus.cc:493] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ee02f77594864349982492862d668a36)
I20260925 14:00:21.400982 24381 raft_consensus.cc:515] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:00:21.404498 24381 leader_election.cc:290] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 14:00:21.407282 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
W20260925 14:00:21.408797 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
I20260925 14:00:21.409193 21732 leader_election.cc:304] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
I20260925 14:00:21.409950 24381 raft_consensus.cc:2749] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 14:00:21.456478 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:21.965639 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:22.350337 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:22.858121 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 14:00:23.146535 24402 raft_consensus.cc:493] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:00:23.147022 24402 raft_consensus.cc:515] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:00:23.149066 24402 leader_election.cc:290] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 14:00:23.150755 22040 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
W20260925 14:00:23.151211 22043 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
I20260925 14:00:23.151512 22043 leader_election.cc:304] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 14:00:23.152007 24402 raft_consensus.cc:2749] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 14:00:23.480587 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:24.024459 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:24.460063 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:24.911643 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:25.458541 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:25.787308 21896 consensus_peers.cc:597] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee -> Peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Couldn't send request to peer ee02f77594864349982492862d668a36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60256642160a44a2bab77b97d78c27a8. This is attempt 91: this message will repeat every 5th retry.
W20260925 14:00:25.841459 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:26.463404 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:26.941339 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:27.389925 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:28.012797 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:28.444128 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:28.848048 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:29.443179 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:30.025218 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:30.559490 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:30.989058 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:31.581377 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:32.094175 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:32.687098 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:33.074406 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:33.603109 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:34.099758 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:34.500100 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:34.907464 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:35.425751 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:35.819116 21896 consensus_peers.cc:597] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee -> Peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Couldn't send request to peer ee02f77594864349982492862d668a36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60256642160a44a2bab77b97d78c27a8. This is attempt 96: this message will repeat every 5th retry.
W20260925 14:00:35.957644 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:36.477254 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:36.814070 22148 debug-util.cc:398] Leaking SignalData structure 0x7b08000c9160 after lost signal to thread 22017
W20260925 14:00:36.816004 22148 debug-util.cc:398] Leaking SignalData structure 0x7b08000d9be0 after lost signal to thread 22151
W20260925 14:00:36.936645 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:37.408262 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 14:00:37.664522 24537 raft_consensus.cc:493] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:00:37.665140 24537 raft_consensus.cc:515] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:00:37.668239 24537 leader_election.cc:290] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 14:00:37.669294 22040 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
W20260925 14:00:37.669795 22043 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
I20260925 14:00:37.670059 22043 leader_election.cc:304] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 14:00:37.670627 24537 raft_consensus.cc:2749] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 14:00:37.767719 24474 raft_consensus.cc:493] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ee02f77594864349982492862d668a36)
I20260925 14:00:37.768376 24474 raft_consensus.cc:515] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:00:37.770965 24474 leader_election.cc:290] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 14:00:37.772379 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
W20260925 14:00:37.772881 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
I20260925 14:00:37.773160 21732 leader_election.cc:304] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
I20260925 14:00:37.773691 24474 raft_consensus.cc:2749] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 14:00:37.894156 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:38.500999 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:39.101117 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:39.537711 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 14:00:39.725330 24537 raft_consensus.cc:493] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:00:39.725843 24537 raft_consensus.cc:515] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:00:39.727746 24537 leader_election.cc:290] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 14:00:39.729050 22040 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
W20260925 14:00:39.729430 22043 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
I20260925 14:00:39.729745 22043 leader_election.cc:304] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 14:00:39.730386 24537 raft_consensus.cc:2749] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 14:00:40.060256 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:40.570310 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:41.061177 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:41.542332 22001 debug-util.cc:398] Leaking SignalData structure 0x7b0800100aa0 after lost signal to thread 21854
W20260925 14:00:41.543215 22001 debug-util.cc:398] Leaking SignalData structure 0x7b08000ec0c0 after lost signal to thread 22004
W20260925 14:00:41.585047 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:42.054438 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:42.483918 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:42.921571 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:43.402407 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:43.861984 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:44.290867 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:44.733574 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:45.337061 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:45.774135 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 14:00:45.899142 24596 raft_consensus.cc:493] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ee02f77594864349982492862d668a36)
I20260925 14:00:45.899874 24596 raft_consensus.cc:515] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:00:45.902448 24596 leader_election.cc:290] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 14:00:45.903585 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
W20260925 14:00:45.904207 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
I20260925 14:00:45.904491 21732 leader_election.cc:304] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
I20260925 14:00:45.905049 24596 raft_consensus.cc:2749] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 14:00:46.168402 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:46.588791 21896 consensus_peers.cc:597] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee -> Peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Couldn't send request to peer ee02f77594864349982492862d668a36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60256642160a44a2bab77b97d78c27a8. This is attempt 101: this message will repeat every 5th retry.
W20260925 14:00:46.707866 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:47.283825 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:47.685415 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:48.310035 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:48.886915 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:49.443099 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:49.848677 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 14:00:49.994601 24537 raft_consensus.cc:493] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:00:49.995236 24537 raft_consensus.cc:515] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:00:49.997248 24537 leader_election.cc:290] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 14:00:49.998838 22040 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
W20260925 14:00:49.999426 22043 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
I20260925 14:00:49.999734 22043 leader_election.cc:304] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 14:00:50.000283 24537 raft_consensus.cc:2749] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 14:00:50.348173 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:50.688668 21619 debug-util.cc:398] Leaking SignalData structure 0x7b0800079fe0 after lost signal to thread 21550
W20260925 14:00:50.689178 21619 debug-util.cc:398] Leaking SignalData structure 0x7b0800070140 after lost signal to thread 21622
W20260925 14:00:50.966557 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:51.455565 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:51.925204 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:52.480206 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:52.997967 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:53.619251 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:54.189844 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:54.705193 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:55.237002 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:55.695762 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:56.286265 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:56.831681 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 14:00:56.940442 24726 raft_consensus.cc:493] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:00:56.940989 24726 raft_consensus.cc:515] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:00:56.943324 24726 leader_election.cc:290] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 14:00:56.944901 22040 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
W20260925 14:00:56.945477 22043 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
I20260925 14:00:56.945843 22043 leader_election.cc:304] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 14:00:56.946493 24726 raft_consensus.cc:2749] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 14:00:57.079567 21896 consensus_peers.cc:597] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee -> Peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Couldn't send request to peer ee02f77594864349982492862d668a36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60256642160a44a2bab77b97d78c27a8. This is attempt 106: this message will repeat every 5th retry.
W20260925 14:00:57.271240 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:57.861794 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:58.474066 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:59.087107 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:59.494207 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:00:59.969257 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 14:01:00.074244 24766 raft_consensus.cc:493] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ee02f77594864349982492862d668a36)
I20260925 14:01:00.074930 24766 raft_consensus.cc:515] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:01:00.077847 24766 leader_election.cc:290] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 14:01:00.079488 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
W20260925 14:01:00.080039 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
I20260925 14:01:00.080313 21732 leader_election.cc:304] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
I20260925 14:01:00.080933 24766 raft_consensus.cc:2749] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 14:01:00.440773 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:00.984969 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 14:01:01.149295 24726 raft_consensus.cc:493] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:01:01.149720 24726 raft_consensus.cc:515] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:01:01.151434 24726 leader_election.cc:290] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 14:01:01.152777 22040 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
W20260925 14:01:01.153311 22043 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
I20260925 14:01:01.153681 22043 leader_election.cc:304] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 14:01:01.154309 24726 raft_consensus.cc:2749] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 14:01:01.407450 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:01.958631 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:02.542605 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:02.945834 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:03.480794 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:03.994297 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:04.606796 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:05.010958 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:05.562849 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:06.022625 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:06.444725 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:06.555663 21896 consensus_peers.cc:597] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee -> Peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Couldn't send request to peer ee02f77594864349982492862d668a36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60256642160a44a2bab77b97d78c27a8. This is attempt 111: this message will repeat every 5th retry.
W20260925 14:01:06.985881 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:07.578465 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:08.173072 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:08.752904 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:08.849260 21843 heartbeater.cc:646] Failed to heartbeat to 127.18.182.253:46177 (428 consecutive failures): Network error: Failed to ping master at 127.18.182.253:46177: Client connection negotiation failed: client connection to 127.18.182.253:46177: connect: Connection refused (error 111)
W20260925 14:01:09.082252 22154 heartbeater.cc:646] Failed to heartbeat to 127.18.182.254:43163 (434 consecutive failures): Network error: Failed to ping master at 127.18.182.254:43163: Client connection negotiation failed: client connection to 127.18.182.254:43163: connect: Connection refused (error 111)
W20260925 14:01:09.147462 22008 heartbeater.cc:646] Failed to heartbeat to 127.18.182.253:46177 (535 consecutive failures): Network error: Failed to ping master at 127.18.182.253:46177: Client connection negotiation failed: client connection to 127.18.182.253:46177: connect: Connection refused (error 111)
W20260925 14:01:09.295673 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 14:01:09.430598 24873 raft_consensus.cc:493] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:01:09.431372 24873 raft_consensus.cc:515] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:01:09.433728 24873 leader_election.cc:290] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 14:01:09.435022 22040 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
W20260925 14:01:09.435632 22043 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
I20260925 14:01:09.436081 22043 leader_election.cc:304] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 14:01:09.436807 24873 raft_consensus.cc:2749] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 14:01:09.904184 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:10.334030 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:10.862227 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:11.449206 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:11.837908 22001 debug-util.cc:398] Leaking SignalData structure 0x7b08000c5220 after lost signal to thread 21854
W20260925 14:01:11.839063 22001 debug-util.cc:398] Leaking SignalData structure 0x7b0800121a60 after lost signal to thread 22004
W20260925 14:01:12.020138 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 14:01:12.460045 24873 raft_consensus.cc:493] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:01:12.460546 24873 raft_consensus.cc:515] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:01:12.462558 24873 leader_election.cc:290] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 14:01:12.464077 22040 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
W20260925 14:01:12.464479 22043 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
I20260925 14:01:12.464776 22043 leader_election.cc:304] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 14:01:12.465289 24873 raft_consensus.cc:2749] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 14:01:12.633184 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:13.064590 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:13.478101 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 14:01:13.818322 24879 raft_consensus.cc:493] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ee02f77594864349982492862d668a36)
I20260925 14:01:13.819041 24879 raft_consensus.cc:515] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:01:13.821146 24879 leader_election.cc:290] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 14:01:13.823071 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
W20260925 14:01:13.823710 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
I20260925 14:01:13.824050 21732 leader_election.cc:304] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
I20260925 14:01:13.824754 24879 raft_consensus.cc:2749] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 14:01:14.033759 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:14.440559 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:14.861435 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:15.380342 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 14:01:15.499414 24928 raft_consensus.cc:493] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:01:15.499909 24928 raft_consensus.cc:515] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:01:15.502214 24928 leader_election.cc:290] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 14:01:15.503965 22040 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
W20260925 14:01:15.504424 22043 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
I20260925 14:01:15.504793 22043 leader_election.cc:304] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 14:01:15.505415 24928 raft_consensus.cc:2749] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 14:01:15.975129 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:16.122205 21896 consensus_peers.cc:597] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee -> Peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Couldn't send request to peer ee02f77594864349982492862d668a36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60256642160a44a2bab77b97d78c27a8. This is attempt 116: this message will repeat every 5th retry.
W20260925 14:01:16.416657 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:17.011925 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:17.078730 21575 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 5eb86772ca2e4306a5c164342514b7b0 (table default.table_d6100b0f2b294ab099c179a7681cb4ed [id=74ee67384ecf4220b7d08fdf0637aefb], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 5eb86772ca2e4306a5c164342514b7b0 consensus state has no leader
W20260925 14:01:17.455215 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:17.848142 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:18.454301 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:19.044559 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:19.577117 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:20.002837 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:20.455760 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:20.875437 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:21.426115 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:22.034646 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:22.518712 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 14:01:22.747076 24879 raft_consensus.cc:493] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ee02f77594864349982492862d668a36)
I20260925 14:01:22.747895 24879 raft_consensus.cc:515] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:01:22.750326 24879 leader_election.cc:290] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 14:01:22.752050 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
W20260925 14:01:22.752595 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
I20260925 14:01:22.752817 21732 leader_election.cc:304] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
I20260925 14:01:22.753304 24879 raft_consensus.cc:2749] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 14:01:22.985071 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 14:01:22.989691 25024 raft_consensus.cc:493] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:01:22.990419 25024 raft_consensus.cc:515] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:01:22.992313 25024 leader_election.cc:290] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 14:01:22.993669 22043 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
W20260925 14:01:22.994239 22040 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
I20260925 14:01:22.994477 22040 leader_election.cc:304] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 14:01:22.994963 25024 raft_consensus.cc:2749] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 14:01:23.596042 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 14:01:23.771708 25024 raft_consensus.cc:493] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:01:23.772145 25024 raft_consensus.cc:515] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:01:23.773766 25024 leader_election.cc:290] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 14:01:23.775278 22040 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
W20260925 14:01:23.775739 22043 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
I20260925 14:01:23.776018 22043 leader_election.cc:304] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 14:01:23.776568 25024 raft_consensus.cc:2749] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 14:01:24.192859 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:24.755569 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:24.988930 21896 consensus_peers.cc:597] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee -> Peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Couldn't send request to peer ee02f77594864349982492862d668a36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60256642160a44a2bab77b97d78c27a8. This is attempt 121: this message will repeat every 5th retry.
W20260925 14:01:25.160662 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:25.552711 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:26.166575 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:26.764529 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:27.152136 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:27.355553 21838 debug-util.cc:398] Leaking SignalData structure 0x7b080003e500 after lost signal to thread 21681
W20260925 14:01:27.356401 21838 debug-util.cc:398] Leaking SignalData structure 0x7b080011bd80 after lost signal to thread 21841
W20260925 14:01:27.691627 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:28.102626 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:28.619381 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:29.091069 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:29.655251 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:30.227123 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 14:01:30.525692 25024 raft_consensus.cc:493] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:01:30.526296 25024 raft_consensus.cc:515] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:01:30.529403 25024 leader_election.cc:290] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 14:01:30.530555 22040 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
W20260925 14:01:30.531203 22043 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
I20260925 14:01:30.531575 22043 leader_election.cc:304] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 14:01:30.532147 25024 raft_consensus.cc:2749] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 14:01:30.661196 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 14:01:30.915171 25089 raft_consensus.cc:493] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ee02f77594864349982492862d668a36)
I20260925 14:01:30.915673 25089 raft_consensus.cc:515] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:01:30.917565 25089 leader_election.cc:290] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 14:01:30.920855 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
W20260925 14:01:30.921422 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
I20260925 14:01:30.921708 21732 leader_election.cc:304] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
I20260925 14:01:30.922302 25089 raft_consensus.cc:2749] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 14:01:31.171205 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:31.583704 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:32.019405 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:32.408039 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:32.810472 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 14:01:32.991417 25024 raft_consensus.cc:493] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:01:32.991901 25024 raft_consensus.cc:515] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:01:32.993762 25024 leader_election.cc:290] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 14:01:32.995218 22040 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
W20260925 14:01:32.995683 22043 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
I20260925 14:01:32.996057 22043 leader_election.cc:304] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 14:01:32.996673 25024 raft_consensus.cc:2749] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 14:01:33.208316 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:33.600893 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:34.013856 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:34.448278 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:34.942102 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:35.458762 21896 consensus_peers.cc:597] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee -> Peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Couldn't send request to peer ee02f77594864349982492862d668a36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60256642160a44a2bab77b97d78c27a8. This is attempt 126: this message will repeat every 5th retry.
W20260925 14:01:35.546329 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:36.101197 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:36.561755 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:37.125607 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:37.566804 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:37.951301 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:38.361945 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 14:01:38.671717 25193 raft_consensus.cc:493] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ee02f77594864349982492862d668a36)
I20260925 14:01:38.672502 25193 raft_consensus.cc:515] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:01:38.674952 25193 leader_election.cc:290] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 14:01:38.676191 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
W20260925 14:01:38.676774 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
I20260925 14:01:38.677023 21732 leader_election.cc:304] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
I20260925 14:01:38.677532 25193 raft_consensus.cc:2749] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 14:01:38.815653 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:39.394395 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:39.959180 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:40.382265 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:41.002705 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:41.382637 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:41.872457 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:42.280507 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:42.714164 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:43.248232 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:43.647768 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:44.239632 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:44.753669 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:45.317749 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:45.430794 21896 consensus_peers.cc:597] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee -> Peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Couldn't send request to peer ee02f77594864349982492862d668a36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60256642160a44a2bab77b97d78c27a8. This is attempt 131: this message will repeat every 5th retry.
W20260925 14:01:45.765239 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:46.214008 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:46.652112 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:47.145241 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 14:01:47.455952 25300 raft_consensus.cc:493] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:01:47.456700 25300 raft_consensus.cc:515] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:01:47.459520 25300 leader_election.cc:290] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 14:01:47.460778 22040 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
W20260925 14:01:47.461268 22043 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
I20260925 14:01:47.461529 22043 leader_election.cc:304] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 14:01:47.462266 25300 raft_consensus.cc:2749] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 14:01:47.666249 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:48.250792 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:48.713685 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:49.221509 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:49.603519 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:50.225685 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 14:01:50.240222 25300 raft_consensus.cc:493] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:01:50.240716 25300 raft_consensus.cc:515] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:01:50.242223 25300 leader_election.cc:290] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 14:01:50.243438 22040 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
W20260925 14:01:50.243894 22043 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
I20260925 14:01:50.244212 22043 leader_election.cc:304] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 14:01:50.244714 25300 raft_consensus.cc:2749] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 14:01:50.756740 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:51.182317 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:51.678555 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:52.298666 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:52.738446 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:53.200809 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:53.610389 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:54.018664 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:54.602698 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:54.955087 21896 consensus_peers.cc:597] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee -> Peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Couldn't send request to peer ee02f77594864349982492862d668a36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60256642160a44a2bab77b97d78c27a8. This is attempt 136: this message will repeat every 5th retry.
W20260925 14:01:55.113804 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:55.725054 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:56.181241 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 14:01:56.749480 25389 raft_consensus.cc:493] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ee02f77594864349982492862d668a36)
I20260925 14:01:56.750182 25389 raft_consensus.cc:515] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:01:56.753005 25389 leader_election.cc:290] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 14:01:56.755244 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
W20260925 14:01:56.755887 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
I20260925 14:01:56.756217 21732 leader_election.cc:304] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
I20260925 14:01:56.756944 25389 raft_consensus.cc:2749] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 14:01:56.790426 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:57.318665 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:57.777531 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 14:01:58.103813 25397 raft_consensus.cc:493] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:01:58.104390 25397 raft_consensus.cc:515] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:01:58.106520 25397 leader_election.cc:290] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 14:01:58.108510 22040 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
W20260925 14:01:58.109091 22043 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
I20260925 14:01:58.109472 22043 leader_election.cc:304] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 14:01:58.110198 25397 raft_consensus.cc:2749] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 14:01:58.267689 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:58.741209 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:59.232949 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:01:59.613409 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:00.219576 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:00.679281 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:01.205888 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:01.733466 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:02.339699 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 14:02:02.420135 25455 raft_consensus.cc:493] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:02:02.421015 25455 raft_consensus.cc:515] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
W20260925 14:02:02.426543 22040 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
I20260925 14:02:02.426769 25455 leader_election.cc:290] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 14:02:02.428627 22043 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
I20260925 14:02:02.428995 22043 leader_election.cc:304] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 14:02:02.429627 25455 raft_consensus.cc:2749] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 14:02:02.933189 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:03.426544 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:04.020895 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 14:02:04.392474 25496 raft_consensus.cc:493] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:02:04.393363 25496 raft_consensus.cc:515] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:02:04.396283 25496 leader_election.cc:290] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 14:02:04.397889 22040 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
W20260925 14:02:04.398867 22043 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
I20260925 14:02:04.399174 22043 leader_election.cc:304] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 14:02:04.399801 25496 raft_consensus.cc:2749] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 14:02:04.474529 21896 consensus_peers.cc:597] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee -> Peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Couldn't send request to peer ee02f77594864349982492862d668a36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60256642160a44a2bab77b97d78c27a8. This is attempt 141: this message will repeat every 5th retry.
W20260925 14:02:04.633023 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:05.194761 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:05.718751 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:06.124881 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:06.692708 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:07.291208 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:07.902722 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 14:02:08.331915 25516 raft_consensus.cc:493] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:02:08.332535 25516 raft_consensus.cc:515] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:02:08.334723 25516 leader_election.cc:290] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 14:02:08.336225 22040 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
W20260925 14:02:08.337175 22043 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
I20260925 14:02:08.337450 22043 leader_election.cc:304] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 14:02:08.338063 25516 raft_consensus.cc:2749] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 14:02:08.468531 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:09.017935 22148 debug-util.cc:398] Leaking SignalData structure 0x7b0800101b60 after lost signal to thread 22017
W20260925 14:02:09.019227 22148 debug-util.cc:398] Leaking SignalData structure 0x7b08000981c0 after lost signal to thread 22151
W20260925 14:02:09.042961 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:09.267448 21848 heartbeater.cc:646] Failed to heartbeat to 127.18.182.254:43163 (552 consecutive failures): Network error: Failed to ping master at 127.18.182.254:43163: Client connection negotiation failed: client connection to 127.18.182.254:43163: connect: Connection refused (error 111)
W20260925 14:02:09.280805 22008 heartbeater.cc:646] Failed to heartbeat to 127.18.182.253:46177 (682 consecutive failures): Network error: Failed to ping master at 127.18.182.253:46177: Client connection negotiation failed: client connection to 127.18.182.253:46177: connect: Connection refused (error 111)
W20260925 14:02:09.372490 22154 heartbeater.cc:646] Failed to heartbeat to 127.18.182.254:43163 (554 consecutive failures): Network error: Failed to ping master at 127.18.182.254:43163: Client connection negotiation failed: client connection to 127.18.182.254:43163: connect: Connection refused (error 111)
W20260925 14:02:09.519681 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:10.020099 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:10.527729 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:10.829409 22001 debug-util.cc:398] Leaking SignalData structure 0x7b08000dab80 after lost signal to thread 21854
W20260925 14:02:10.830430 22001 debug-util.cc:398] Leaking SignalData structure 0x7b08000ddfc0 after lost signal to thread 22004
W20260925 14:02:10.832096 21619 debug-util.cc:398] Leaking SignalData structure 0x7b080003c2a0 after lost signal to thread 21550
W20260925 14:02:10.832760 21619 debug-util.cc:398] Leaking SignalData structure 0x7b080004b620 after lost signal to thread 21622
W20260925 14:02:10.916249 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:11.297887 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:11.844062 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:12.360679 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:12.959044 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:13.407811 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 14:02:13.508002 25516 raft_consensus.cc:493] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:02:13.508479 25516 raft_consensus.cc:515] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:02:13.510430 25516 leader_election.cc:290] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 14:02:13.512140 22040 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
W20260925 14:02:13.512683 22043 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
I20260925 14:02:13.512979 22043 leader_election.cc:304] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 14:02:13.513499 25516 raft_consensus.cc:2749] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 14:02:13.800312 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:14.412811 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:14.733634 21896 consensus_peers.cc:597] T 60256642160a44a2bab77b97d78c27a8 P 145e520e0a5a445d943a53cf8c27efee -> Peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Couldn't send request to peer ee02f77594864349982492862d668a36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60256642160a44a2bab77b97d78c27a8. This is attempt 146: this message will repeat every 5th retry.
W20260925 14:02:14.985571 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:15.443291 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 14:02:15.557857 25555 raft_consensus.cc:493] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ee02f77594864349982492862d668a36)
I20260925 14:02:15.558636 25555 raft_consensus.cc:515] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:02:15.561188 25555 leader_election.cc:290] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 14:02:15.562801 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
W20260925 14:02:15.563472 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
I20260925 14:02:15.563733 21732 leader_election.cc:304] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
I20260925 14:02:15.564343 25555 raft_consensus.cc:2749] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 14:02:15.845719 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:16.227133 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:16.790966 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:17.097535 21575 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 5eb86772ca2e4306a5c164342514b7b0 (table default.table_d6100b0f2b294ab099c179a7681cb4ed [id=74ee67384ecf4220b7d08fdf0637aefb], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 5eb86772ca2e4306a5c164342514b7b0 consensus state has no leader
W20260925 14:02:17.294253 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:17.837816 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 14:02:18.327914 25639 raft_consensus.cc:493] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:02:18.328652 25639 raft_consensus.cc:515] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:02:18.331393 25639 leader_election.cc:290] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 14:02:18.332779 22043 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
W20260925 14:02:18.334218 22040 leader_election.cc:343] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: 71ff569d68154b61bcc14937b2e37fe5
I20260925 14:02:18.334514 22040 leader_election.cc:304] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 14:02:18.335085 25639 raft_consensus.cc:2749] T 71ff569d68154b61bcc14937b2e37fe5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 14:02:18.362470 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:18.925500 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:19.384089 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:19.878371 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:20.418229 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:21.015515 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 14:02:21.394672 25639 raft_consensus.cc:493] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:02:21.395282 25639 raft_consensus.cc:515] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:02:21.397866 25639 leader_election.cc:290] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881), 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291)
W20260925 14:02:21.399166 22040 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e2731975dbf433aade4a859df9699e2 (127.18.182.193:39881): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
W20260925 14:02:21.399994 22043 leader_election.cc:343] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: ccefd36f4e2a48f9b3d66cb147a022b5
I20260925 14:02:21.400316 22043 leader_election.cc:304] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee02f77594864349982492862d668a36; no voters: 145e520e0a5a445d943a53cf8c27efee, 5e2731975dbf433aade4a859df9699e2
I20260925 14:02:21.400985 25639 raft_consensus.cc:2749] T ccefd36f4e2a48f9b3d66cb147a022b5 P ee02f77594864349982492862d668a36 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 14:02:21.623474 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:22.013731 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:22.417203 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:23.013437 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260925 14:02:23.056046 25658 raft_consensus.cc:493] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ee02f77594864349982492862d668a36)
I20260925 14:02:23.056831 25658 raft_consensus.cc:515] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "145e520e0a5a445d943a53cf8c27efee" member_type: VOTER last_known_addr { host: "127.18.182.194" port: 44291 } } peers { permanent_uuid: "5e2731975dbf433aade4a859df9699e2" member_type: VOTER last_known_addr { host: "127.18.182.193" port: 39881 } } peers { permanent_uuid: "ee02f77594864349982492862d668a36" member_type: VOTER last_known_addr { host: "127.18.182.195" port: 40873 } }
I20260925 14:02:23.059130 25658 leader_election.cc:290] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291), ee02f77594864349982492862d668a36 (127.18.182.195:40873)
W20260925 14:02:23.060927 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 145e520e0a5a445d943a53cf8c27efee (127.18.182.194:44291): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
W20260925 14:02:23.061431 21732 leader_election.cc:343] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee02f77594864349982492862d668a36 (127.18.182.195:40873): Not found: Tablet not found: f5bd96a2bdb842e9b3bc1f706786d6b1
I20260925 14:02:23.061717 21732 leader_election.cc:304] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e2731975dbf433aade4a859df9699e2; no voters: 145e520e0a5a445d943a53cf8c27efee, ee02f77594864349982492862d668a36
I20260925 14:02:23.062407 25658 raft_consensus.cc:2749] T f5bd96a2bdb842e9b3bc1f706786d6b1 P 5e2731975dbf433aade4a859df9699e2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 14:02:23.584668 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260925 14:02:23.773370 21619 debug-util.cc:398] Leaking SignalData structure 0x7b080004b9a0 after lost signal to thread 21550
W20260925 14:02:23.773936 21619 debug-util.cc:398] Leaking SignalData structure 0x7b0800077b00 after lost signal to thread 21622
W20260925 14:02:24.004426 22163 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.18.182.254:43163,127.18.182.253:46177,127.18.182.252:39283, num_attempts: 754) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.18.182.253:46177: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
F20260925 14:02:24.023732 22172 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8229f26c3e104bf1885bce04e09f8e29 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.18.182.254:43163,127.18.182.253:46177,127.18.182.252:39283, num_attempts: 754) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.18.182.253:46177: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
*** Check failure stack trace: ***
*** Aborted at 1790344944 (unix time) try "date -d @1790344944" if you are using GNU date ***
PC: @ 0x0 (unknown)
*** SIGABRT (@0x3e800004adb) received by PID 19163 (TID 0x7fb2cc8fb700) from PID 19163; stack trace: ***
@ 0x2b0620 __tsan::CallUserSignalHandler() at /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1915
@ 0x2b2b03 rtl_sigaction() at /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:2005
@ 0x7fb2df227980 (unknown) at ??:0
@ 0x7fb2da3d1e87 gsignal at ??:0
@ 0x7fb2da3d37f1 abort at ??:0
@ 0x2b6416 __interceptor_abort at /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:263
@ 0x7fb2dd46fd74 google::LogMessage::SendToLog() at ??:0
@ 0x7fb2dd470920 google::LogMessage::Flush() at ??:0
@ 0x7fb2dd475a5b google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x35e444 kudu::MasterStressTest::CreateTableThread() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/master-stress-test.cc:?
@ 0x38bc3e _ZNSt3__18__invokeIMN4kudu16MasterStressTestEFvvEPNS1_26MasterStressTest_Test_TestEJEvEEDTcldsdeclsr3std3__1E7forwardIT0_Efp0_Efp_spclsr3std3__1E7forwardIT1_Efp1_EEEOT_OS7_DpOS8_ at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3846
@ 0x38baf1 _ZNSt3__116__thread_executeINS_10unique_ptrINS_15__thread_structENS_14default_deleteIS2_EEEEMN4kudu16MasterStressTestEFvvEJPNS6_26MasterStressTest_Test_TestEEJLm2EEEEvRNS_5tupleIJT_T0_DpT1_EEENS_15__tuple_indicesIJXspT2_EEEE at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/thread:281
@ 0x38b23b std::__1::__thread_proxy<>() at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/thread:?
@ 0x2b04e0 __tsan_thread_start_func at /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:959
@ 0x7fb2df21c6db start_thread at ??:0
@ 0x7fb2da4b461f clone at ??:0
W20260925 14:02:24.122885 21573 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed